Install Hermes 3 - a model that actually does tool-use

Stock Llama models can do tool-use in principle, but they're not specifically trained for it. Hermes 3 by NousResearch is a Llama-3.1 fine-tune optimized for function calling, JSON mode, and structured output. That makes it a much better choice if you want to build agents that reliably invoke tools locally.

In this tutorial:

• Pull Hermes 3 via Ollama
• See the difference vs. stock Llama in tool-call behavior
• Try a tool definition through the OpenAI-compatible endpoint

Step 1 - Pull Hermes 3

NousResearch ships several sizes. To get started, the 8B variant fits any reasonably modern machine.

Download size around 4.7 GB at the default quantization (q4_K_M). If RAM is tight:

Step 2 - Quick terminal sanity check

Confirm it runs:

Hermes is noticeably disciplined about following structural instructions - exactly the property you want for tool-calling.

Step 3 - Tool definition over the OpenAI-compatible API

In addition to its native API, Ollama also speaks OpenAI Chat Completions on localhost:11434/v1. That means any OpenAI SDK works straight out of the box.

Quick test with curl:

In the response you'll see a tool_calls array instead of a text body - Hermes recognized it needs the weather tool and returned the function call with arguments:

Step 4 - Set Hermes as the Playground default

In your .env (or .env.local):

When you run npm run dev, the Playground will now talk only to your local Hermes model. No token limits, no cloud latency, everything stays on-device.

When to use which model?