The LLM Call

We call a language model for the first time: one message out, one answer back - and then the line goes dead. No state, no memory, no tools. That hanging up explains almost everything that follows. And we're not calling just any model: we talk to our own - qwen3.6-35b-fast, running on our inference stack. How to run a model yourself is Block III's topic; for now we simply assume one exists somewhere.

Chat + GPT: who does what here?

The name ChatGPT gives away the division of labour this whole block is about: GPT is the model - it runs on GPUs somewhere and can do exactly one thing: text in, text out. Chat is the software around it - it remembers the conversation, gives the model a role, hands it tools. That software part is what we rebuild here, layer by layer - just not as a chat, but as an agent. The harness is the chat part. The model is rented - or, in our case, self-hosted.

One call, no subscription

Now that the video has shown us that all the magic behind "AI" and ChatGPT actually sits in the LLM, that is what we will look at first. So for now we assume we have access to a language model. How to run language models yourself is the topic of Block III (inference). How to get hold of an API is covered, among other places, in our OpenRouter key tutorial. Let's try calling it: the simplest possible form is a bare web request - curl is enough. The endpoint address (LLM_BASE_URL, in our case a self-hosted vLLM server) and the key (LLM_API_KEY) come from the environment - that is where you put whichever provider or own server you want to talk to. In the call itself we say which model we want to talk to (model), how long the answer may get at most (max_tokens), and what should be said (messages):

curl "$LLM_BASE_URL"/chat/completions \
  -H "Authorization: Bearer $LLM_API_KEY" \
  -H "content-type: application/json" \
  -d '{
    "model": "qwen3.6-35b-fast",
    "max_tokens": 200,
    "messages": [
      { "role": "user", "content": "Explain in 2 sentences what an AI agent is." }
    ]
  }'

What comes back is - again - just JSON, and far more technical than anything ChatGPT ever shows:

{
  "choices": [
    {
      "message": { "role": "assistant", "content": "An AI agent is a program that …" },
      "finish_reason": "stop"
    }
  ],
  "usage": { "prompt_tokens": 23, "completion_tokens": 61 }
}

Three fields determine everything that follows: choices[0].message is the answer. finish_reason says why the model stopped talking - stop means "finished on its own", length means "cut off". And usage counts tokens - the currency inference is measured in, even on your own hardware.

From code, the same call is a few lines of Python or JavaScript - no library required, an HTTP client is enough. In the experiments below you see the raw request and raw response of every run; further down you'll find both as ready-made reference files and as a prompt for your coding agent to build it yourself.

The pen pal with no short-term memory

The most surprising thing about the first call is what does not happen: the model remembers nothing. The second call is a request like any other - the model doesn't know there ever was a first one, even if it happened milliseconds ago. Think of a pen pal who forgets everything between two calls: every time we ring, we have to retell everything that ever happened - and the moment he has answered, we hang up.

An LLM call is nothing magical: JSON in, JSON out. No login, no session, no account balance on the other side - just one request and one response.

And the call is stateless: the next call starts from zero, the voice at the other end remembers nothing - technically it is one huge, clever random-number generator with no memory at all. Everything an agent can do later - conversation, role, tools - the harness has to bring along anew on every call. The brain is not inside the agent. It's at the other end of the line - and over the next components we build everything around it.