What's inside a model download

You picked an open-weights model, started the download - and you get a folder. A handful of large files, a dozen small ones, 5, 30 or 140 gigabytes together. That's it. No program, no installer, nothing that does anything on its own.

What sits in that folder later decides whether you understand why the same model answers differently on your machine than at the provider. So let's walk through it, file by file.

The folder you get

This is what a real repo looks like - here Qwen/Qwen3-8B, but the shape is the same for almost every Hugging Face model:

Qwen3-8B/
├── config.json                              1 KB
├── generation_config.json                   239 B
├── model-00001-of-00005.safetensors         3.9 GB
├── model-00002-of-00005.safetensors         3.9 GB
├── model-00003-of-00005.safetensors         3.9 GB
├── model-00004-of-00005.safetensors         3.7 GB
├── model-00005-of-00005.safetensors         1.1 GB
├── model.safetensors.index.json             36 KB
├── tokenizer.json                           11 MB
├── tokenizer_config.json                    9 KB
├── vocab.json                               2.7 MB
├── merges.txt                               1.6 MB
├── LICENSE                                  11 KB
└── README.md                                18 KB

Two observations up front. First: almost the entire volume sits in five files, everything else is text. Second: all that remains - a few hundred kilobytes - is exactly what turns a bag of numbers into a usable model.

config.json - what the model is

The blueprint. It holds the architecture, not what was learned:

{
  "architectures": ["Qwen3ForCausalLM"],
  "hidden_size": 4096,
  "num_hidden_layers": 36,
  "num_attention_heads": 32,
  "num_key_value_heads": 8,
  "vocab_size": 151936,
  "max_position_embeddings": 40960,
  "rope_theta": 1000000.0,
  "torch_dtype": "bfloat16"
}

The engine reads this file first. architectures tells it which model type to build at all - if the name is one it doesn't know, loading aborts even though the weights are perfectly fine. That's exactly where the "unsupported architecture" error on brand-new models comes from: the engine is too old for the blueprint, not for the numbers.

The rest are the dimensions: layers (how deep), hidden size (how wide), attention heads and - the most important footnote - num_key_value_heads. Fewer KV heads than attention heads means a smaller KV cache, and therefore more concurrent users on the same card.

vocab_size is the size of the vocabulary, max_position_embeddings the context window it was trained for, and rope_theta belongs to RoPE (rotary position embedding), the mechanism by which the model knows where a token sits at all. That number is what the context-extension tricks turn, the ones that later make 128,000 tokens out of the 40,960 on the spec sheet - that's the second, larger context figure in the model card.

model-*.safetensors - the weights

The actual model: the learned numbers, split into shards of roughly 4 GB each. The split isn't a necessity, it's common sense - an aborted download then costs you one shard instead of 16 GB.

Because the numbers are spread across several files, there has to be a table of contents: model.safetensors.index.json lists, for every weight tensor, which shard it lives in. If that file is missing or one shard didn't make it, nothing loads at all - "all parts or none".

The file name also carries a small security story. These files used to be called pytorch_model.bin and were Python pickle: a format that doesn't just read data on load but may execute arbitrary code. Downloading a "model" therefore meant running a program. safetensors can't do that: a JSON header with the name, type and position of every tensor, raw numbers behind it. Nothing about it is executable - and as a bonus it loads faster, because the file can be mapped straight into memory instead of being unpacked. If you still run into a .bin today, that's legacy, not a feature.

tokenizer.json & friends - the learned vocabulary

The model doesn't read letters, it reads tokens - the LLM call has the essentials on that. Which text fragments it knows is not something the server software could possibly know: it is learned, and therefore ships with the download.

• tokenizer.json - the complete vocabulary plus the splitting rules, in a single file. Around 11 MB, because there are 151,936 entries in it.
• vocab.json and merges.txt - the same information in the older, two-part format (the vocabulary and the learned merge rules of byte-pair encoding). Many repos include both so older tooling can cope.
• tokenizer_config.json - the manual that goes with it: which tokenizer class, how long an input may be, which special tokens exist. And - more on this in a moment - often the chat template as well.

The consequence matters: vocabulary and weights belong together. Token number 8,623 means a particular text fragment for this model, because that's how it learned it. Mix one model's tokenizer with another's weights and you don't get an error - you get gibberish.

special_tokens_map.json - where <|im_start|> comes from

Next to the learned text fragments there is a handful of special tokens that aren't language but control characters: start of turn, end of turn, padding. On our Qwen those are <|im_start|> and <|im_end|> - the same markers that show up in the raw string in the LLM call deep dive.

special_tokens_map.json is the file that defines which token plays which role: eos_token, bos_token, pad_token. It isn't in every repo - Qwen, for instance, writes that mapping directly into tokenizer_config.json, other families put the separate file alongside. Both ways are normal; what matters is that the information sits in the download and not in the server.

Why it counts: the eos_token is the "I'm done" signal. If the engine doesn't know it, or knows the wrong one, the model never stops talking and happily writes the next user question itself.

generation_config.json - the factory setting

A small file with the sampling defaults the vendor recommends for its model:

{
  "temperature": 0.6,
  "top_p": 0.95,
  "top_k": 20,
  "eos_token_id": [151645, 151643]
}

That isn't a rule, it's a suggestion - but a well-founded one, since it comes from the people who post-trained the model. Anyone puzzled why the same model feels differently creative in two tools often finds the answer here: some engines adopt these defaults, others put their own on top.

LICENSE and README.md

The licence says what you're allowed to do with the model - and with open weights that's not a formality, it ranges from "Apache-2.0, do what you like" to bespoke licences with usage limits. The README.md is the model card: description, benchmarks, sample code. How to read it without falling for the marketing is covered in the Models part.

The chat template ships in the folder

Now for the point this chapter is actually about.

The chat template is the stencil that turns your messages array into the single stream of text the model really reads. It's the reason {"role": "user", "content": "Hello"} ends up looking like this:

<|im_start|>user
Hello<|im_end|>
<|im_start|>assistant

The obvious assumption would be that this conversion belongs to the server software - surely vLLM or Ollama must know how to format a chat. It doesn't belong there. The template comes with the model, in one of these two forms:

• as a separate file chat_template.jinja in the folder, or
• as the field "chat_template" inside tokenizer_config.json.

Both spellings are in circulation; the separate file is the newer one and is catching on, because a multi-line template inside a single JSON line is hard to read. Substantively they're identical: a Jinja template, i.e. pure text processing. messages array in, one string out. No weights, no computation, no model - a stencil.

Three things follow that you'll recognise in operation:

• A wrong template turns a good model into a stutterer. The API layer looks completely unchanged while it happens: the same POST /v1/chat/completions, the same fields, valid JSON back. Only the content of the answer is suddenly clipped, repetitive, or never-ending. Anyone tweaking the prompt at this point is looking at the wrong end.
• Swapping the template changes the model's behaviour - without touching a single weight. One extra sentence in the stencil lands in the context on every request. That's a legitimate tool and, if you forget about it, a trap.
• With GGUF the template sits in the file's metadata, not beside it - a .gguf is a container that packs weights, tokenizer and template together. Ollama can additionally override it via a Modelfile. That is one of the most common reasons why "the same" model answers differently in two tools.

Why the same model exists in five sizes

Searching for a model rarely turns up one repo; it turns up a list: the original plus half a dozen derivatives with abbreviations in the name. Behind that is quantisation - the weights are stored more coarsely, 8, 4 or fewer bits per number instead of 16. The model shrinks, and the answers stay surprisingly similar for a surprisingly long time.

All that matters here is the sorting. There isn't one quantisation format, there are several, and they target different hardware:

The formats are not interchangeable: a GGUF doesn't run in vLLM the way an AWQ does, and you won't get an AWQ repo started in Ollama. So when you're staring at a list of repos, sort by format first (does it fit my engine?) and only then by size. What Q4_K_M, IQ3 and Q8_0 mean in detail, and how much quality they cost, is in the Models part.

Who reads what

That lets us answer the opening question. The three usual engines read different parts of this folder - and that's precisely where the behavioural differences come from:

Two rows of this table explain almost every "it behaves differently on my machine" report. For the template it comes down to whether the model's own template is used at all, or a substitute stencil supplied by the tool. And for the sampling defaults, to the fact that the vendor's recommendation is only read by some.

So the folder isn't just a download. It's a small bundle of blueprint, numbers, dictionary and manual - and every engine takes from it whatever it understands.

A model isn't a program, it's a folder: config.json says what it is, the *.safetensors are what it learned, the tokenizer files are its dictionary - and the chat template is part of the download, not part of the server.

It sits in chat_template.jinja or as a field in tokenizer_config.json (with GGUF: in the file's metadata). Swap it and you change the model's behaviour without touching a weight. And because vLLM, llama.cpp and Ollama read different parts of this folder, "the same" model really does answer differently in two tools.