The same model is published many times over, prepared three different ways. Three questions decide which copy you download.
You want to teach a model your own way of doing something. You look up its name, open the page, and find far more than you expected. Not one download, but a long list of them, many with names that differ by a few characters.
This looks like clutter. It is not. Every copy on that list is the same model, prepared differently for a different purpose. The list is long because there are three separate preparations, and every combination of them gets published.
Once you know what the three are, the list reads as a menu instead of noise.
Question 1: which version
Nearly every model is published twice, and both versions carry almost the same name. Llama-3.1-8B and Llama-3.1-8B-Instruct. Sometimes the ending is -it or -chat instead.
The plain version continues text. That is its entire function. Give it “write a product description for a linen shirt” and a perfectly reasonable output is three more lines that also look like tasks, because it is completing a document that appeared to be a list. It is not misunderstanding you. Nobody taught it that a question is something you answer.
The Instruct version is the same model after a second round of training that taught it exactly that: this shape is a request, this shape is a reply.
Take the Instruct one. You are adding your own way of working on top of something that already understands what being asked means. The plain version means teaching that first, from nothing, and that job takes tens of thousands of examples. Someone has already done it and given away the result.
Question 2: compressed or not
The second preparation is compression. You will see copies with 4-bit in the name, such as unsloth/Llama-3.1-8B-Instruct-bnb-4bit, alongside copies without it.
Same model, compression already applied. For an 8B model the difference is roughly 16 GB down to 4 GB, both to download and to hold in memory while it runs.
If you take the uncompressed copy, your own computer performs the compression as it loads, every time you start. That works. It just spends time and memory redoing something that was already done and published for free.
(The bnb in the name is only the tool that did the compressing. It asks nothing of you.)
Question 3: which packaging
The first two questions are about the model. This one is about the container it comes in, and it is the one that catches people, because both options sit in the same file list with nothing to distinguish their purpose.
.safetensors holds the model in a form that can be opened and changed. That property is what makes teaching possible.
.gguf holds the model in a form built for running quickly, in the applications most people use on their own machine: Ollama, LM Studio, and the rest. Reaching that speed means sealing shut the parts that would have to change during teaching.
So the two files are not alternatives. They belong at opposite ends of the same job.
You teach the .safetensors version. When you are finished, you convert your result to .gguf, and that is what Ollama runs. The .gguf is something you produce, not something you begin with.
This is worth stating plainly because the order most people meet these tools works against it. Ollama is usually where someone first runs a model on their own computer, and Ollama uses .gguf. It is the familiar file. Reaching for it first is a reasonable instinct, and the training software will stop with a message about missing settings that never mentions the file format.
The name carries the answers
Two of the three questions are answered in the name, before you open anything:
unsloth/Llama-3.1-8B-Instruct-bnb-4bit
| Piece | What it tells you |
|---|---|
unsloth |
Who published this particular copy |
Llama-3.1 |
Which model family, and which version of it |
8B |
How large the model is |
Instruct |
Question one, answered |
bnb-4bit |
Question two, answered |
The third question is answered by the file list on that page. That is the whole system.
If something does not work
The messages you get rarely name the real cause. These five cover most of it.
| What you are seeing | What it usually means |
|---|---|
| The software will not load the model and reports missing settings | You took the .gguf. Use the .safetensors files. |
| The model continues from your question instead of answering it | You took the plain version. Use the Instruct one. |
| Very large download, and it runs out of memory on opening | You took the uncompressed copy. Look for 4-bit. |
| Everything finished, but Ollama will not run your result | The final conversion is missing. Convert it to .gguf. |
The files end in .bin rather than .safetensors |
An older copy. It generally works, though a newer one is preferable. |
Before you download anything
Three questions, on the page, before the first byte moves. Is this the Instruct one? Is there a 4-bit copy? Am I taking the .safetensors files rather than the .gguf?
The page will not group them for you or mark which is which. It presents every preparation at once and assumes you know which one you came for. Now you do.