Llama 3.2 3B review: what a 3B model can and can't do
Llama 3.2 3B (llama-3.2-3b-instruct) can't be trusted to write or decide on its own. In our five-task test only the summary came out usable; the rest ranged from awkward to wrong. What it does well is answer fast and fill a JSON Schema reliably, so it fits light extraction and classification where something downstream checks its output.
Specs
Meta released Llama 3.2 3B Instruct in September 2024 as a small open model.
- Parameters: 3B
- Context: about 131K tokens
- Input and output: text only
- No tool calling; structured output with JSON Schema works
It is billed per million input and output tokens; current prices are on the model page.
Five writing and reasoning tasks
We sent the same five tasks, all written in Japanese, to several models.
| Task | Result |
|---|---|
| Apology email for a late delivery | Doubled the customer honorific and gave an incoherent explanation of the cause |
| Three-point summary | Picked up the main points, with a few wrong word choices |
| Fix two honorific errors | Wrong; it did not recognize the incorrect form as an error |
| Math word problem | Wrong, and its answer (24 oranges and 10 apples) broke the problem's own total of 15 fruit |
| Haiku with a seasonal word | Wrote no haiku; it explained the theme's words as if they were seasonal words |
The word problem was built so that no integer solution exists, and the right answer is to say so. Llama 3.2 3B instead slipped an unsupported assumption into its algebra and returned, with full confidence, an answer that contradicts the conditions it was given.
On the plus side, it answered in 1 to 3 seconds, and on summarizing, where the job is mostly lifting words from the source, it did reasonably well.
Using it in production
- Check its output with a person or another model. Don't let it make the final call.
- Give it a JSON Schema. Structured output came back in exactly the requested shape, which is far easier to handle than free text.
- Plan for bursts to fail. Only a few hosts serve this model, so a large number of concurrent requests can briefly get 429 responses. Retry with backoff.
Good fit, poor fit
Good fit
- Short classification and tagging of English text
- Simple extraction with the output format fixed by a JSON Schema
- Preprocessing in a pipeline that validates the result later
Poor fit
- Writing or proofreading Japanese text
- Arithmetic and multi-step reasoning
- Agents that depend on tool calls
FAQ
Q. Should I pick Llama 3.2 3B or Llama 3.1 8B?
If you need tool calls, Llama 3.1 8B; the 3B model does not support them. Neither is a good writer in Japanese, so for Japanese-heavy work choose a larger model.
Q. Does Llama 3.2 3B support structured output?
Yes. With a JSON Schema in response_format, it returned JSON with exactly the requested keys.
Q. Why does Llama 3.2 3B return 429 errors?
The upstream hosts serving it have limited capacity. Wait a moment and retry.
Try it on FastMetal
curl https://api.fastmetal.ai/v1/chat/completions \
-H "Authorization: Bearer $FASTMETAL_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "llama-3.2-3b-instruct",
"max_tokens": 200,
"messages": [{"role": "user", "content": "Extract the city and population from: Tokyo has about 14 million residents."}],
"response_format": {
"type": "json_schema",
"json_schema": {
"name": "city",
"strict": true,
"schema": {
"type": "object",
"properties": {"city": {"type": "string"}, "population_millions": {"type": "number"}},
"required": ["city", "population_millions"],
"additionalProperties": false
}
}
}
}'
The same API key and balance work across the catalog, so you can change model and compare. Prices are on the pricing page and every model is in the model catalog.