Llama 3.1 8B review: weak writer, still useful for cheap bulk work
Llama 3.1 8B (llama-3.1-8b-instruct) is the wrong model for writing in Japanese: in our five-task test only the summary was usable. It is cheap and fast, though, and it handles tool calls correctly. Keep it to English text, short classification and fixed-format extraction, and it earns its place.
Specs
Meta released Llama 3.1 8B Instruct in July 2024 as an open model.
- Parameters: 8B
- Context: about 131K tokens
- Input and output: text only
- Tool calling supported
It is billed per million input and output tokens; current prices are on the model page.
Five writing and reasoning tasks
We sent the same five tasks, all written in Japanese, to several models.
| Task | Result |
|---|---|
| Apology email for a late delivery | Stilted from the first line, with no greeting and no addressee |
| Three-point summary | Practical, built from the source's own wording |
| Fix two honorific errors | Wrong; its corrected sentence did not make sense |
| Math word problem | Repeated the same sentence until it hit the 4,000-token output limit |
| Haiku with a seasonal word | Not a haiku, and it put the seasonal word in the wrong season (shigure, an early-winter rain, is a winter word) |
The math result is the one to plan for. The model fell into a loop, writing the same sentence about subtracting the total number of fruit from the total number of fruit over and over until it reached the 4,000-token cap. That call took about 55 seconds, and every one of those output tokens is billed.
A modest max_tokens limits the damage from a loop like this. With small models, always set one.
Why it is still worth having
- Cheap. It was in the lowest price band of the models we compared; see the model page.
- Fast. Apart from the one looping call, answers came back in about 2 seconds.
- Summaries and extraction hold up. Work that reuses the source's own words rarely breaks.
- Tool calls work. It chose the right function and produced valid arguments.
Its strength is reading Japanese and processing English at volume, while writing Japanese is where it falls down.
Good fit, poor fit
Good fit
- Classifying, summarizing and extracting from English text
- Short extractions from Japanese documents that reuse the source's wording
- High-volume, lightweight preprocessing and routing
Poor fit
- Writing Japanese email, prose or copy
- Judging or correcting honorifics
- Arithmetic and reasoning, where loops and wrong answers appeared
FAQ
Q. Can I use Llama 3.1 8B for Japanese?
For reading-heavy work, yes. For writing Japanese text, choose a larger model.
Q. What do I do if Llama 3.1 8B won't stop generating?
Set max_tokens. Everything up to the limit is billed, which matters most with small models.
Q. Does Llama 3.1 8B support tool calls?
Yes. Given tools, it returned the function name and its arguments.
Try it on FastMetal
curl https://api.fastmetal.ai/v1/chat/completions \
-H "Authorization: Bearer $FASTMETAL_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "llama-3.1-8b-instruct",
"max_tokens": 300,
"messages": [{"role": "user", "content": "Classify the sentiment of this review as positive, negative or neutral. Answer with one word. Review: The battery lasts two days, but the screen scratches easily."}]
}'
The same API key and balance work across the catalog, so you can change model and compare. Prices are on the pricing page and every model is in the model catalog.