LLM-jp-3.1 review: Japan's fully open MoE model, from NII

llm-jp-3.1-8x13b-instruct4 is a fully open Japanese model whose instruction following is good enough to beat GPT-4-0613 on Japanese MT-Bench, and that is a strong result for an open model of its size. It trails the big international models on code and math. If your workload is Japanese text (writing, summarizing, Q&A, humanities-style questions), it is worth a serious look.

What it is

LLM-jp-3.1 comes from the Research and Development Center for Large Language Models (LLMC) at Japan's National Institute of Informatics (NII). It refines the LLM-jp-3 series that started in September 2024, adding mid-training and better post-training to lift instruction following.

llm-jp-3.1-8x13b-instruct4 is the flagship: a Mixture-of-Experts model with 73B total parameters and 22B active per token. The standard version has a 4,096-token context window. NII also publishes a 32K variant, llm-jp-3.1-8x13b-32K-instruct4, which FastMetal does not serve.

Training, as NII describes it:

• Starts from a checkpoint pretrained on 2.1T tokens built on llm-jp-corpus v3
• Adds 400B tokens of continued training that mixes about 90B tokens of instruction-response pairs into pretraining data, for 2.5T tokens in total
• Fine-tuned in two stages, SFT then DPO

The license is Apache 2.0, so commercial use and modification are unrestricted.

Strengths

Japanese instruction following

It beats GPT-4-0613 on Japanese MT-Bench, with its best scores in the humanities category. Expect steady output on Japanese writing, summaries and Q&A.

A clear jump over the previous generation

The improvement shows across the whole series. NII's 1.8B model, for example, went from 4.64 to 6.30 on Japanese MT-Bench, which is the mid-training paying off.

Fully open

Training data, code and weights are all public. That matters where provenance has to be explained, such as government, medical or legal pilots.

MoE efficiency

With 22B of its 73B parameters active per token, it is lighter on memory than a dense model of similar total size.

Weak spots

• Code and math: NII's own release notes say it trails OpenAI models and Qwen-based models here. For programming help or numeric analysis, pick something else.
• Context: the standard version stops at 4,096 tokens, which rules out long-document work.
• Languages: it was built for Japanese and English. Other languages can behave unexpectedly.
• Overall quality: it is well behind current frontier models. If quality is the only criterion, compare it against one before you commit.

How it compares

The figures come from different evaluation setups at different times. Treat them as a rough guide.

Only llm-jp-3.1-8x13b-instruct4 from this table is on FastMetal. For a measured comparison against a model we do serve, see the LLM-jp-3.1 vs Claude Haiku 4.5 head-to-head.

Data and cost

On FastMetal the model is served in Japan, from a host whose contract says prompts are never used for training and are not stored, beyond at most a short encrypted hold for maintenance. It is billed per million input and output tokens and, like most open models, is one of the cheaper models in the catalog; the current rate is on the model page.

Where it fits

Good fit:

• Japanese writing, summaries and Q&A
• Tidying and summarizing Japanese business documents and meeting notes
• Prototyping a Japanese chatbot or support assistant
• Pilots that need a Japanese, fully open model (government, medical, legal)
• Automating Japanese text tasks on a tight budget

Poor fit:

• Code generation or code review
• Math-heavy reasoning
• Languages other than Japanese and English
• Production work that has to match frontier quality

FAQ

What is the context window?

4,096 tokens for llm-jp-3.1-8x13b-instruct4. NII publishes a 32K variant for longer documents, but FastMetal does not serve it.

Can I use it commercially?

Yes. It is Apache 2.0, so commercial use, modification and redistribution are allowed. Read the license text against your own use case.

How does it compare with frontier models?

It holds up on everyday Japanese writing and instruction following, but overall it is clearly behind. It makes sense where cost, a Japanese provenance or full openness matters more than peak quality.

Try it on FastMetal

curl https://api.fastmetal.ai/v1/chat/completions \
  -H "Authorization: Bearer $FASTMETAL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llm-jp-3.1-8x13b-instruct4",
    "messages": [{"role": "user", "content": "日本語で自己紹介してください"}]
  }'

The prompt asks it to introduce itself in Japanese. The side-by-side page shows its answers next to Claude Haiku 4.5's on the same prompts, and every other model is in the catalog.

Public benchmarks only get you a shortlist. Run your own prompts before you decide.