JetBrains Mellum2.1 is a 12B coding model that learned to use a terminal
The new Mellum is Apache 2.0, downloadable today, and went from 2 percent to 47 percent on SWE-bench Verified in one release. It still trails a smaller Qwen on agent tasks.
- Model
- Mellum2.1 Thinking
- Maker
- JetBrains
- Size
- 12B total, 2.5B active
- Context
- 131K
- License
- Apache 2.0
- Weights
- Downloadable →
Of the three open-weight models in this week’s coverage, this is the one you can actually download. JetBrains has published Mellum2.1 Thinking on Hugging Face under Apache 2.0. Release trackers date it to October 8; the model card itself is undated.
The previous version barely registered on agent benchmarks. This one is built for them: JetBrains says to use it for working in a repository, running commands, and calling tools.
How it works
Mellum2.1 is a mixture-of-experts model with 12 billion parameters. It has 64 expert sub-networks and routes each token through 8 of them, so only about 2.5 billion parameters do work at any moment.
In principle that means it needs the memory of a 12B model but does the per-token work of a much smaller one. The context window is 131,072 tokens.
JetBrains calls it a thinking model, and the serving instructions on the card include a reasoning parser. The quickstart allows up to 81,920 output tokens, which leaves a lot of room for working things out before answering.
The benchmarks
These are from JetBrains’ model card. Every number is self-reported. Higher is better.
| Benchmark | Mellum2.1 | Mellum2 | Gemma 4 E4B | Qwen3.5 9B |
|---|---|---|---|---|
| LiveCodeBench v6 | 82.0 | 69.4 | 69.4 | 75.4 |
| HumanEval+ | 91.5 | 90.9 | 89.1 | 89.6 |
| MBPP+ | 79.4 | 75.4 | 70.9 | 69.8 |
| SWE-bench Verified | 47.0 | 2.0 | 23.0 | 50.0 |
| SWE-bench Pro | 28.0 | 0.0 | 4.0 | 38.0 |
| Terminal-Bench 2.1 | 17.4 | 0.6 | 3.4 | 21.7 |
| BFCL v4 (tool calling) | 62.3 | 49.6 | 52.5 | 58.5 |
| AIME 25/26 | 83.3 | 60.1 | 45.0 | 86.7 |
| GPQA Diamond | 64.6 | 51.0 | 53.1 | 77.8 |
Two things stand out.
First, the agent rows. Mellum2 scored 2.0 on SWE-bench Verified and 0.0 on SWE-bench Pro. Mellum2.1 scores 47.0 and 28.0. That is not an incremental improvement; the earlier model could not do the task at all.
Second, the last column. Qwen3.5 9B still beats Mellum2.1 on all three agent benchmarks, and by a wide margin on science questions. Mellum2.1 leads on the pure coding tests and on the BFCL tool-calling test.
Read the footnotes
JetBrains ran every model through the same setup: greedy decoding for the non-agent tests, and for the agent tests an open-source harness called Pi with shell and file tools, a 114K-token context, and up to 16K tokens per turn. Running rivals through your own pipeline is fair, but it means the numbers will not match what those rivals publish.
They do not. JetBrains measured Qwen3.5 9B at 75.4 on LiveCodeBench v6 and 77.8 on GPQA Diamond. Qwen’s own model card reports 65.6 and 81.7. One of those is about ten points higher than the vendor claims for its own model, the other about four points lower. Nobody is necessarily wrong. It is a reminder that a benchmark score is a property of the model and the harness together.
Try it yourself
The weights are on Hugging Face in BF16, and the card links 16 community quantizations. Official GGUF builds for llama.cpp, Ollama, and LM Studio are listed as coming soon. We have not run it ourselves: this is based on the two model cards.