Blog
Kimi K3.1 Pro vs Claude Sonnet 5.5: same price, 2.7x the bill
Kimi K3.1 Pro and Claude Sonnet 5.5 cost the same per token on FastMetal. On 36 identical prompts, Kimi billed 2.7x the output tokens and took a median 150 seconds to Sonnet's 14. The visible answers were about the same length.
MiniMax M3.1 Flash vs DeepSeek V4.1 Flash: same rate, 8x the cost
MiniMax M3.1 Flash and DeepSeek V4.1 Flash bill output at the same rate. On the same 14 prompts, M3.1 Flash thought its way to 8x the output tokens and 8x the cost.
GPT-6.1 Sol vs Claude Sonnet 5.5: same price, 2.1x the output
GPT-6.1 Sol and Claude Sonnet 5.5 cost the same per token on FastMetal. On the same 41 prompts, Sonnet 5.5 wrote 2.1x the output tokens and still answered faster. The extra tokens are mostly annotations.
DeepSeek V4.1 Flash vs GLM 5.3 Flash: same answer, 12x the tokens
We ran DeepSeek V4.1 Flash and GLM 5.3 Flash on the same 43 prompts. The answers were about the same length, but DeepSeek used 1.7x the output tokens on average and 12x on one SQL task. All of it was thinking.
Which model do I trust with eight hundred websites?
I own eight hundred dropped domains and about four hours a week. This is the benchmark I built to decide which language model gets to write on all of them, and the two runs it took to stop measuring my own instrument.
Mercury 2.5 vs Solar Pro 4: is a diffusion LLM really faster?
Mercury 2.5 is a diffusion LLM that generates tokens in parallel. On the same 43 prompts it answered 7.3x faster than Solar Pro 4 on average, while writing about half as much text and using 1.8x the output tokens.
GPT-6 Astra vs Claude Fable 5.1: same price, 2.4x the bill
GPT-6 Astra and Claude Fable 5.1 cost the same per token on FastMetal. On the same 43 prompts, Fable 5.1 used more output tokens on every one, about 2.4x in total, and took twice as long.
Gemini 3.7 Flash vs GPT-5.6 Luna: multimodal or cheapest text
Google's Gemini 3.7 Flash and OpenAI's GPT-5.6 Luna are both cheap, fast models for high-volume work. Luna costs about a third as much; Gemini takes audio and video. Here is how to pick.
Grok 4.6 vs GPT-5.6 Terra: STEM depth or long context
xAI's Grok 4.6 and OpenAI's GPT-5.6 Terra cost about the same to send a prompt to, but Grok's output is half the price and Terra's context is twice the size. Here is how to choose.
GPT-5.6 Terra vs Claude Sonnet 5: which mid-tier model to use
GPT-5.6 Terra and Claude Sonnet 5 are OpenAI's and Anthropic's mid-tier workhorses. Same input price, 1M-token context on both. Sonnet 5 for multi-step agents and coding, Terra for everyday OpenAI-style work.
Showing 1-10 of 29