Local LLM vs hosted API: small models local, big ones hosted

The first fork in putting an LLM to work is whether to run it yourself or call it over an API. The practical answer is both: run small models locally, and send the ones that won't fit in your memory to a hosted API. Because both sides speak the OpenAI API, the same code works against either one.

Small models: run them locally with Ollama

A model with a few billion parameters runs fine on an ordinary computer. The standard tool is Ollama, which runs on macOS, Linux and Windows.

On Linux, install it with one line:

curl -fsSL https://ollama.com/install.sh | sh

On macOS and Windows, download the app from ollama.com.

Pull a small model and start it:

ollama run llama3.2

Ollama serves an OpenAI-compatible API at http://localhost:11434/v1, so the code you already have can call it:

# Local (Ollama): a small model
curl http://localhost:11434/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model": "llama3.2", "messages": [{"role": "user", "content": "Hello"}]}'

A small model needs a few gigabytes of memory, works offline, and keeps your data on your machine. That makes it a good fit for prototypes and for pre-processing sensitive data.

Big open models hit a memory wall

The trouble is that the strongest open models are no longer small. DeepSeek V4 Pro has 1.6 trillion parameters, Kimi K3 has 2.8 trillion, and GLM-5.2 is in the same class: the open models that do well at coding and agent work now run from hundreds of billions to trillions of parameters.

Serving one takes hundreds of gigabytes of GPU memory. A typical workstation has 16 to 32 GB of RAM and 8 to 24 GB of GPU memory, so these models don't come close to fitting. Open weights don't mean you can run them at home.

Big models: rent them through a hosted API

That's where a hosted API earns its place: you rent the large open model instead of owning the hardware. On FastMetal, the open models you can't run locally are an API call away.

And because FastMetal is OpenAI-compatible too, you change base_url and model and keep everything else:

# FastMetal: a large open model
curl https://api.fastmetal.ai/v1/chat/completions \
  -H "Authorization: Bearer $FASTMETAL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "deepseek-v4-pro", "messages": [{"role": "user", "content": "Hello"}]}'

Swap model for kimi-k3 or glm-5.2, or for a closed frontier model from Anthropic, OpenAI or Google, and the code stays the same.

One codebase, two backends

Since both ends share an API, the split is just configuration: small models on Ollama, large open models and frontier models on FastMetal, chosen by base_url.

• Local (Ollama) fits small models, offline work, and pre-processing data that shouldn't leave your machine.
• Hosted (FastMetal) fits open models too large for your memory, the latest frontier models, and teams that don't want to run GPU infrastructure.

Cost and data

Local inference is mostly a fixed cost: the hardware. A hosted API bills per token, which raises the worry of a bill you can't predict. Prepaid billing removes that. On FastMetal you top up first (from $5 by card), and spend stops at your balance, so there's no surprise invoice at the end of the month.

Bottom line

Small models on Ollama, models too big for your memory on FastMetal. Because both are OpenAI-compatible, you can start with a small local model and point base_url at FastMetal when you need something bigger. The served models are in the model catalog, and prices are on the pricing page.