In short
- nimble-v3 is a 9B decision model from Bespoke Labs, fine-tuned from Qwen3.5-9B, for classification, routing and scoring — not chat.
nimble-v3 is now on Ollama, and it's an odd one in the best way. It's a 9B model from Bespoke Labs, fine-tuned from Qwen3.5-9B, and it isn't a chatbot. It's a decision model: you give it facts plus questions with defined answers, and it returns probabilities for those answers. No free-form generation, no reasoning trace.
That makes it useful for the jobs people currently fob off onto chat models with a strict prompt and a regex: routing a request to the right handler, checking a condition, assigning a rating. Here's what it is, what it needs, and how to run it.
What nimble-v3 is
Nimble v3 is a 9B decision model from Bespoke Labs, fine-tuned from Qwen3.5-9B (the base is separately Apache 2.0). The idea is simple: instead of asking a model to write an answer and then parsing what it wrote, you ask it questions whose answers are already defined — a set of options, a true/false, a rubric — and it returns probabilities over those answers.
Think of the places you'd use it:
- Routing: which department handles this return? Which tool should this query go to?
- Condition checks: is this item within the return window? Does this ticket match the SLA?
- Scoring: how damaged was the item, on a scale from undamaged to destroyed?
It scores candidate answer tokens directly, so there's no generation step at all. That's where the speed comes from — more on the numbers below.
What's new and different
Three things set nimble-v3 apart from the general-purpose small models most of us run.
Three question types. A request can include:
choice— categorical options, with names mapped to descriptions. You get the selected option plus a probability for each.noul— a Boolean question, optionally with descriptions for true and false. You get a probability from 0 to 1 that the answer is true.score— an ordered rubric: an array of level descriptions in ascending order. You get a probability-weighted level numbered from 0, plus per-level probabilities.
Multiple questions per request. You can evaluate several aspects of the same input in one call — check eligibility, pick a department and rate damage together, in the example on the model page.
32K text context, including the formatted questions — up to 32,768 tokens, enough for a decent chunk of evidence alongside your question definitions.
One change worth flagging: the licence. Nimble v3 is CC BY-NC 4.0 — noncommercial use only — unlike the earlier nimble release. If you're thinking about putting this in a product, read Bespoke Labs' license statement first. For homelab and personal projects it's fine.
Sizes, memory and hardware
The rule of thumb: a model needs roughly its download size in RAM or VRAM, plus room for context. Ollama is explicit that runtime memory also includes caches and working buffers on top of the file size, so treat the download sizes as a floor, not a ceiling.
The default tags (nimble-v3, :latest, :9b) pick MXFP8 via MLX on supported Apple Silicon, or Q8_0 via llama.cpp elsewhere — 9.3–9.5GB downloads. The full variant list:
| Tag | Backend | Weight format | Download size |
|---|---|---|---|
nimble-v3:9b-mxfp8 | MLX | MXFP8 | 9.3 GB |
nimble-v3:9b-nvfp4 | MLX | NVFP4 (selected weights MXFP8) | 6.9 GB |
nimble-v3:9b-mlx-bf16 | MLX | Unquantized BF16 | 17.9 GB |
nimble-v3:9b-q8_0 | llama.cpp | GGUF Q8_0 | 9.5 GB |
nimble-v3:9b-q4_K_M | llama.cpp | GGUF Q4_K_M | 5.6 GB |
nimble-v3:9b-bf16 | llama.cpp | Unquantized GGUF BF16 | 17.9 GB |
As rough guidance — and it is guidance, since your usable memory depends on what else is running:
- The 5.6GB Q4_K_M is the lightest option, and the obvious pick where memory is tight.
- The default 9.3–9.5GB quants suit machines with a comfortable margin above that — a 16GB-class GPU or a well-specced Mac.
- The 17.9GB BF16 variants are for hardware with memory to spare; expect to want something in the 24GB-or-more range to hold the weights plus context comfortably.
So this is a laptop-and-up model in quantised form. Any modern gaming GPU or a decent Apple Silicon Mac can run the default quant; only the unquantized variants ask for serious memory.
How to run it
You need Ollama 0.40.4 or later — earlier versions don't support nimble-v3. If you're setting up from scratch, our Ollama guide covers installation; on the Mac side, MLX is one of the two backends and gets picked automatically on supported Apple Silicon.
ollama pull nimble-v3
ollama run nimble-v3
To pin a specific format:
ollama pull nimble-v3:9b-q4_K_M # smallest, 5.6GB
ollama pull nimble-v3:9b-mxfp8 # Apple Silicon default, 9.3GB
MLX and GGUF packages are both published, with automatic backend selection — you don't need to think about which one you're getting unless you want to.
Calling the API
This is the part most coverage will skip, and it's the bit that matters. You don't chat with nimble-v3; you POST to Ollama's /v1/systemone endpoint. The evidence goes in state, and each question is named in questions. Ollama formats the prompt and returns an answers object keyed by your names.
The example from the model page checks return eligibility (noul), routes to a department (choice) and rates damage (score) in one request:
curl --fail-with-body -sS http://localhost:11434/v1/systemone \
-H 'Content-Type: application/json' \
-d '{
"model": "nimble-v3",
"state": "Returns are allowed within 30 days. This item was bought 12 days ago. It arrived undamaged.",
"questions": {
"eligible": {
"type": "noul",
"instructions": "Is this item within the store return window?",
"criteria": {}
},
"department": {
"type": "choice",
"instructions": "Which department handles a return?",
"criteria": {
"returns": "Product returns and refunds",
"technical": "Software installation"
}
},
"damage": {
"type": "score",
"instructions": "How damaged was the item?",
"criteria": ["Undamaged", "Minor damage", "Destroyed"]
}
}
}'
What to expect: a high answers.eligible.noul, answers.department.choice of returns, and answers.damage.score near zero. Change the purchase date from 12 to 90 days and eligibility should flip. Exact probabilities vary with the weight format and backend — a useful sanity check when you're testing quants.
How accurate and how fast
Two sets of numbers, from two different evaluations — they're not comparable to each other.
Bespoke Labs' own figure: a Decision Index 0.2.1 score of 56.88, with the full run, inference implementation and reproduction instructions published. It's the publisher's index, measured with its reference implementation, so treat it as the vendor's number.
Ollama's JevBench run: 231 cases across the original, easy and hard sets, prompts spanning 190–4,112 tokens, on an Apple M5 Max with a 40-core GPU running Ollama 0.40.4-rc0. One warmup request, then a complete serial run per variant, AC power, fans maxed, no competing GPU workload. Every variant completed all requests without API errors.
| Variant | Correct / 231 | Median latency | p95 latency |
|---|---|---|---|
| MLX BF16 | 201 | 88 ms | 873 ms |
| MLX MXFP8 | 201 | 103 ms | 1,057 ms |
| MLX NVFP4 | 201 | 99 ms | 929 ms |
| GGUF BF16 | 202 | 176 ms | 1,336 ms |
| GGUF Q8_0 | 200 | 163 ms | 1,273 ms |
| GGUF Q4_K_M | 198 | 168 ms | 1,344 ms |
Two things stand out. First, quantisation barely dents accuracy here: 198–202 of 231 across every variant, including the 5.6GB Q4_K_M. Second, the latency: 88–176ms median full HTTP request times with the model already loaded. A chat model has to generate tokens one at a time; nimble-v3 scores answer tokens directly, so there's nothing to generate. p95 reaches 1,344ms on the GGUF Q4_K_M variant, and latency depends on prompt length, question count and hardware — but for routing and checking, these are the kinds of numbers that let you put it in a hot path.
What it's good for — and what it isn't
Good for: routing, guardrail-style checks, rubric scoring — anywhere you're currently prompting a chat model for a one-word answer and parsing its output with regex and hope. The structured request and probability outputs replace all of that.
Not for: open-ended generation, summarising, chat. It returns probabilities, not prose. If you need text, this isn't your model.
The licence is a real constraint. CC BY-NC 4.0 rules out most commercial deployments. For a personal project or internal tooling it's fine; for a product, check the publisher's statement or pick something else.
How does it compare with what you might already use?
- General small models like qwen3:8b or llama3.1:8b with a strict prompt can do routing and classification, and they're more flexible — they can also chat, summarise and write. But you're back to parsing free text, and generation is slower. Nimble-v3 trades that flexibility for structured, fast, reliable answers.
- Guardrail models like llama-guard3:1b cover a narrower slice — safety classification specifically. Nimble-v3 is general-purpose decision-making: any routing, checking or scoring you can express as choices, Booleans or rubrics.
What we'd pick: if you're building a local pipeline that needs routing or scoring in a hot path and the noncommercial licence fits, nimble-v3 is a genuinely different tool — run the Q4_K_M where memory is tight, or the default quant on anything with headroom. If you need one model that also writes, stick with a general-purpose 8–9B and accept the parsing.
Questions people ask
What is nimble-v3?
A 9B decision model from Bespoke Labs, fine-tuned from Qwen3.5-9B. You give it facts and questions with defined answers, and it returns probabilities for those answers — for routing, classification and scoring rather than chat.
Models in this post — live on Inpherio
Have a GPU, a Mac or a spare PC?
Let it earn while you sleep.
Run open models for people who need them. You set the price and keep everything but the 15% fee.
Become a hostBuilding with AI?
Every open model, one API.
OpenAI-compatible. Priced by the people who run the hardware.
Start free See the marketplaceRead “nimble-v3 on Ollama: a 9B decision model that answers with probabilities, not prose” at https://inpherio.co.uk/blog/nimble-v3-on-ollama-a-9b-decision-model-that-answers-with-probabilitie