# Strands Decider on Ollama: a 2B model that answers with probabilities, not prose

A new kind of local model has appeared on Ollama: one that doesn't chat at all, but returns probabilities for decisions you define. Here's a practical first look at Strands Decider.

Published 2026-10-13 · https://inpherio.co.uk/blog/strands-decider-on-ollama-a-2b-model-that-answers-with-probabilities-n

Key points:
- Strands Decider is a 2B decision model from Strands Agents, built on Qwen3.5-2B-Base with a trained decision head — it returns probabilities for answers you define instead of generating text.

Strands Decider is now on [Ollama](https://ollama.com/library/strands-decider), and it's an unusual release: a 2B model from Strands Agents that doesn't generate text at all. Instead, it assigns probabilities to answers you define — for classification, routing and scoring. If you've ever wanted a local model that answers "which department?" with a probability instead of a paragraph, this is that.

## What Strands Decider is

Strands Decider is a 2B-parameter decision model built on Qwen3.5-2B-Base, with a trained decision head on top. That head is the whole point: rather than generating a free-form response, the model outputs probabilities over the answers you specify. This release is the publisher's v1-2610 checkpoint, licensed under Apache 2.0.

It supports three question types:

- **choice** — pick a category from options you name, with a probability for each option
- a Boolean question type — answer true or false, returning the probability that it's true
- **score** — rate an input against an ordered rubric, returning a probability-weighted score with levels numbered from zero

One caveat before you copy anything: the readme's question-type table looks garbled on the Ollama page — it names the Boolean type as `noul`, which reads like a scrape artefact where `bool` almost certainly belongs. The worked example uses `"type": "noul"` too, so the page is at least internally consistent, but check the live API before building on it. We'll use the readme's spelling below so the example runs as published.

You can ask several questions about the same text in one request — useful when you want to triage a ticket on several axes at once.

The publisher's suggested uses: choosing a tool, routing a support ticket, applying a written policy, or evaluating an answer against a rubric.

## How to run it locally

First, the constraint: these packages run with [MLX](/engines/mlx), so you need **Ollama 0.40.0 or later on an Apple silicon Mac**. This is a Mac-only release for now — more on that below.

```bash
ollama pull strands-decider
# or
ollama run strands-decider
```

It's not a chat model, so `ollama run` won't give you a conversation. You talk to it through the decision API at `/v1/systemone`: put your evidence in `state` and your named questions in `questions`. The response's `answers` object uses the same question names you gave.

Here's the readme's worked example — a returns policy with one question of each type:

```bash
curl --fail-with-body -sS http://localhost:11434/v1/systemone \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "strands-decider",
    "state": "Returns are allowed within 30 days. This item was bought 12 days ago. It arrived undamaged.",
    "questions": {
      "eligible": {
        "type": "noul",
        "instructions": "Is this item within the store return window?",
        "criteria": {}
      },
      "department": {
        "type": "choice",
        "instructions": "Which department handles a return?",
        "criteria": {
          "returns": "Product returns and refunds",
          "technical": "Software installation"
        }
      },
      "damage": {
        "type": "score",
        "instructions": "How damaged was the item?",
        "criteria": ["Undamaged", "Minor damage", "Destroyed"]
      }
    }
  }'
```

For this example, the readme says to expect eligibility above 0.5, the returns department, and a damage score below 0.5 — exact probabilities vary by weight format.

Note what's absent: chat generation and image input are not supported. Use the decision API; there's no other interface.

If you're new to Ollama itself, our [Ollama engine guide](/engines/ollama) covers setup in more depth.

## Sizes, memory and which Mac you need

There are three MLX variants. Download sizes are approximate, and the readme notes that runtime memory also includes caches and working buffers.

| Tag | Weight format | Download size |
|---|---|---|
| `strands-decider`, `:latest`, `:2b` | MXFP8 (default) | 2.0 GB |
| `strands-decider:2b-mlx-bf16` | Unquantised BF16 | 3.8 GB |
| `strands-decider:2b-nvfp4` | NVFP4, with MXFP8 embeddings | 1.3 GB |

The usual rule of thumb applies: a model needs roughly its file size in RAM or VRAM, plus room for context. Here the context is capped at 4K — your formatted state and questions must fit within 4,096 tokens — so the overhead is modest. On that basis, the default 2.0 GB variant should be comfortable on an 8 GB Apple silicon Mac, and the 3.8 GB BF16 variant should still fit on machines with more memory. Treat that as an estimate, not a guarantee — the source only gives download sizes, not measured memory use.

This is firmly a laptop-class model, not a gaming-GPU or server one. Because it's MLX-only, NVIDIA and AMD GPUs and plain CPU boxes are out for now. If you're not on a Mac, this release isn't for you today.

One detail worth knowing: the MLX quantisation uses NVIDIA Model Optimiser, and the decision head is retained at its original precision — so the part that actually makes decisions isn't quantised even in the optimised small variants.

## How fast and how accurate

The publisher reports **180/231 correct on JevBench**, measured with its reference implementation at a 4,096-token window.

Ollama also ran its own measurements with unmodified JevBench — all 231 original, easy and hard cases, no API or schema errors, prompts ranging from 82 to 3,936 tokens. The setup: Apple M5 Pro, 64 GiB RAM, macOS 27.0, Ollama 0.40.0, a fresh runner per variant, one warmup request, one serial pass, no competing GPU workload.

| MLX variant | Correct / 231 | Median latency | p95 latency |
|---|---|---|---|
| BF16 | 182 | 33.3 ms | 343.5 ms |
| MXFP8 | 183 | 42.5 ms | 384.8 ms |
| NVFP4 | 182 | 41.5 ms | 372.8 ms |

Latencies are full HTTP request times with the model loaded. As the source itself caveats: different requests and hardware will produce different timings.

Two things stand out. First, the speed: 33–43 ms median for a decision is fast enough to sit in a request path without anyone noticing. Second, quantisation barely costs accuracy here — the 1.3 GB NVFP4 variant matches BF16 within one question. If you're memory-tight, the small one is the sensible pick.

## What it's good for — and where it falls over

The publisher's suggested uses are the honest ones: choosing a tool, routing a support ticket, applying a written policy, evaluating an answer against a rubric. Anywhere you currently ask a small LLM "which of these categories does this belong to?" and parse its reply, this is a cleaner shape.

The limits, also from the publisher:

- **Weaker on long, multi-step documents** and unfamiliar Boolean or scoring tasks.
- **Question wording may not behave as you expect** — rewording a question may not change the answer as intended. Use clear options.
- **Probabilities aren't guarantees.** Calibration was fitted on short classification tasks. Test your actual prompts before relying on a decision threshold.
- **The 4K window is a real constraint** for routing: a long ticket or document plus your questions must fit inside 4,096 tokens.

Where it sits in a local stack: it's a cheap, fast gate in front of bigger models. Route or score first with Decider, then send only what genuinely needs generation to a chat model. That's a pattern that saves both memory and latency on a single machine.

If you want a general-purpose small model for the chat side of that pipeline, something like [qwen3:1.7b](/models/qwen3:1.7b) is a reasonable pairing:

(Live on Inpherio: https://inpherio.co.uk/models/qwen3:1.7b)

## How it compares with the usual suspects

If you're choosing something for local classification, these are the realistic options:

| | What it does | Output | Where it runs |
|---|---|---|---|
| **Strands Decider** | General routing, Boolean and rubric scoring with your own options | Probabilities, no free-form text | MLX / Apple silicon Mac via Ollama |
| **Small instruct models** (e.g. [qwen3:1.7b](/models/qwen3:1.7b)) | Anything: chat, classification, extraction | Generated tokens you have to parse | Everywhere |
| **Guard models** (e.g. [llama-guard3:1b](/models/llama-guard3:1b), [granite3-guardian:2b](/models/granite3-guardian:2b)) | Safety classification | Probabilities over fixed safety labels | Everywhere |

The key distinction is the output shape. A normal small model asked to classify still generates tokens, and you have to parse them — which is where things go wrong: it might add a preamble, hedge, or pick a label you didn't offer. Decider returns probabilities directly, which is faster and more predictable when you're thresholding. But it only does that for its three question types.

Guard models are the closest established category — probabilities over fixed labels — but they're trained for safety classification with fixed taxonomies. Decider is general-purpose: you bring your own options and rubrics.

The trade-off is availability. Decider is Mac-only today via Ollama. If you're on CUDA or CPU, a small instruct model with structured output (JSON schema constraints, or just "reply with one word") is the practical fallback — less elegant, but it works everywhere.

## Worth a look if you route or triage locally

Our verdict: this is a genuinely different shape of local model — small, fast, probability-first — and the quantisation results suggest the 1.3 GB NVFP4 variant is the one most people should grab. If you triage tickets, pick tools, or apply policies on a Mac, it's worth an hour of testing with your own prompts.

If you don't have an Apple silicon Mac, similar routing work can be done with small, cheap models through an API — prompted with structured output rather than a dedicated decision head. See what's on the [marketplace](/marketplace) and what it costs.

And if you do have a spare Mac sitting idle, models this small are cheap to host and serve.

## FAQ

**Can I run Strands Decider on a Windows PC or NVIDIA GPU?** Not via Ollama, at least for now. These packages run with MLX, so you need Ollama 0.40.0 or later on an Apple silicon Mac. On CUDA or CPU, a small instruct model with structured output is the practical fallback.
