Strands Decider on Ollama: a 2B model that answers with probabilities, not prose

A new kind of local model has appeared on Ollama: one that doesn't chat at all, but returns probabilities for decisions you define. Here's a practical first look at Strands Decider.

Listen to this post · 10 min

Accessibility: the whole post is read aloud — Space plays or pauses, ← → skip 10 seconds, and you can change the speed. It keeps playing with your screen locked.

Share this post

In short

  • Strands Decider is a 2B decision model from Strands Agents, built on Qwen3.5-2B-Base with a trained decision head — it returns probabilities for answers you define instead of generating text.

Strands Decider is now on Ollama, and it's an unusual release: a 2B model from Strands Agents that doesn't generate text at all. Instead, it assigns probabilities to answers you define — for classification, routing and scoring. If you've ever wanted a local model that answers "which department?" with a probability instead of a paragraph, this is that.

What Strands Decider is

Strands Decider is a 2B-parameter decision model built on Qwen3.5-2B-Base, with a trained decision head on top. That head is the whole point: rather than generating a free-form response, the model outputs probabilities over the answers you specify. This release is the publisher's v1-2610 checkpoint, licensed under Apache 2.0.

It supports three question types:

  • choice — pick a category from options you name, with a probability for each option
  • a Boolean question type — answer true or false, returning the probability that it's true
  • score — rate an input against an ordered rubric, returning a probability-weighted score with levels numbered from zero

One caveat before you copy anything: the readme's question-type table looks garbled on the Ollama page — it names the Boolean type as noul, which reads like a scrape artefact where bool almost certainly belongs. The worked example uses "type": "noul" too, so the page is at least internally consistent, but check the live API before building on it. We'll use the readme's spelling below so the example runs as published.

You can ask several questions about the same text in one request — useful when you want to triage a ticket on several axes at once.

The publisher's suggested uses: choosing a tool, routing a support ticket, applying a written policy, or evaluating an answer against a rubric.

How to run it locally

First, the constraint: these packages run with MLX, so you need Ollama 0.40.0 or later on an Apple silicon Mac. This is a Mac-only release for now — more on that below.

ollama pull strands-decider
# or
ollama run strands-decider

It's not a chat model, so ollama run won't give you a conversation. You talk to it through the decision API at /v1/systemone: put your evidence in state and your named questions in questions. The response's answers object uses the same question names you gave.

Here's the readme's worked example — a returns policy with one question of each type:

curl --fail-with-body -sS http://localhost:11434/v1/systemone \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "strands-decider",
    "state": "Returns are allowed within 30 days. This item was bought 12 days ago. It arrived undamaged.",
    "questions": {
      "eligible": {
        "type": "noul",
        "instructions": "Is this item within the store return window?",
        "criteria": {}
      },
      "department": {
        "type": "choice",
        "instructions": "Which department handles a return?",
        "criteria": {
          "returns": "Product returns and refunds",
          "technical": "Software installation"
        }
      },
      "damage": {
        "type": "score",
        "instructions": "How damaged was the item?",
        "criteria": ["Undamaged", "Minor damage", "Destroyed"]
      }
    }
  }'

For this example, the readme says to expect eligibility above 0.5, the returns department, and a damage score below 0.5 — exact probabilities vary by weight format.

Note what's absent: chat generation and image input are not supported. Use the decision API; there's no other interface.

If you're new to Ollama itself, our Ollama engine guide covers setup in more depth.

Sizes, memory and which Mac you need

There are three MLX variants. Download sizes are approximate, and the readme notes that runtime memory also includes caches and working buffers.

TagWeight formatDownload size
strands-decider, :latest, :2bMXFP8 (default)2.0 GB
strands-decider:2b-mlx-bf16Unquantised BF163.8 GB
strands-decider:2b-nvfp4NVFP4, with MXFP8 embeddings1.3 GB

The usual rule of thumb applies: a model needs roughly its file size in RAM or VRAM, plus room for context. Here the context is capped at 4K — your formatted state and questions must fit within 4,096 tokens — so the overhead is modest. On that basis, the default 2.0 GB variant should be comfortable on an 8 GB Apple silicon Mac, and the 3.8 GB BF16 variant should still fit on machines with more memory. Treat that as an estimate, not a guarantee — the source only gives download sizes, not measured memory use.

This is firmly a laptop-class model, not a gaming-GPU or server one. Because it's MLX-only, NVIDIA and AMD GPUs and plain CPU boxes are out for now. If you're not on a Mac, this release isn't for you today.

One detail worth knowing: the MLX quantisation uses NVIDIA Model Optimiser, and the decision head is retained at its original precision — so the part that actually makes decisions isn't quantised even in the optimised small variants.

How fast and how accurate

The publisher reports 180/231 correct on JevBench, measured with its reference implementation at a 4,096-token window.

Ollama also ran its own measurements with unmodified JevBench — all 231 original, easy and hard cases, no API or schema errors, prompts ranging from 82 to 3,936 tokens. The setup: Apple M5 Pro, 64 GiB RAM, macOS 27.0, Ollama 0.40.0, a fresh runner per variant, one warmup request, one serial pass, no competing GPU workload.

MLX variantCorrect / 231Median latencyp95 latency
BF1618233.3 ms343.5 ms
MXFP818342.5 ms384.8 ms
NVFP418241.5 ms372.8 ms

Latencies are full HTTP request times with the model loaded. As the source itself caveats: different requests and hardware will produce different timings.

Two things stand out. First, the speed: 33–43 ms median for a decision is fast enough to sit in a request path without anyone noticing. Second, quantisation barely costs accuracy here — the 1.3 GB NVFP4 variant matches BF16 within one question. If you're memory-tight, the small one is the sensible pick.

What it's good for — and where it falls over

The publisher's suggested uses are the honest ones: choosing a tool, routing a support ticket, applying a written policy, evaluating an answer against a rubric. Anywhere you currently ask a small LLM "which of these categories does this belong to?" and parse its reply, this is a cleaner shape.

The limits, also from the publisher:

  • Weaker on long, multi-step documents and unfamiliar Boolean or scoring tasks.
  • Question wording may not behave as you expect — rewording a question may not change the answer as intended. Use clear options.
  • Probabilities aren't guarantees. Calibration was fitted on short classification tasks. Test your actual prompts before relying on a decision threshold.
  • The 4K window is a real constraint for routing: a long ticket or document plus your questions must fit inside 4,096 tokens.

Where it sits in a local stack: it's a cheap, fast gate in front of bigger models. Route or score first with Decider, then send only what genuinely needs generation to a chat model. That's a pattern that saves both memory and latency on a single machine.

If you want a general-purpose small model for the chat side of that pipeline, something like qwen3:1.7b is a reasonable pairing:

How it compares with the usual suspects

If you're choosing something for local classification, these are the realistic options:

What it doesOutputWhere it runs
Strands DeciderGeneral routing, Boolean and rubric scoring with your own optionsProbabilities, no free-form textMLX / Apple silicon Mac via Ollama
Small instruct models (e.g. qwen3:1.7b)Anything: chat, classification, extractionGenerated tokens you have to parseEverywhere
Guard models (e.g. llama-guard3:1b, granite3-guardian:2b)Safety classificationProbabilities over fixed safety labelsEverywhere

The key distinction is the output shape. A normal small model asked to classify still generates tokens, and you have to parse them — which is where things go wrong: it might add a preamble, hedge, or pick a label you didn't offer. Decider returns probabilities directly, which is faster and more predictable when you're thresholding. But it only does that for its three question types.

Guard models are the closest established category — probabilities over fixed labels — but they're trained for safety classification with fixed taxonomies. Decider is general-purpose: you bring your own options and rubrics.

The trade-off is availability. Decider is Mac-only today via Ollama. If you're on CUDA or CPU, a small instruct model with structured output (JSON schema constraints, or just "reply with one word") is the practical fallback — less elegant, but it works everywhere.

Worth a look if you route or triage locally

Our verdict: this is a genuinely different shape of local model — small, fast, probability-first — and the quantisation results suggest the 1.3 GB NVFP4 variant is the one most people should grab. If you triage tickets, pick tools, or apply policies on a Mac, it's worth an hour of testing with your own prompts.

If you don't have an Apple silicon Mac, similar routing work can be done with small, cheap models through an API — prompted with structured output rather than a dedicated decision head. See what's on the marketplace and what it costs.

And if you do have a spare Mac sitting idle, models this small are cheap to host and serve.

Questions people ask

Can I run Strands Decider on a Windows PC or NVIDIA GPU?

Not via Ollama, at least for now. These packages run with MLX, so you need Ollama 0.40.0 or later on an Apple silicon Mac. On CUDA or CPU, a small instruct model with structured output is the practical fallback.

Models in this post — live on Inpherio

Have a GPU, a Mac or a spare PC?

Let it earn while you sleep.

Run open models for people who need them. You set the price and keep everything but the 15% fee.

Become a host

Building with AI?

Every open model, one API.

OpenAI-compatible. Priced by the people who run the hardware.

Start free See the marketplace

Read “Strands Decider on Ollama: a 2B model that answers with probabilities, not prose” at https://inpherio.co.uk/blog/strands-decider-on-ollama-a-2b-model-that-answers-with-probabilities-n