# Bonsai 2 on Ollama: a 27B model that fits in 8.3GB

Prism ML's Bonsai 2 arrived on Ollama today with an unusual pitch: ternary weights that roughly halve file sizes versus 4-bit quants. Here's what each size needs and how to run it.

Published 2026-10-16 · https://inpherio.co.uk/blog/bonsai-2-on-ollama-a-27b-model-that-fits-in-8-3gb

Key points:
- Bonsai 2 is Prism ML's family of vision models built for ternary weights (−1, 0, +1), claiming roughly half the file size of 4-bit quants at close to full-precision quality
- Four sizes — 2b (911MB), 4b (1.6GB), 9b (3.2GB) and 27b (8.3GB) — all with vision, tools, thinking and a 262K context window
- Run it with `ollama run bonsai-2` or pick a size with `ollama run bonsai-2:27b`
- Budget roughly the download size in RAM/VRAM plus context headroom; the 27b fits a 12GB GPU or 16GB Mac
- Quality claims are Prism ML's own, from the 27B whitepaper — worth trying, but wait for independent comparisons before switching from a 4-bit Qwen or Gemma you already trust

Bonsai 2 is a family of vision models from [Prism ML](https://ollama.com/library/bonsai-2), released on Ollama today (16 October 2026), and it arrives with an unusual pitch: ternary weights that the makers claim roughly halve file sizes versus standard 4-bit quants. If that holds up, the 27b model — an 8.3GB download — becomes interesting to anyone with a 12GB GPU. Here's what each size needs and how to run it.

## What is bonsai-2?

Bonsai 2 comes in four sizes — 2b, 4b, 9b and 27b — and every one of them supports image input, tool calling, thinking mode and a 262K-token context window, according to the [Ollama library page](https://ollama.com/library/bonsai-2). It's Apache 2.0 licensed, so you can run it, host it and build on it commercially without asking anyone's permission.

The headline feature is what's underneath: ternary weights. Each weight in the model is −1, 0 or +1 instead of a full-precision number. Prism ML claims that makes the files about half the size of standard 4-bit quantised models, at close to full-precision quality.

That's a claim worth unpacking, because it's the whole reason this release is interesting. Let's do that next.

## Why ternary weights matter if you run models at home

Almost every local model you've downloaded is a quantisation of a full-precision model — someone trained it in 16-bit (or similar), then compressed it to 4-bit or 8-bit so it fits on your GPU. Quality loss is the accepted price.

Bonsai-2 takes a different route: it was **designed for ternary weights from the start** rather than compressed afterwards. The practical payoff is disk and memory. The 27b ships at 8.3GB. For comparison, a 4-bit quant needs roughly half a byte per parameter, so a conventional 27B model lands around 13–14GB before overhead — bonsai-2's 8.3GB is still a big cut, and Prism ML claims quality close to full precision rather than the usual quantisation loss.

That claim is the one to watch. "Close to full-precision quality" is Prism ML's own wording, backed by a [27B whitepaper](https://github.com/PrismML-Eng/Bonsai-demo/blob/main/bonsai-2-27b-whitepaper.pdf) — not something we've verified, and not something anyone outside Prism ML had verified on release day.

Ternary models can also be faster on some hardware, because the maths is simpler — additions instead of full multiplications. But Ollama's page quotes no speeds, so treat that as a maybe until you've tested it yourself. As we've written before when comparing [vLLM and Ollama](/blog/vllm-vs-ollama-which-inference-engine-should-you-actually-run), published performance numbers rarely transfer to your machine. The same applies here, doubly so for a brand-new weight format.

## Sizes, download sizes and how much memory you'll need

Here's the full lineup from the [Ollama library page](https://ollama.com/library/bonsai-2):

| Tag | Download | Context | Inputs | Comfortable on |
|---|---|---|---|---|
| `bonsai-2:2b` | 911MB | 256K | Text, Image | Almost anything — phones, old laptops |
| `bonsai-2:4b` | 1.6GB | 256K | Text, Image | Any modern laptop, entry GPU |
| `bonsai-2:9b` | 3.2GB | 256K | Text, Image | 8GB gaming GPU, 16GB Mac |
| `bonsai-2:27b` | 8.3GB | 256K | Text, Image | 12GB GPU, 16GB Mac |
| `bonsai-2:latest` | 3.2GB | 256K | Text, Image | Same footprint as the 9b |

The rule of thumb: a model needs roughly its download size in RAM or VRAM, plus headroom for context. And 256K of context can eat a lot of extra memory if you actually fill it — the key-value cache for a long conversation grows well beyond the weights. If you're doing long-document work, budget accordingly.

The 27b is the interesting one. At 8.3GB, it has roughly the footprint of a 4-bit 16B model — half its own size class. If you have a 12GB GPU, you've previously been capped around the 16B class by disk and memory. This puts a 27B-class model (with vision, no less) inside that envelope.

Note that no size gives anything up: the 2b gets the same 262K context, image input and tool calling as the 27b.

## How to run it locally

If you have Ollama installed, it's one command:

```bash
ollama run bonsai-2
```

That pulls the default tag, which downloads 3.2GB. To pick a size explicitly:

```bash
ollama run bonsai-2:2b
ollama run bonsai-2:4b
ollama run bonsai-2:9b
ollama run bonsai-2:27b
```

Image input works the way it does for other vision models on the library: in the terminal you can drag or paste an image into the prompt, and through Ollama's API you pass images alongside text as you would with any vision model.

Because thinking and tool calling are supported across all sizes, it drops straight into Open WebUI or any tool-calling agent setup you already run — nothing about the integration is exotic.

What about other engines? The Ollama page doesn't list any. If you want it in LM Studio, llama.cpp or vLLM, check [Prism ML's Hugging Face page](https://huggingface.co/prism-ml) for official weights first — a new weight format needs engine-side support, so don't assume your usual GGUF workflow applies. And if you're weighing Ollama against vLLM for serving this to more than one person at a time, our [engine comparison](/blog/vllm-vs-ollama-which-inference-engine-should-you-actually-run) still holds: the choice depends more on your hardware than on any benchmark.

## What it's good for — and where to be careful

**Good fits:**

- **Vision plus tools plus thinking in one package.** Every size offers all three — so the 2b and 4b give you image input and tool calling at under 2GB, which is a combination worth trying even if you already run a small model.
- **Long-context work.** 262K tokens across every size is generous, especially at the small end.
- **The 27b on mid-range hardware.** If you have 12GB of VRAM and want a large model that previously needed more, this is the pitch.

**Where to be careful:**

- The quality claims come from Prism ML's own [whitepaper](https://github.com/PrismML-Eng/Bonsai-demo/blob/main/bonsai-2-27b-whitepaper.pdf). Independent comparisons weren't out as of release day.
- Ternary weights are a young approach. If a 4-bit Qwen or Gemma of the same size already does your job well, there's no reason to switch until real-world comparisons exist.
- Watch the small print on context: actually using 262K tokens multiplies memory use well beyond the download size.

(Live on Inpherio: https://inpherio.co.uk/models/bonsai-2:9b)

## How it compares with familiar models

Positioning by size and footprint — no invented benchmarks:

- **At 2b**, it sits alongside small vision models like Qwen3-VL 2B and [gemma4:e2b](/models/gemma4:e2b).
- **At 9b / 3.2GB**, it competes with 4-bit 7–9B models like [llama3.1:8b](/models/llama3.1:8b). At roughly half a byte per parameter, an 8B-class model typically downloads at around 4–5GB — so bonsai-2:9b is comparable on disk, while adding image input on top.
- **At 27b / 8.3GB**, it undercuts the ~13–14GB you'd expect for a 4-bit 27B-class model by around 40%. That's the whole pitch.

Whether quality matches those rivals at equal size is exactly what the whitepaper claims and what early community benchmarks will need to confirm. Until then, the honest position is: interesting, cheap to try, unproven.

**What we'd pick:** if you're curious, start with the 9b — 3.2GB is a cheap experiment, and it exercises everything the family offers. If the 27b quality holds up in independent tests, the 12GB-GPU story becomes genuinely compelling.

## Try it, or put your GPU to work

All four sizes are already callable through Inpherio's OpenAI-compatible API if you want to test bonsai-2 before downloading gigabytes — the 9b starts at £0.098 per 1M output tokens, and the [model page](/models/bonsai-2:9b) has live prices for the rest. If you'd rather call it from code first, the [developer docs](/developers) cover the quick start.

Going the other way: if your 12GB GPU or 16GB Mac can run the 27b, you can host it and earn per request. Hosts keep everything except the 15% platform fee, and new hosts get 12 months fee-free.

## FAQ

**Can bonsai-2 run on a laptop with no dedicated GPU?** Yes. The 2b is 911MB and the 4b is 1.6GB, so both run comfortably in ordinary laptop RAM; the 9b (3.2GB) is also fine on most 16GB machines. | How much VRAM do I need for bonsai-2:27b? | The download is 8.3GB, so a 12GB GPU covers the weights plus a reasonable context window. A 16GB Mac handles it too. | Does the small 2b model really support images and tools? | Yes — every size in the family supports vision, tool calling, thinking and the full 262K context window, according to the Ollama library page. | Is bonsai-2 better than Qwen or Gemma at the same size? | Nobody knows yet. The quality claims come from Prism ML's own whitepaper; independent comparisons weren't out on release day. | Can I run bonsai-2 in LM Studio or vLLM instead of Ollama? | The Ollama page doesn't list other engines. Check Prism ML's Hugging Face page for official weights before assuming support elsewhere.
