Marketplace · Text & chat

bonsai-2:2b

Call bonsai-2:2b through one OpenAI-compatible API. It's served by 1 host on the Inpherio grid, each at a price it sets — your request goes to the cheapest healthy one, and fails over to the next if it's down.

From
£0.23
per 1M output tokens
£1 buys about 4.3 million tokens (≈ 3.3 million words)
Hosts
1
Loaded now
0

At a glance.

vision tools reasoning

  • Type: Text & chat
  • Hosts serving it now: 1 (0 with it loaded in memory)
  • Engines: ollama
  • Context window: up to 262,144 tokens
  • Rentable: a whole host from £0.04 an hour (rentals)
python · openai sdk
from openai import OpenAI
client = OpenAI(base_url="https://inpherio.co.uk/v1",
                api_key=INPHERIO_API_KEY)
reply = client.chat.completions.create(
    model="bonsai-2:2b",
    messages=[{"role": "user", "content": "Hello"}],
)

bonsai-2:2b in detail.

Bonsai 2 from Prism ML is a family of vision models built for ternary weights – smaller models with close to full-precision quality.

Published by
prism-ml
Parameters
1.9B
Download size
911 MB (as packaged for Ollama)
Quantisation
Q2_0
Format
gguf
Architecture
qwen35
Licence
Apache 2.0
Based on
Qwen/Qwen3.5-2B
Task
image text to text
Library
llama.cpp
First published
23 Sep 2026
Last updated
4 Nov 2026
Downloads (30 days)
873,081 on Hugging Face
Likes
48

llama.cpp gguf ternary 2-bit llama-cpp cuda metal on-device hybrid-attention vision prismml bonsai image-text-to-text

The model card

Published by the model's authors on Hugging Face — shown as they wrote it, unedited by Inpherio.

Prism ML Website  |  Demo & Examples  |  Discord

Bonsai 2 Ternary 2B: GGUF

A 2B-class vision-language model with ternary transformer weights, for llama.cpp (CUDA, Metal, CPU).

~0.52 GB language model (PQ2_0), down from ~3.8 GB at FP16

Highlights

  • Ternary language weights across embeddings, attention projections, MLP projections and the (tied) LM head, stored with a group-wise FP16 scale.
  • Three GGUF packings of the same weights, consumed directly by custom kernels and never expanded back to FP16: PTQ1_0 (dense trits, 0.43 GB), PQ2_0 (2-bit slots, 0.52 GB) and Q2_0 (0.55 GB).
  • Vision input through a separate mmproj pack (Q8_0, or BF16 for reference precision), loaded only when an image arrives.
  • 262K-token context, inherited from the Qwen3.5-2B hybrid-attention backbone (18 linear-attention and 6 full-attention layers).

Model Overview

ItemSpecification
Base modelDerived from Qwen3.5-2B (architecture unchanged)
Parameters~1.88B language model (24 blocks, hidden 2048, tied embedding/LM head) + ~0.33B vision tower (24 blocks)
ArchitectureHybrid attention (3 linear-attention : 1 full-attention), SwiGLU MLP, RoPE, RMSNorm
Context length262,144 tokens (inherited from the base model)
Weight formatTernary {−1, 0, +1} with FP16 group-wise scales, packed as PTQ1_0, PQ2_0 or Q2_0
Weight basisBlockwise Hadamard rotation folded into the stored weights; the runtime applies the matching transform to activations
Higher-precision tensorsNormalization weights and the linear-attention state path (ssm_alpha, ssm_beta) stay F32
Vision towerOptional mmproj pack: Q8_0 (0.36 GB, recommended) or BF16 (0.67 GB)
Backendsllama.cpp (CUDA, Metal, CPU): every file runs on the PrismML fork; Q2_0 also runs on upstream llama.cpp b11556 or newer
LicenseApache 2.0

Files

FileContentsSizeBits/weight (language model)
Ternary-Bonsai-2-2B-PTQ1_0.gguflanguage model, dense trits0.43 GB~1.83
Ternary-Bonsai-2-2B-PQ2_0.gguflanguage model, 2-bit slots0.52 GB~2.20
Ternary-Bonsai-2-2B-Q2_0.gguflanguage model, Q2_00.55 GB~2.33
Ternary-Bonsai-2-2B-mmproj-Q8_0.ggufvision tower0.36 GB
Ternary-Bonsai-2-2B-mmproj-BF16.ggufvision tower (reference precision)0.67 GB

Bits/weight is file size over the ~1.88B language-model parameters, so it includes the F32 tensors and metadata. The three language-model packs hold the same ternary weights; they differ only in storage layout and kernel speed.

Quickstart

**PrismML-Eng/Bonsai-demo is the source of truth for running these models.** It has the tested setup for every backend and pins a known-good binary release.

Use the PrismML llama.cpp build

The ternary kernels live in the PrismML-Eng/llama.cpp fork (branch prism). It runs every file here. Upstream llama.cpp runs the Q2_0 file from release b11556 onward (see Use upstream llama.cpp below). It doesn't have the PTQ1_0 and PQ2_0 types yet, so it rejects those files as unknown types.

### prebuilt: pick the archive for your platform
### https://github.com/PrismML-Eng/llama.cpp/releases/latest
tar -xzf llama--bin-.tar.gz -C bin --strip-components=1
Read the rest of the model card

or build it

git clone https://github.com/PrismML-Eng/llama.cpp && cd llama.cpp cmake -B build -DGGML_CUDA=ON && cmake --build build -j # drop -DGGML_CUDA=ON on macOS; Metal is the default


### Text

./bin/llama-cli -m Ternary-Bonsai-2-2B-PQ2_0.gguf -ngl 99 -fa on \ -p "Explain in three sentences why the sky is blue."


### Image + text

./bin/llama-mtmd-cli -m Ternary-Bonsai-2-2B-PQ2_0.gguf \ --mmproj Ternary-Bonsai-2-2B-mmproj-Q8_0.gguf -ngl 99 \ --image photo.jpg -p "Describe this image."


**Vision projector:** use the Q8_0 mmproj. With the current fork build, the BF16 mmproj can crash llama-server when several image requests run at once; Q8_0 is not affected and quality is nearly the same.

**Small text in images:** for small or dense text, raising the image token budget with `--image-min-tokens 1024` may help.

### OpenAI-compatible server

./bin/llama-server -m Ternary-Bonsai-2-2B-PQ2_0.gguf \ --mmproj Ternary-Bonsai-2-2B-mmproj-Q8_0.gguf -ngl 99 -fa on -c 32768 --port 8080

curl http://localhost:8080/v1/chat/completions -H "Content-Type: application/json" -d '{ "messages": [{"role": "user", "content": [ {"type": "text", "text": "What is in this image?"}, {"type": "image_url", "image_url": {"url": "https://upload.wikimedia.org/wikipedia/commons/4/47/PNG_transparency_demonstration_1.png"}} ]}], "chat_template_kwargs": {"enable_thinking": false} }'


### Use upstream llama.cpp

The `Q2_0` file also runs on stock llama.cpp from release [b11556](https://github.com/ggml-org/llama.cpp/releases/tag/b11556) onward, which added the Hadamard activation runtime these weights need. Older builds lack that runtime and give wrong output without any warning, so use a current release. Text and image input both work:

hf download prism-ml/Ternary-Bonsai-2-2B-gguf Ternary-Bonsai-2-2B-Q2_0.gguf Ternary-Bonsai-2-2B-mmproj-Q8_0.gguf --local-dir . llama-cli -m Ternary-Bonsai-2-2B-Q2_0.gguf -ngl 99 -fa on \ -p "Explain in three sentences why the sky is blue." llama-mtmd-cli -m Ternary-Bonsai-2-2B-Q2_0.gguf \ --mmproj Ternary-Bonsai-2-2B-mmproj-Q8_0.gguf -ngl 99 \ --image photo.jpg -p "Describe this image."


`llama-server` takes the same `-m` and `--mmproj` arguments as above.

### Best Practices

### Generation Parameters

We recommend these sampling settings. They are the same for every packing and for MLX:

> - Thinking mode (default): `temperature=1.0`, `top_p=0.95`, `top_k=20`, `min_p=0.0`, `presence_penalty=1.0`, `repetition_penalty=1.0`
> - Non-thinking mode: `temperature=0.7`, `top_p=0.80`, `top_k=20`, `min_p=0.0`, `presence_penalty=1.5`, `repetition_penalty=1.0`

Apply the penalties over the last 64 tokens (llama.cpp's default window). Presence penalty has no GGUF metadata key, so pass it on the command line or per request.

thinking on

./bin/llama-server -m Ternary-Bonsai-2-2B-PQ2_0.gguf -ngl 99 -fa on \ --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0 --presence-penalty 1.0 --repeat-penalty 1.0 --repeat-last-n 64

thinking off: start the server with these values, and send

"chat_template_kwargs": {"enable_thinking": false} with each request

./bin/llama-server -m Ternary-Bonsai-2-2B-PQ2_0.gguf -ngl 99 -fa on \ --temp 0.7 --top-p 0.8 --top-k 20 --min-p 0 --presence-penalty 1.5 --repeat-penalty 1.0 --repeat-last-n 64


Avoid greedy decoding (`temperature=0`) with thinking on; it can get stuck reasoning on some prompts.

### Output Length

Give thinking mode room to finish. Even simple prompts can use 3,000+ thinking tokens, so allow at least 8,192 output tokens with thinking on. For short answers, turn thinking off.

### System Prompt

You can use a simple system prompt such as:

You are a helpful assistant


### MLX

On Apple Silicon you can also use the MLX build of the same weights, [Ternary-Bonsai-2-2B-mlx-2bit](https://huggingface.co/prism-ml/Ternary-Bonsai-2-2B-mlx-2bit). It loads natively in mlx-vlm 0.7.8 and newer (the released mlx-lm cannot load it yet). Pass the values above explicitly: mlx-vlm's Python `generate()` is greedy by default. Its model card has the commands.

### Choosing a Packing

- **PQ2_0**: the default; cheaper to unpack, and the pack measured on Apple Silicon.
- **PTQ1_0**: the smallest file; use it where memory is tightest.
- **Q2_0**: an alternative layout, and the one format upstream llama.cpp can also run (b11556 or newer).

### Speculative Decoding

The 2B ships without a draft model. At this size a decode step is already cheap, and in our tests speculative decoding with a draft head was slower than plain decoding on everyday prompts, so plain decoding is the recommended setup.

### License

Apache 2.0, following the base model.

Sources: prism-ml/Ternary-Bonsai-2-2B-gguf on Hugging Face · bonsai-2:2b on Ollama. Collected from public sources on 8 Nov 2026; the host serving it may run a different build or quantisation — the marketplace shows each host's own measurements.

Who's serving it.

Every host benchmarked this model on its own hardware before serving it. Prices are set by each host; health is scored by the platform from what it observes.

Hosts serving bonsai-2:2b right now
HostTierPriceMeasured speedHealthStateRent
7B-A1B (5G)L1£0.057 in · £0.23 out20 tok/s · 135 ms first token82loads on request£0.04/h

Live data, refreshed on every visit. How billing works.

Room on the grid for your machine.

Right now one machine serves bonsai-2:2b. If it drops, so does every request for it — a second host is what turns it into failover, and the grid needs that more than another copy. The router weighs price, measured speed, health and whether the model is already loaded — so a host with a sharper price or quicker hardware wins the next call.

Hosts now
1
0 loaded in memory
Price to beat
£0.23
per 1M output tokens
Fastest measured
20 tok/s

Set your own price — you keep 85% of what it earns, and new hosts start with 12 months fee-free (up to £250 of earnings). A payout fee of £2 + 1% per payout to a UK bank account, £2 + 2% to any other country applies when you're paid. Install the connector, load bonsai-2:2b, and the platform benchmarks it on your hardware before your first request.