Marketplace · Text & chat
bonsai-2:2b
Call bonsai-2:2b through one OpenAI-compatible API. It's served by 1 host on the Inpherio grid, each at a price it sets — your request goes to the cheapest healthy one, and fails over to the next if it's down.
At a glance.
vision tools reasoning
from openai import OpenAI
client = OpenAI(base_url="https://inpherio.co.uk/v1",
api_key=INPHERIO_API_KEY)
reply = client.chat.completions.create(
model="bonsai-2:2b",
messages=[{"role": "user", "content": "Hello"}],
)bonsai-2:2b in detail.
Bonsai 2 from Prism ML is a family of vision models built for ternary weights – smaller models with close to full-precision quality.
- Published by
- prism-ml
- Parameters
- 1.9B
- Download size
- 911 MB (as packaged for Ollama)
- Quantisation
- Q2_0
- Format
- gguf
- Architecture
- qwen35
- Licence
- Apache 2.0
- Based on
- Qwen/Qwen3.5-2B
- Task
- image text to text
- Library
- llama.cpp
- First published
- 23 Sep 2026
- Last updated
- 4 Nov 2026
- Downloads (30 days)
- 873,081 on Hugging Face
- Likes
- 48
llama.cpp gguf ternary 2-bit llama-cpp cuda metal on-device hybrid-attention vision prismml bonsai image-text-to-text
The model card
Published by the model's authors on Hugging Face — shown as they wrote it, unedited by Inpherio.
Prism ML Website | Demo & Examples | Discord
Bonsai 2 Ternary 2B: GGUF
A 2B-class vision-language model with ternary transformer weights, for llama.cpp (CUDA, Metal, CPU).
~0.52 GB language model (PQ2_0), down from ~3.8 GB at FP16
Highlights
- Ternary language weights across embeddings, attention projections, MLP projections and the (tied) LM head, stored with a group-wise FP16 scale.
- Three GGUF packings of the same weights, consumed directly by custom kernels and never expanded back to FP16: PTQ1_0 (dense trits, 0.43 GB), PQ2_0 (2-bit slots, 0.52 GB) and Q2_0 (0.55 GB).
- Vision input through a separate mmproj pack (Q8_0, or BF16 for reference precision), loaded only when an image arrives.
- 262K-token context, inherited from the Qwen3.5-2B hybrid-attention backbone (18 linear-attention and 6 full-attention layers).
Model Overview
| Item | Specification |
|---|---|
| Base model | Derived from Qwen3.5-2B (architecture unchanged) |
| Parameters | ~1.88B language model (24 blocks, hidden 2048, tied embedding/LM head) + ~0.33B vision tower (24 blocks) |
| Architecture | Hybrid attention (3 linear-attention : 1 full-attention), SwiGLU MLP, RoPE, RMSNorm |
| Context length | 262,144 tokens (inherited from the base model) |
| Weight format | Ternary {−1, 0, +1} with FP16 group-wise scales, packed as PTQ1_0, PQ2_0 or Q2_0 |
| Weight basis | Blockwise Hadamard rotation folded into the stored weights; the runtime applies the matching transform to activations |
| Higher-precision tensors | Normalization weights and the linear-attention state path (ssm_alpha, ssm_beta) stay F32 |
| Vision tower | Optional mmproj pack: Q8_0 (0.36 GB, recommended) or BF16 (0.67 GB) |
| Backends | llama.cpp (CUDA, Metal, CPU): every file runs on the PrismML fork; Q2_0 also runs on upstream llama.cpp b11556 or newer |
| License | Apache 2.0 |
Files
| File | Contents | Size | Bits/weight (language model) |
|---|---|---|---|
Ternary-Bonsai-2-2B-PTQ1_0.gguf | language model, dense trits | 0.43 GB | ~1.83 |
Ternary-Bonsai-2-2B-PQ2_0.gguf | language model, 2-bit slots | 0.52 GB | ~2.20 |
Ternary-Bonsai-2-2B-Q2_0.gguf | language model, Q2_0 | 0.55 GB | ~2.33 |
Ternary-Bonsai-2-2B-mmproj-Q8_0.gguf | vision tower | 0.36 GB | |
Ternary-Bonsai-2-2B-mmproj-BF16.gguf | vision tower (reference precision) | 0.67 GB |
Bits/weight is file size over the ~1.88B language-model parameters, so it includes the F32 tensors and metadata. The three language-model packs hold the same ternary weights; they differ only in storage layout and kernel speed.
Quickstart
**PrismML-Eng/Bonsai-demo is the source of truth for running these models.** It has the tested setup for every backend and pins a known-good binary release.
Use the PrismML llama.cpp build
The ternary kernels live in the PrismML-Eng/llama.cpp fork (branch prism). It runs every file here. Upstream llama.cpp runs the Q2_0 file from release b11556 onward (see Use upstream llama.cpp below). It doesn't have the PTQ1_0 and PQ2_0 types yet, so it rejects those files as unknown types.
### prebuilt: pick the archive for your platform
### https://github.com/PrismML-Eng/llama.cpp/releases/latest
tar -xzf llama--bin-.tar.gz -C bin --strip-components=1
Read the rest of the model card
or build it
git clone https://github.com/PrismML-Eng/llama.cpp && cd llama.cpp cmake -B build -DGGML_CUDA=ON && cmake --build build -j # drop -DGGML_CUDA=ON on macOS; Metal is the default
### Text
./bin/llama-cli -m Ternary-Bonsai-2-2B-PQ2_0.gguf -ngl 99 -fa on \ -p "Explain in three sentences why the sky is blue."
### Image + text
./bin/llama-mtmd-cli -m Ternary-Bonsai-2-2B-PQ2_0.gguf \ --mmproj Ternary-Bonsai-2-2B-mmproj-Q8_0.gguf -ngl 99 \ --image photo.jpg -p "Describe this image."
**Vision projector:** use the Q8_0 mmproj. With the current fork build, the BF16 mmproj can crash llama-server when several image requests run at once; Q8_0 is not affected and quality is nearly the same.
**Small text in images:** for small or dense text, raising the image token budget with `--image-min-tokens 1024` may help.
### OpenAI-compatible server
./bin/llama-server -m Ternary-Bonsai-2-2B-PQ2_0.gguf \ --mmproj Ternary-Bonsai-2-2B-mmproj-Q8_0.gguf -ngl 99 -fa on -c 32768 --port 8080
curl http://localhost:8080/v1/chat/completions -H "Content-Type: application/json" -d '{ "messages": [{"role": "user", "content": [ {"type": "text", "text": "What is in this image?"}, {"type": "image_url", "image_url": {"url": "https://upload.wikimedia.org/wikipedia/commons/4/47/PNG_transparency_demonstration_1.png"}} ]}], "chat_template_kwargs": {"enable_thinking": false} }'
### Use upstream llama.cpp
The `Q2_0` file also runs on stock llama.cpp from release [b11556](https://github.com/ggml-org/llama.cpp/releases/tag/b11556) onward, which added the Hadamard activation runtime these weights need. Older builds lack that runtime and give wrong output without any warning, so use a current release. Text and image input both work:
hf download prism-ml/Ternary-Bonsai-2-2B-gguf Ternary-Bonsai-2-2B-Q2_0.gguf Ternary-Bonsai-2-2B-mmproj-Q8_0.gguf --local-dir . llama-cli -m Ternary-Bonsai-2-2B-Q2_0.gguf -ngl 99 -fa on \ -p "Explain in three sentences why the sky is blue." llama-mtmd-cli -m Ternary-Bonsai-2-2B-Q2_0.gguf \ --mmproj Ternary-Bonsai-2-2B-mmproj-Q8_0.gguf -ngl 99 \ --image photo.jpg -p "Describe this image."
`llama-server` takes the same `-m` and `--mmproj` arguments as above.
### Best Practices
### Generation Parameters
We recommend these sampling settings. They are the same for every packing and for MLX:
> - Thinking mode (default): `temperature=1.0`, `top_p=0.95`, `top_k=20`, `min_p=0.0`, `presence_penalty=1.0`, `repetition_penalty=1.0`
> - Non-thinking mode: `temperature=0.7`, `top_p=0.80`, `top_k=20`, `min_p=0.0`, `presence_penalty=1.5`, `repetition_penalty=1.0`
Apply the penalties over the last 64 tokens (llama.cpp's default window). Presence penalty has no GGUF metadata key, so pass it on the command line or per request.
thinking on
./bin/llama-server -m Ternary-Bonsai-2-2B-PQ2_0.gguf -ngl 99 -fa on \ --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0 --presence-penalty 1.0 --repeat-penalty 1.0 --repeat-last-n 64
thinking off: start the server with these values, and send
"chat_template_kwargs": {"enable_thinking": false} with each request
./bin/llama-server -m Ternary-Bonsai-2-2B-PQ2_0.gguf -ngl 99 -fa on \ --temp 0.7 --top-p 0.8 --top-k 20 --min-p 0 --presence-penalty 1.5 --repeat-penalty 1.0 --repeat-last-n 64
Avoid greedy decoding (`temperature=0`) with thinking on; it can get stuck reasoning on some prompts.
### Output Length
Give thinking mode room to finish. Even simple prompts can use 3,000+ thinking tokens, so allow at least 8,192 output tokens with thinking on. For short answers, turn thinking off.
### System Prompt
You can use a simple system prompt such as:
You are a helpful assistant
### MLX
On Apple Silicon you can also use the MLX build of the same weights, [Ternary-Bonsai-2-2B-mlx-2bit](https://huggingface.co/prism-ml/Ternary-Bonsai-2-2B-mlx-2bit). It loads natively in mlx-vlm 0.7.8 and newer (the released mlx-lm cannot load it yet). Pass the values above explicitly: mlx-vlm's Python `generate()` is greedy by default. Its model card has the commands.
### Choosing a Packing
- **PQ2_0**: the default; cheaper to unpack, and the pack measured on Apple Silicon.
- **PTQ1_0**: the smallest file; use it where memory is tightest.
- **Q2_0**: an alternative layout, and the one format upstream llama.cpp can also run (b11556 or newer).
### Speculative Decoding
The 2B ships without a draft model. At this size a decode step is already cheap, and in our tests speculative decoding with a draft head was slower than plain decoding on everyday prompts, so plain decoding is the recommended setup.
### License
Apache 2.0, following the base model.Sources: prism-ml/Ternary-Bonsai-2-2B-gguf on Hugging Face · bonsai-2:2b on Ollama. Collected from public sources on 8 Nov 2026; the host serving it may run a different build or quantisation — the marketplace shows each host's own measurements.
Who's serving it.
Every host benchmarked this model on its own hardware before serving it. Prices are set by each host; health is scored by the platform from what it observes.
| Host | Tier | Price | Measured speed | Health | State | Rent |
|---|---|---|---|---|---|---|
| 7B-A1B (5G) | L1 | £0.057 in · £0.23 out | 20 tok/s · 135 ms first token | 82 | loads on request | £0.04/h |
Live data, refreshed on every visit. How billing works.
Room on the grid for your machine.
Right now one machine serves bonsai-2:2b. If it drops, so does every request for it — a second host is what turns it into failover, and the grid needs that more than another copy. The router weighs price, measured speed, health and whether the model is already loaded — so a host with a sharper price or quicker hardware wins the next call.
Set your own price — you keep 85% of what it earns, and new hosts start with 12 months fee-free (up to £250 of earnings). A payout fee of £2 + 1% per payout to a UK bank account, £2 + 2% to any other country applies when you're paid. Install the connector, load bonsai-2:2b, and the platform benchmarks it on your hardware before your first request.