Marketplace · Text & chat
bonsai-2:9b
Call bonsai-2:9b through one OpenAI-compatible API, served by independent machines at prices their owners set. A host has just loaded bonsai-2:9b and the platform is benchmarking it on their hardware; it goes live when that finishes.
The grid is one machine short.
bonsai-2:9b is about 3.0 GB of weights, so a machine with a little more than that in free memory — a gaming GPU, an Apple-silicon Mac, a well-specced mini PC — can usually serve it. Set your own price — you keep 85% of what it earns, and new hosts start with 12 months fee-free (up to £250 of earnings). A payout fee of £2 + 1% per payout to a UK bank account, £2 + 2% to any other country applies when you're paid.
- Install the connector. One command on the machine that runs your models. It dials out over an encrypted link — no ports to open. The connector
- Load
bonsai-2:9b. It has run here on ollama; any supported engine works. The platform benchmarks it on your hardware first. - Set your price. The last host listed it from £0.098 per 1M output tokens. Developers see your price beside the speed your machine measures.
- Get paid when it's called. Switch it off whenever you like. How hosting works
About bonsai-2:9b.
vision tools reasoning
- Type: Text & chat
- Size: 9B parameters, about 3.0 GB of weights
- Context window: up to 262,144 tokens
- Runs on: ollama
- Last listed from: £0.098 per 1M output tokens
- First on the grid: 1 Nov 2026
- Hosts serving it now: 0
from openai import OpenAI
client = OpenAI(base_url="https://inpherio.co.uk/v1",
api_key=INPHERIO_API_KEY)
reply = client.chat.completions.create(
model="bonsai-2:9b",
messages=[{"role": "user", "content": "Hello"}],
)bonsai-2:9b in detail.
Bonsai 2 from Prism ML is a family of vision models built for ternary weights – smaller models with close to full-precision quality.
- Published by
- prism-ml
- Parameters
- 9.4B
- Download size
- 3.2 GB (as packaged for Ollama)
- Quantisation
- Q2_0
- Format
- gguf
- Architecture
- prism_hadamard_qwen35
- Licence
- Apache 2.0
- Based on
- Qwen/Qwen3.5-9B
- Task
- image text to text
- Library
- mlx
- First published
- 1 Oct 2026
- Last updated
- 14 Oct 2026
- Downloads (30 days)
- 15,171 on Hugging Face
- Likes
- 22
mlx prism_hadamard_qwen35 ternary 2-bit metal on-device hybrid-attention vision prismml bonsai image-text-to-text
The model card
Published by the model's authors on Hugging Face — shown as they wrote it, unedited by Inpherio.
Prism ML Website | Demo & Examples | Discord
Bonsai 2 Ternary 9B: MLX
A 9B-class vision-language model with ternary transformer weights, packed for MLX on Apple Silicon.
3.46 GB on disk, language model + vision tower (FP16 language model: ~17.9 GB)
Highlights
- Ternary language weights stored in MLX's 2-bit grouped format (group 128). MLX stores a scale and a bias per group, so the container costs a little more than the GGUF packings; the weights decode to exactly the same ternary values.
- Vision input through the official vision tower, carried unrotated in FP16.
- 262K-token context, inherited from the Qwen3.5-9B hybrid-attention backbone (24 linear-attention and 8 full-attention layers).
Model Overview
| Item | Specification |
|---|---|
| Base model | Derived from Qwen3.5-9B (architecture unchanged) |
| Parameters | ~8.95B language model (32 blocks, hidden 4096) + ~0.46B vision tower |
| Weight format | Ternary {−1, 0, +1} in MLX affine 2-bit, group 128 |
| Weight basis | Blockwise Hadamard rotation folded into the stored weights; the bundled runtime applies the matching transform to activations |
| Backends | MLX on Apple Silicon (Metal): mlx-vlm 0.7.8+ natively, or the loader in runtime/ |
| License | Apache 2.0 |
Files
| File | Contents | Size |
|---|---|---|
model.safetensors | language model (2-bit, group 128) + vision tower (FP16) | 3.46 GB |
mtp/ | multi-token-prediction draft head, 4-bit (optional, see below) | 0.14 GB |
runtime/ | a bundled loader for older mlx-vlm versions, plus pinned requirements.txt | |
quickstart.py | one-command test with the recommended sampler | |
PACK-RUNTIME.md | the bundled loader's contract and limits |
Quickstart
**PrismML-Eng/Bonsai-demo is the source of truth for running these models.** Where anything here disagrees with it, it is right.
hf download prism-ml/Ternary-Bonsai-2-9B-mlx-2bit --local-dir bonsai2-9b-mlx
pip install "mlx-vlm>=0.7.8"
mlx-vlm 0.7.8 and newer load this pack natively (model_type: prism_hadamard_qwen35):
from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
model, processor = load("bonsai2-9b-mlx")
prompt = apply_chat_template(processor, model.config, "What is in this image?", num_images=1)
print(generate(model, processor, prompt, ["photo.jpg"], max_tokens=4096,
temperature=1.0, top_p=0.95, top_k=20))
For an OpenAI-compatible server, run python -m mlx_vlm.server --model bonsai2-9b-mlx and send the sampling settings in each request.
The released mlx-lm (0.32.0) cannot load this pack yet; its support is merged and arrives with the next mlx-lm release. On older mlx-vlm versions, use the loader bundled in runtime/ (pip install -r bonsai2-9b-mlx/runtime/requirements.txt, then python bonsai2-9b-mlx/quickstart.py "your prompt"); PACK-RUNTIME.md documents it. quickstart.py is a smoke test: it answers with thinking off and stops at 256 tokens.
Pass no images for text-only use.
For CUDA, CPU, and llama.cpp on Metal, use the GGUF packs of the same weights: **Ternary-Bonsai-2-9B-gguf**.
Best Practices
Generation Parameters
We recommend these sampling settings (the same as the GGUF packs):
- Thinking mode (default):temperature=1.0,top_p=0.95,top_k=20,min_p=0.0,presence_penalty=1.0,repetition_penalty=1.0- Non-thinking mode:temperature=0.7,top_p=0.80,top_k=20,min_p=0.0,presence_penalty=1.5,repetition_penalty=1.0
Pass the sampler explicitly. mlx-vlm's Python generate() decodes greedily unless you pass these arguments. mlx_vlm.server takes temperature, top_p and top_k from generation_config.json when a request omits them, but not presence penalty or min_p, so send the full set per request. Apply penalties over the last 64 tokens where your client supports a penalty window. Avoid greedy decoding with thinking on; it can get stuck reasoning on some prompts. To turn thinking off, render the chat template with enable_thinking=False.
Read the rest of the model card
Output Length
Give thinking mode room to finish: even simple prompts can use 3,000+ thinking tokens, so allow at least 8,192 output tokens with thinking on. For short answers, turn thinking off.
System Prompt
You can use a simple system prompt such as:
You are a helpful assistant
Speculative Decoding (optional)
mtp/ holds this model's trained multi-token-prediction head as an mlx-vlm drafter. The main model verifies every drafted token, so the output matches plain decoding (up to float16 near-ties). It helps most on copy-heavy work such as code edits and rewrites; on general chat it is about as fast as plain decoding or slightly slower, so plain decoding stays the default. MTP.md has the code. It was validated with the bundled runtime on mlx-vlm 0.6.3 and has not yet been re-checked on 0.7.8.
License
Apache 2.0, following the base model.
Sources: prism-ml/Ternary-Bonsai-2-9B-mlx-2bit on Hugging Face · bonsai-2:9b on Ollama. Collected from public sources on 8 Nov 2026; the host serving it may run a different build or quantisation — the marketplace shows each host's own measurements.
Live now instead.
Similar text & chat models with hosts serving them right now.
- bonsai-2:4b — from £0.06 per 1M output tokens · 2 hosts
- bonsai-2:2b — from £0.23 per 1M output tokens · 1 host
- qwen2.5:0.5b — from £0.009 per 1M output tokens · 5 hosts
- functiongemma:latest — from £0.015 per 1M output tokens · 4 hosts
- glm-ocr:latest — from £0.06 per 1M output tokens · 3 hosts
- granite4.2:3b — from £0.06 per 1M output tokens · 3 hosts
Questions about bonsai-2:9b.
Can I use bonsai-2:9b through an API?
Yes. Inpherio serves bonsai-2:9b through one OpenAI-compatible API whenever a host on the grid is running it — the code on this page works unchanged. With no host right now, a call returns a "not available on any node" error and is never charged, and similar models are live on the marketplace.
How do I host bonsai-2:9b and get paid?
Install the Inpherio connector on the machine that runs your models, load bonsai-2:9b in a supported engine, and set a price. The platform benchmarks it on your hardware, then requests for it reach your machine. Set your own price — you keep 85% of what it earns, and new hosts start with 12 months fee-free (up to £250 of earnings). A payout fee of £2 + 1% per payout to a UK bank account, £2 + 2% to any other country applies when you're paid.
What hardware does bonsai-2:9b need?
Its weights are about 3.0 GB, so plan for a little more than that in free GPU or unified memory. Many gaming GPUs, Apple-silicon Macs and mini PCs qualify.
Why is bonsai-2:9b not being served right now?
Hosts on Inpherio are independent machines that come and go. It was last live on 11 Nov 2026. When a host loads it again, this page shows live prices and measured speeds.