Marketplace · Text & chat

bonsai-2:4b

Call bonsai-2:4b through one OpenAI-compatible API. It's served by 2 hosts on the Inpherio grid, each at a price it sets — your request goes to the cheapest healthy one, and fails over to the next if it's down.

From
£0.06
per 1M output tokens
£1 buys about 17 million tokens (≈ 13 million words)
Hosts
2
Loaded now
0

At a glance.

vision tools reasoning

  • Type: Text & chat
  • Hosts serving it now: 2 (0 with it loaded in memory)
  • Engines: ollama
  • Context window: up to 262,144 tokens
  • Rentable: a whole host from £0.04 an hour (rentals)
python · openai sdk
from openai import OpenAI
client = OpenAI(base_url="https://inpherio.co.uk/v1",
                api_key=INPHERIO_API_KEY)
reply = client.chat.completions.create(
    model="bonsai-2:4b",
    messages=[{"role": "user", "content": "Hello"}],
)

bonsai-2:4b in detail.

Bonsai 2 from Prism ML is a family of vision models built for ternary weights – smaller models with close to full-precision quality.

Published by
prism-ml
Parameters
4.5B
Download size
1.6 GB (as packaged for Ollama)
Quantisation
Q2_0
Format
gguf
Architecture
prism_hadamard_qwen35
Licence
Apache 2.0
Based on
Qwen/Qwen3.5-4B
Task
image text to text
Library
mlx
First published
1 Oct 2026
Last updated
15 Oct 2026
Downloads (30 days)
9,909 on Hugging Face
Likes
10

mlx prism_hadamard_qwen35 ternary 2-bit metal on-device hybrid-attention vision prismml bonsai image-text-to-text

The model card

Published by the model's authors on Hugging Face — shown as they wrote it, unedited by Inpherio.

Prism ML Website  |  Demo & Examples  |  Discord

Bonsai 2 Ternary 4B: MLX

A 4B-class vision-language model with ternary transformer weights, packed for MLX on Apple Silicon.

1.87 GB on disk, language model + vision tower (FP16 language model: ~8.4 GB)

Highlights

  • Ternary language weights stored in MLX's 2-bit grouped format (group 128). MLX stores a scale and a bias per group, so the container costs a little more than the GGUF packings; the weights decode to exactly the same ternary values.
  • Vision input through the official vision tower, carried unrotated in FP16.
  • 262K-token context, inherited from the Qwen3.5-4B hybrid-attention backbone (24 linear-attention and 8 full-attention layers).

Model Overview

ItemSpecification
Base modelDerived from Qwen3.5-4B (architecture unchanged)
Parameters~4.21B language model (32 blocks, hidden 2560) + ~0.33B vision tower
Weight formatTernary {−1, 0, +1} in MLX affine 2-bit, group 128
Weight basisBlockwise Hadamard rotation folded into the stored weights; the bundled runtime applies the matching transform to activations
BackendsMLX on Apple Silicon (Metal): mlx-vlm 0.7.8+ natively, or the loader in runtime/
LicenseApache 2.0

Files

FileContentsSize
model.safetensorslanguage model (2-bit, group 128) + vision tower (FP16)1.87 GB
mtp/multi-token-prediction draft head, 4-bit (optional, see below)0.07 GB
runtime/a bundled loader for older mlx-vlm versions, plus pinned requirements.txt
quickstart.pyone-command test with the recommended sampler
PACK-RUNTIME.mdthe bundled loader's contract and limits

Quickstart

**PrismML-Eng/Bonsai-demo is the source of truth for running these models.** Where anything here disagrees with it, it is right.
hf download prism-ml/Ternary-Bonsai-2-4B-mlx-2bit --local-dir bonsai2-4b-mlx
pip install "mlx-vlm>=0.7.8"

mlx-vlm 0.7.8 and newer load this pack natively (model_type: prism_hadamard_qwen35):

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template

model, processor = load("bonsai2-4b-mlx")
prompt = apply_chat_template(processor, model.config, "What is in this image?", num_images=1)
print(generate(model, processor, prompt, ["photo.jpg"], max_tokens=4096,
               temperature=1.0, top_p=0.95, top_k=20))

For an OpenAI-compatible server, run python -m mlx_vlm.server --model bonsai2-4b-mlx and send the sampling settings in each request.

The released mlx-lm (0.32.0) cannot load this pack yet; its support is merged and arrives with the next mlx-lm release. On older mlx-vlm versions, use the loader bundled in runtime/ (pip install -r bonsai2-4b-mlx/runtime/requirements.txt, then python bonsai2-4b-mlx/quickstart.py "your prompt"); PACK-RUNTIME.md documents it. quickstart.py is a smoke test: it answers with thinking off and stops at 256 tokens.

Pass no images for text-only use.

For CUDA, CPU, and llama.cpp on Metal, use the GGUF packs of the same weights: **Ternary-Bonsai-2-4B-gguf**.

Best Practices

Generation Parameters

We recommend these sampling settings (the same as the GGUF packs):

- Thinking mode (default): temperature=1.0, top_p=0.95, top_k=20, min_p=0.0, presence_penalty=1.0, repetition_penalty=1.0 - Non-thinking mode: temperature=0.7, top_p=0.80, top_k=20, min_p=0.0, presence_penalty=1.5, repetition_penalty=1.0

Pass the sampler explicitly. mlx-vlm's Python generate() decodes greedily unless you pass these arguments. mlx_vlm.server takes temperature, top_p and top_k from generation_config.json when a request omits them, but not presence penalty or min_p, so send the full set per request. Apply penalties over the last 64 tokens where your client supports a penalty window. Avoid greedy decoding with thinking on; it can get stuck reasoning on some prompts. To turn thinking off, render the chat template with enable_thinking=False.

Read the rest of the model card

Output Length

Give thinking mode room to finish: even simple prompts can use 3,000+ thinking tokens, so allow at least 8,192 output tokens with thinking on. For short answers, turn thinking off.

System Prompt

You can use a simple system prompt such as:

You are a helpful assistant

Speculative Decoding (optional)

mtp/ holds this model's trained multi-token-prediction head as an mlx-vlm drafter. The main model verifies every drafted token, so the output matches plain decoding (up to float16 near-ties). It helps most on copy-heavy work such as code edits and rewrites; on general chat it is about as fast as plain decoding or slightly slower, so plain decoding stays the default. MTP.md has the code. It was validated with the bundled runtime on mlx-vlm 0.6.3 and has not yet been re-checked on 0.7.8.

License

Apache 2.0, following the base model.

Sources: prism-ml/Ternary-Bonsai-2-4B-mlx-2bit on Hugging Face · bonsai-2:4b on Ollama. Collected from public sources on 8 Nov 2026; the host serving it may run a different build or quantisation — the marketplace shows each host's own measurements.

Who's serving it.

Every host benchmarked this model on its own hardware before serving it. Prices are set by each host; health is scored by the platform from what it observes.

Hosts serving bonsai-2:4b right now
HostTierPriceMeasured speedHealthStateRent
7B-A1B (5G)L1£0.12 in · £0.47 out9 tok/s · 257 ms first token82loads on request£0.04/h
W11BOXL1£0.03 in · £0.06 out8 tok/s · 313 ms first token77loads on request—

Live data, refreshed on every visit. How billing works.

Room on the grid for your machine.

2 machines serve bonsai-2:4b today. Every extra host is somewhere a request can fail over to when one drops, and spare capacity when demand spikes. The router weighs price, measured speed, health and whether the model is already loaded — so a host with a sharper price or quicker hardware wins the next call.

Hosts now
2
0 loaded in memory
Price to beat
£0.06
per 1M output tokens
Fastest measured
9 tok/s

Set your own price — you keep 85% of what it earns, and new hosts start with 12 months fee-free (up to £250 of earnings). A payout fee of £2 + 1% per payout to a UK bank account, £2 + 2% to any other country applies when you're paid. Install the connector, load bonsai-2:4b, and the platform benchmarks it on your hardware before your first request.