Inpherio for developers
One API for every model on the grid.
Every open LLM, embedding, and speech model on the grid sits behind a single OpenAI-compatible endpoint. Keep your SDK, swap the base URL, and pay competition prices in pounds — with automatic failover and spend caps you set yourself.
Integration is a base URL.
If your code already talks to OpenAI, it already talks to Inpherio. Point the client at the grid, set your budget and quality preferences once in the dashboard, and they apply to every request. No special headers or body extensions required.
curl https://inpherio.co.uk/v1/chat/completions \
-H "Authorization: Bearer $INPHERIO_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "llama3.1:8b",
"stream": true,
"messages": [{"role": "user", "content": "Hello, grid"}]
}'from openai import OpenAI
client = OpenAI(
base_url="https://inpherio.co.uk/v1",
api_key=INPHERIO_API_KEY,
)
stream = client.chat.completions.create(
model="qwen3:8b",
messages=[{"role": "user", "content": "Refactor this loop"}],
stream=True,
)vec = client.embeddings.create(
model="nomic-embed-text:latest",
input="chunks to index for retrieval",
)speech = client.audio.speech.create(
model="speaches-ai/Kokoro-82M-v1.0-ONNX",
voice="af_heart",
input="Hello from the Inpherio grid.",
)
speech.write_to_file("audio.mp3")
text = client.audio.transcriptions.create(
model="Systran/faster-whisper-small",
file=open("audio.mp3", "rb"),
)from anthropic import Anthropic
claude_style = Anthropic(
base_url="https://inpherio.co.uk", # no /v1 — the SDK adds it
api_key=INPHERIO_API_KEY,
)
msg = claude_style.messages.create(
model="qwen3:8b",
max_tokens=512,
messages=[{"role": "user", "content": "Summarise this diff"}],
)from ollama import Client
grid = Client(
host="https://inpherio.co.uk",
headers={"Authorization": f"Bearer {INPHERIO_API_KEY}"},
)
reply = grid.chat(
model="llama3.1:8b",
messages=[{"role": "user", "content": "Hello, grid"}],
)Node.js, Go, Rust and any other OpenAI SDK work the same way — set the base URL and go. The API key is a Bearer token; keep it server-side.
Every workload, one key.
Supported endpoints
Every model is reachable through standard, OpenAI-compatible routes:
/v1/chat/completions— text generation, streaming or not./v1/embeddings— vector embeddings for search and RAG./v1/audio/speech— text-to-speech./v1/audio/transcriptions— speech-to-text./v1/systemone— decision models: classify, answer yes/no and score text./v1/models— list everything live on the grid right now.
Formats we also speak
Not on the OpenAI SDK? The grid answers two more request shapes natively:
/v1/messages— Anthropic Messages format, for the Anthropic SDK./api/chat— Ollama's native chat format, for existing Ollama clients.
Chat endpoints stream token by token; embeddings and audio return a complete response. Use /v1/models to discover exact model IDs before you hard-code them.
Decision models.
Some models don't chat: they read a piece of text and answer named questions about it — a choice, a score, or a number — with a probability for every answer. Good for routing tickets, moderation, triage and policy checks. They have their own endpoint, POST /v1/systemone, and are listed under Decision on the marketplace when a host offers one.
import requests
r = requests.post(
"https://inpherio.co.uk/v1/systemone",
headers={"Authorization": "Bearer YOUR_API_KEY"},
json={
"model": "MODEL_ID",
"state": "My order arrived broken - can I get my money back?",
"questions": {
"refund": {"instructions": "Is the customer asking for a refund?",
"type": "choice",
"criteria": {"yes": "they ask for their money back",
"no": "anything else"}},
"mood": {"instructions": "How does the customer feel?",
"type": "score",
"criteria": ["angry", "unhappy", "neutral", "happy"]},
},
},
)
print(r.json()["answers"])What you send, what you get
state— the text to decide about (up to 32,000 characters).questions— 1 to 64, each withinstructionsand atype:choice(withcriteriaas label → description) returnschoice,probabilitiesandconfidence;score(with 2–10 levels) returns ascorefrom 0 to 1 with itslegend;noulreturns a number.- Billed per input token at the model's input price, on the host engine's own count (
usage.input_tokens). The engine wraps every request in a template of roughly 150 tokens, plus more for each question. - A malformed question comes back as HTTP 400 with the engine's reason, and nothing is charged. A decision model called through chat or embeddings returns HTTP 400 pointing here.
Live in four steps.
Add credit
Top up your prepaid balance with a debit/credit card via Stripe, or redeem a voucher. There's no subscription and no minimum spend — you pay for requests as you make them.
Create a key
Generate an API key in the dashboard. Give each app its own key with its own spend budget, optionally an expiry date, and revoke any of them instantly.
Point your SDK
Set
base_urltohttps://inpherio.co.uk/v1and pass the key as a Bearer token. Nothing else in your code changes.Send a request
Call any model on the grid. The edge proxy picks the best-scoring healthy provider inside your caps (whether the model is loaded, reliability, price and speed), meters the request, and writes an itemised ledger entry you can read back any time.
Spend that can't surprise you.
Billing is prepaid and metered to the token. You can only ever spend what you've loaded, and you set the ceilings.
- Prepaid, never in debt. Before every request we hold what it can cost. If your balance can't pay for a whole reply, the reply is capped to what it can — it stops there with
finish_reason: "length", the headerX-Inpherio-Stopped: balanceand a short closing note in plain-text replies — and you pay for what you got. If it can't cover even the prompt, or a speech or embedding request in full, the request is refused with402and nothing runs. - Per-key budgets. Cap how much any individual API key can spend, so a leaked or runaway key is contained.
- Daily and monthly spend caps. Set hard limits across your whole account. Once hit, requests are refused with
429until the period resets. - Priced per token, in pounds. Metered on exactly what you use — provider rates per 1M input and 1M output tokens for text, per 1M input for embeddings, or per hour of audio for speech (billed on the length of the audio, to the second). No base fee, and no platform markup on the provider's price.
- Metered on completion. Streaming requests settle when the stream ends; the unused part of the hold is released, and a charge can never take your balance below zero. Failed requests aren't billed at all.
- An itemised ledger entry for every request — model, tokens, and the exact charge. You always know what you paid for.
You choose which nodes serve you.
Set these once in the dashboard and they apply to every request — the router never exceeds your limits, and never picks a node you've excluded.
- Max price per 1M tokens — reject any listing priced above your ceiling.
- Minimum health score — require proven reliability before a node is eligible.
- Maximum tier — restrict routing to benchmarked performance tiers (L1 is fastest).
- Warm-only — prefer nodes with the model already loaded, for the lowest time-to-first-token.
Need to override a preference for a single call? Two headers take precedence over your dashboard defaults, per request:
X-Inpherio-Budget-Per-1M-Tokens | Price ceiling per 1M output tokens for this request. The header takes pence (the API's unit; decimals allowed) — 40 means £0.40, 0.5 means £0.005. Nodes above it are skipped. |
|---|---|
X-Inpherio-Min-Reliability | Minimum health score (0–100) a node must have to serve this request. |
One key per job: routing policies on API keys
Make up to 50 API keys per account and give each its own routing — a batch key that always takes the cheapest provider, a chat key that wants the fastest, a production key that only accepts high-health L1 nodes and two named models. Set it under API Keys → Routing… in your dashboard, with a live preview of which providers would serve each model.
| Pick providers by | Balanced (default) · Cheapest · Fastest (measured tokens/second) · Most reliable (health score) |
|---|---|
| Price limits | Separate maximums for input and output price per 1M tokens |
| Quality limits | Minimum health score, performance tier, minimum tokens/second, minimum context window, model already loaded |
| Allowed models | e.g. qwen*, llama3.1:8b — requests for anything else are refused with a clear error |
Precedence, most specific first: request headers → the key's policy → your per-model profiles → your account defaults. If no provider qualifies, the request fails with an error that names the key and its policy — you never silently get a different model or a node outside your limits.
Your dashboard · Routing & budgets
Decide what your money is spent on.
In your dashboard you set, once, the rules every request follows: a ceiling on the price per 1M input and output tokens, a minimum reliability (health score) for the hosts that serve you, whether to use only hosts with the model already loaded, and hard daily and monthly spend caps. The router never goes past them. Try the real Routing & budgets card below — the list under it narrows, live, to the models on the marketplace that fit your limits.
Every model live on the grid right now.
Move the limits above and this list narrows to exactly the pool the router would choose from. Pick any model to see its hosts, prices and measured speeds.
67 text models live on the grid right now, cheapest first.
| Model | Input from | Output from | Fastest measured | State |
|---|---|---|---|---|
| qwen2.5:0.5b | £0.005 | £0.009 | 371 tok/s | on request |
| smollm2:360m | £0.005 | £0.009 | 117 tok/s | on request |
| functiongemma:latest | £0.0072 | £0.015 | 77 tok/s | on request |
| sileader/qwen3guard:0.6b | £0.015 | £0.015 | 42 tok/s | on request |
| llama-guard3:1b | £0.018 | £0.018 | 135 tok/s | on request |
| qwen2.5:7b-instruct | £0.012 | £0.027 | 21 tok/s | on request |
| qwen3:1.7b | £0.013 | £0.027 | 210 tok/s | on request |
| granite3-guardian:2b | £0.029 | £0.029 | 19 tok/s | on request |
| phi3:mini | £0.018 | £0.036 | 42 tok/s | on request |
| lfm3:latest | £0.022 | £0.043 | 77 tok/s | on request |
| rafunga/qwen3guard-gen:4b | £0.043 | £0.043 | 8 tok/s | on request |
| moondream:1.8b | £0.018 | £0.045 | 106 tok/s | on request |
| deepseek-r1:8b | £0.022 | £0.05 | 68 tok/s | on request |
| llama3.2:latest | £0.023 | £0.054 | 45 tok/s | on request |
| mistral:7b | £0.027 | £0.054 | 22 tok/s | on request |
| lfm3-vl:latest | £0.029 | £0.058 | 34 tok/s | on request |
| bespoke-minicheck:7b | £0.03 | £0.06 | 7 tok/s | on request |
| bonsai-2:4b | £0.03 | £0.06 | 9 tok/s | on request |
| deepseek-r1:14b | £0.03 | £0.06 | 25 tok/s | on request |
| falcon3:1b | £0.03 | £0.06 | 24 tok/s | on request |
| falcon3:3b | £0.03 | £0.06 | 16 tok/s | on request |
| falcon3:7b | £0.03 | £0.06 | 6 tok/s | on request |
| gemma3:12b | £0.03 | £0.06 | 26 tok/s | on request |
| gemma4:e2b | £0.03 | £0.06 | 15 tok/s | on request |
| glm-ocr:latest | £0.03 | £0.06 | 26 tok/s | on request |
| granite3.1-moe:1b | £0.03 | £0.06 | 55 tok/s | on request |
| granite3.1-moe:3b | £0.03 | £0.06 | 34 tok/s | on request |
| granite3.3:2b | £0.03 | £0.06 | 12 tok/s | loaded |
| granite3.3:8b | £0.03 | £0.06 | 5 tok/s | on request |
| granite4.2:3b | £0.029 | £0.06 | 13 tok/s | loaded |
| llama3.1:8b | £0.01 | £0.06 | 26 tok/s | on request |
| mimo-v2.6:9b | £0.03 | £0.06 | 4 tok/s | on request |
| olmo2:7b | £0.03 | £0.06 | 5 tok/s | on request |
| phi4-mini:3.8b | £0.03 | £0.06 | 8 tok/s | loaded |
| qwen3.5:2b | £0.03 | £0.06 | 11 tok/s | on request |
| qwen3.5:4b | £0.03 | £0.06 | 8 tok/s | on request |
| qwen3:14b | £0.03 | £0.06 | 24 tok/s | on request |
| starcoder2:3b | £0.03 | £0.06 | 11 tok/s | loaded |
| qwen2.5-coder:7b | £0.032 | £0.063 | 21 tok/s | on request |
| gemma4:e4b | £0.038 | £0.075 | 64 tok/s | on request |
| qwen3:8b | £0.045 | £0.081 | 19 tok/s | on request |
| duckdb-nsql:7b | £0.043 | £0.086 | 5 tok/s | on request |
| granite4.1-guardian:8b | £0.086 | £0.086 | 3 tok/s | on request |
| shieldgemma:9b | £0.086 | £0.086 | 6 tok/s | on request |
| llama-guard3:8b | £0.095 | £0.095 | 10 tok/s | on request |
| aya-expanse:8b | £0.058 | £0.12 | 5 tok/s | on request |
| deepseek-ocr:latest | £0.058 | £0.12 | 19 tok/s | on request |
| dolphin3:8b | £0.058 | £0.12 | 6 tok/s | on request |
| gemma4:12b | £0.038 | £0.12 | 4 tok/s | on request |
| granite4.2:8b | £0.045 | £0.12 | 5 tok/s | on request |
| hermes3:8b | £0.058 | £0.12 | 5 tok/s | on request |
| medgemma:4b | £0.058 | £0.12 | 9 tok/s | on request |
| sqlcoder:7b | £0.058 | £0.12 | 6 tok/s | on request |
| nemotron-3.5-bolt:e4b | £0.053 | £0.15 | 4 tok/s | on request |
| qwen2.5:7b | £0.075 | £0.15 | 6 tok/s | on request |
| richardyoung/olmocr2:7b-q8 | £0.072 | £0.15 | 3 tok/s | on request |
| qwen2.5:14b | £0.09 | £0.18 | 3 tok/s | on request |
| qwen3.5:9b | £0.13 | £0.19 | 4 tok/s | on request |
| qwen3.5:0.8b | £0.05 | £0.20 | 22 tok/s | on request |
| shieldgemma:2b | £0.049 | £0.20 | 23 tok/s | on request |
| bonsai-2:2b | £0.057 | £0.23 | 20 tok/s | on request |
| qwen3-vl:2b | £0.063 | £0.25 | 18 tok/s | on request |
| qwen3-vl:4b | £0.14 | £0.53 | 8 tok/s | on request |
| qwen3-vl:8b | £0.19 | £0.56 | 4 tok/s | on request |
| translategemma:4b | £0.15 | £0.59 | 8 tok/s | on request |
| olmoe-7b-colibri | £0.17 | £0.65 | 7 tok/s | loaded |
| qwen3.6-colibri | £0.22 | £0.85 | 10 tok/s | loaded |
Nothing on the grid fits these limits right now — the router would refuse with an error that says which limit.
Also live: embeddings and speech
- Qwen/Qwen3-ASR-0.6B Speech-to-text
- Qwen/Qwen3-ASR-1.7B Speech-to-text
- Qwen/Qwen3-TTS-12Hz-0.6B-CustomVoice Text-to-speech
- ResembleAI/chatterbox-turbo Text-to-speech
- Systran/faster-whisper-base Speech-to-text
- Systran/faster-whisper-large-v3 Speech-to-text
- Systran/faster-whisper-medium Speech-to-text
- Systran/faster-whisper-small Speech-to-text
- Systran/faster-whisper-tiny Speech-to-text
- all-minilm:22m Embeddings
- bge-m3:latest Embeddings
- d1:3b Decision (classify · yes/no · score)
- d3:2b Decision (classify · yes/no · score)
- d3:4b Decision (classify · yes/no · score)
- embeddinggemma-2:270m Embeddings
- embeddinggemma-2:440m Embeddings
- embeddinggemma:latest Embeddings
- granite-embedding:278m Embeddings
- granite-embedding:30m Embeddings
- istupakov/parakeet-tdt-0.6b-v3-onnx Speech-to-text
- mxbai-embed-large:latest Embeddings
- nimble:9b Decision (classify · yes/no · score)
- nomic-embed-text:latest Embeddings
- qwen3-embedding:0.6b Embeddings
- qwen3-embedding:4b Embeddings
- qwen3-embedding:8b Embeddings
- snowflake-arctic-embed2:latest Embeddings
- speaches-ai/Kokoro-82M-v1.0-ONNX Text-to-speech
- speaches-ai/piper-en_GB-alan-medium Text-to-speech
- speaches-ai/piper-en_GB-cori-high Text-to-speech
- speaches-ai/piper-en_US-lessac-high Text-to-speech
- speaches-ai/piper-en_US-ryan-high Text-to-speech
Live from the marketplace, refreshed every 30 seconds. Prices per 1M tokens.
Failover is automatic.
Residential nodes come and go; your requests shouldn't. Two layers of failover keep a single flaky provider from becoming your error.
Before dispatch
If the chosen node is unreachable when your request goes out, the proxy retries the next-best node automatically. You see a brief latency spike, not an error.
Mid-stream
If a node fails before it has sent any of the reply, the proxy sends your request to the next-best node and you see a brief pause. If it fails after the reply has started arriving, the stream ends with a clear error event instead of being replayed — a replay would send you the start of the reply twice — and nothing is charged for it; retry and the request routes around the failed node.
Free models
Start without a card.
Models marked Free* on the marketplace cost nothing. Call them with any API key, exactly like a paid model — same endpoints, same SDKs.
- Limits per model: up to 50 calls an hour for each account, and a reply-length and prompt-size cap shown on the model's listing. 100 free calls a day across all free models.
- Past a limit, nothing breaks: the call goes through billed at the listed price if you have credit, or comes back
402with that price if you don't. - Every response says which:
X-Inpherio-Free-Tier: free, orpaid; reason=…with the limit that applied. - Free calls are content-checked, and a paid request for the same hardware comes first. Free usage terms.
Every response tells you what happened.
Response headers make each request auditable in real time — which model actually served you, what it cost, and whether failover kicked in.
X-Inpherio-Model | The exact model that served the request. |
|---|---|
X-Inpherio-Charged-GBP | What this request cost, in pounds (8 decimal places). Non-streamed responses only — a stream's charge is settled when it ends; see the receipt in your dashboard or /api/consumer/usage. |
X-Inpherio-Charged-Cents | The same charge in micro-pence (1p = 1,000,000) — the unit the ledger stores, for exact integer accounting. |
X-Inpherio-Billed-Tokens | Tokens counted for billing (input + output). |
X-Inpherio-Stream | Whether the response was streamed. |
X-Inpherio-Failover | true if the request was re-routed after a node failed. Non-streamed responses only — a stream's headers are sent before any failover can happen. |
X-Inpherio-Failed-Nodes | How many nodes failed before one completed the request. |
X-Inpherio-Audio-Seconds | Length of the audio: the speech produced (/v1/audio/speech) or the recording transcribed (/v1/audio/transcriptions). This, not processing time, is what you pay for. |
|---|---|
X-Inpherio-Billed-Seconds | Seconds charged: the audio length rounded up to the second, at least 1. |
X-Inpherio-Audio-Measured | true when the length was read from the audio itself; false when the format couldn't be read and it was estimated (TTS: from the text at a typical speaking rate; STT: from the node's measured speed). |
When something goes wrong.
Errors use standard HTTP status codes, so your existing retry logic works. These are the ones you'll actually see.
400 | Malformed request — invalid JSON, a missing model, or a field of the wrong type. |
|---|---|
401 | Missing, invalid, revoked or expired API key. |
402 | Your balance can't cover the prompt (or a speech or embedding request in full) — nothing ran. Top up and retry; you're never put into debt. (A reply your balance only partly covers isn't refused: it stops where your credit ends — see above.) |
403 | A disabled account, or a key without the scope this request needs. (A prompt refused by content moderation comes back as an ordinary reply explaining the refusal, and isn't charged.) |
404 | No provider is currently serving that model, or the model ID is wrong. Check /v1/models. |
429 | Rate limit (60 requests/minute per key), a daily/monthly spend cap, or the key's own spend budget was hit (Retry-After says when it resets). Back off and retry. |
502 | The serving node errored and failover couldn't recover the request. Safe to retry. |
503 | The grid is temporarily at capacity for that model. Retry with backoff. |
Priced by competition, not by rate card.
Providers undercut each other under every listing, so the price of an open model falls as supply grows — and you pay the provider's rate with no platform markup on top. See what each model costs right now on the marketplace, or read the full fee schedule and worked examples on the pricing page.
Stop paying rate-card prices for open models.
Sign up, add credit, and make your first request in minutes — no waitlist.