Inpherio for developers

One API for every model on the grid.

Every open LLM, embedding, and speech model on the grid sits behind a single OpenAI-compatible endpoint. Keep your SDK, swap the base URL, and pay competition prices in pounds — with automatic failover and spend caps you set yourself.

Integration is a base URL.

If your code already talks to OpenAI, it already talks to Inpherio. Point the client at the grid, set your budget and quality preferences once in the dashboard, and they apply to every request. No special headers or body extensions required.

curl · chat completion
curl https://inpherio.co.uk/v1/chat/completions \
  -H "Authorization: Bearer $INPHERIO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llama3.1:8b",
    "stream": true,
    "messages": [{"role": "user", "content": "Hello, grid"}]
  }'
python · openai sdk, one line changed
from openai import OpenAI

client = OpenAI(
    base_url="https://inpherio.co.uk/v1",
    api_key=INPHERIO_API_KEY,
)

stream = client.chat.completions.create(
    model="qwen3:8b",
    messages=[{"role": "user", "content": "Refactor this loop"}],
    stream=True,
)
python · embeddings for RAG
vec = client.embeddings.create(
    model="nomic-embed-text:latest",
    input="chunks to index for retrieval",
)
python · speech (TTS & STT)
speech = client.audio.speech.create(
    model="speaches-ai/Kokoro-82M-v1.0-ONNX",
    voice="af_heart",
    input="Hello from the Inpherio grid.",
)
speech.write_to_file("audio.mp3")

text = client.audio.transcriptions.create(
    model="Systran/faster-whisper-small",
    file=open("audio.mp3", "rb"),
)
python · anthropic sdk
from anthropic import Anthropic

claude_style = Anthropic(
    base_url="https://inpherio.co.uk",  # no /v1 — the SDK adds it
    api_key=INPHERIO_API_KEY,
)

msg = claude_style.messages.create(
    model="qwen3:8b",
    max_tokens=512,
    messages=[{"role": "user", "content": "Summarise this diff"}],
)
python · ollama client
from ollama import Client

grid = Client(
    host="https://inpherio.co.uk",
    headers={"Authorization": f"Bearer {INPHERIO_API_KEY}"},
)

reply = grid.chat(
    model="llama3.1:8b",
    messages=[{"role": "user", "content": "Hello, grid"}],
)

Node.js, Go, Rust and any other OpenAI SDK work the same way — set the base URL and go. The API key is a Bearer token; keep it server-side.

Every workload, one key.

Supported endpoints

Every model is reachable through standard, OpenAI-compatible routes:

  • /v1/chat/completions — text generation, streaming or not.
  • /v1/embeddings — vector embeddings for search and RAG.
  • /v1/audio/speech — text-to-speech.
  • /v1/audio/transcriptions — speech-to-text.
  • /v1/systemone — decision models: classify, answer yes/no and score text.
  • /v1/models — list everything live on the grid right now.

Formats we also speak

Not on the OpenAI SDK? The grid answers two more request shapes natively:

  • /v1/messages — Anthropic Messages format, for the Anthropic SDK.
  • /api/chat — Ollama's native chat format, for existing Ollama clients.

Chat endpoints stream token by token; embeddings and audio return a complete response. Use /v1/models to discover exact model IDs before you hard-code them.

Decision models.

Some models don't chat: they read a piece of text and answer named questions about it — a choice, a score, or a number — with a probability for every answer. Good for routing tickets, moderation, triage and policy checks. They have their own endpoint, POST /v1/systemone, and are listed under Decision on the marketplace when a host offers one.

python · a decision
import requests
r = requests.post(
    "https://inpherio.co.uk/v1/systemone",
    headers={"Authorization": "Bearer YOUR_API_KEY"},
    json={
        "model": "MODEL_ID",
        "state": "My order arrived broken - can I get my money back?",
        "questions": {
            "refund": {"instructions": "Is the customer asking for a refund?",
                       "type": "choice",
                       "criteria": {"yes": "they ask for their money back",
                                    "no": "anything else"}},
            "mood": {"instructions": "How does the customer feel?",
                     "type": "score",
                     "criteria": ["angry", "unhappy", "neutral", "happy"]},
        },
    },
)
print(r.json()["answers"])

What you send, what you get

  • state — the text to decide about (up to 32,000 characters).
  • questions — 1 to 64, each with instructions and a type: choice (with criteria as label → description) returns choice, probabilities and confidence; score (with 2–10 levels) returns a score from 0 to 1 with its legend; noul returns a number.
  • Billed per input token at the model's input price, on the host engine's own count (usage.input_tokens). The engine wraps every request in a template of roughly 150 tokens, plus more for each question.
  • A malformed question comes back as HTTP 400 with the engine's reason, and nothing is charged. A decision model called through chat or embeddings returns HTTP 400 pointing here.

Live in four steps.

  1. Add credit

    Top up your prepaid balance with a debit/credit card via Stripe, or redeem a voucher. There's no subscription and no minimum spend — you pay for requests as you make them.

  2. Create a key

    Generate an API key in the dashboard. Give each app its own key with its own spend budget, optionally an expiry date, and revoke any of them instantly.

  3. Point your SDK

    Set base_url to https://inpherio.co.uk/v1 and pass the key as a Bearer token. Nothing else in your code changes.

  4. Send a request

    Call any model on the grid. The edge proxy picks the best-scoring healthy provider inside your caps (whether the model is loaded, reliability, price and speed), meters the request, and writes an itemised ledger entry you can read back any time.

Spend that can't surprise you.

Billing is prepaid and metered to the token. You can only ever spend what you've loaded, and you set the ceilings.

  • Prepaid, never in debt. Before every request we hold what it can cost. If your balance can't pay for a whole reply, the reply is capped to what it can — it stops there with finish_reason: "length", the header X-Inpherio-Stopped: balance and a short closing note in plain-text replies — and you pay for what you got. If it can't cover even the prompt, or a speech or embedding request in full, the request is refused with 402 and nothing runs.
  • Per-key budgets. Cap how much any individual API key can spend, so a leaked or runaway key is contained.
  • Daily and monthly spend caps. Set hard limits across your whole account. Once hit, requests are refused with 429 until the period resets.
  • Priced per token, in pounds. Metered on exactly what you use — provider rates per 1M input and 1M output tokens for text, per 1M input for embeddings, or per hour of audio for speech (billed on the length of the audio, to the second). No base fee, and no platform markup on the provider's price.
  • Metered on completion. Streaming requests settle when the stream ends; the unused part of the hold is released, and a charge can never take your balance below zero. Failed requests aren't billed at all.
  • An itemised ledger entry for every request — model, tokens, and the exact charge. You always know what you paid for.

You choose which nodes serve you.

Set these once in the dashboard and they apply to every request — the router never exceeds your limits, and never picks a node you've excluded.

  • Max price per 1M tokens — reject any listing priced above your ceiling.
  • Minimum health score — require proven reliability before a node is eligible.
  • Maximum tier — restrict routing to benchmarked performance tiers (L1 is fastest).
  • Warm-only — prefer nodes with the model already loaded, for the lowest time-to-first-token.

Need to override a preference for a single call? Two headers take precedence over your dashboard defaults, per request:

Per-request routing headers
X-Inpherio-Budget-Per-1M-TokensPrice ceiling per 1M output tokens for this request. The header takes pence (the API's unit; decimals allowed) — 40 means £0.40, 0.5 means £0.005. Nodes above it are skipped.
X-Inpherio-Min-ReliabilityMinimum health score (0–100) a node must have to serve this request.

One key per job: routing policies on API keys

Make up to 50 API keys per account and give each its own routing — a batch key that always takes the cheapest provider, a chat key that wants the fastest, a production key that only accepts high-health L1 nodes and two named models. Set it under API Keys → Routing… in your dashboard, with a live preview of which providers would serve each model.

What a key's policy can set
Pick providers byBalanced (default) · Cheapest · Fastest (measured tokens/second) · Most reliable (health score)
Price limitsSeparate maximums for input and output price per 1M tokens
Quality limitsMinimum health score, performance tier, minimum tokens/second, minimum context window, model already loaded
Allowed modelse.g. qwen*, llama3.1:8b — requests for anything else are refused with a clear error

Precedence, most specific first: request headers → the key's policy → your per-model profiles → your account defaults. If no provider qualifies, the request fails with an error that names the key and its policy — you never silently get a different model or a node outside your limits.

Your dashboard · Routing & budgets

Decide what your money is spent on.

In your dashboard you set, once, the rules every request follows: a ceiling on the price per 1M input and output tokens, a minimum reliability (health score) for the hosts that serve you, whether to use only hosts with the model already loaded, and hard daily and monthly spend caps. The router never goes past them. Try the real Routing & budgets card below — the list under it narrows, live, to the models on the marketplace that fit your limits.

Every model live on the grid right now.

Move the limits above and this list narrows to exactly the pool the router would choose from. Pick any model to see its hosts, prices and measured speeds.

67 text models live on the grid right now, cheapest first.

Text models live now, with the lowest listed prices per 1M tokens
ModelInput fromOutput fromFastest measuredState
qwen2.5:0.5b£0.005£0.009371 tok/son request
smollm2:360m£0.005£0.009117 tok/son request
functiongemma:latest£0.0072£0.01577 tok/son request
sileader/qwen3guard:0.6b£0.015£0.01542 tok/son request
llama-guard3:1b£0.018£0.018135 tok/son request
qwen2.5:7b-instruct£0.012£0.02721 tok/son request
qwen3:1.7b£0.013£0.027210 tok/son request
granite3-guardian:2b£0.029£0.02919 tok/son request
phi3:mini£0.018£0.03642 tok/son request
lfm3:latest£0.022£0.04377 tok/son request
rafunga/qwen3guard-gen:4b£0.043£0.0438 tok/son request
moondream:1.8b£0.018£0.045106 tok/son request
deepseek-r1:8b£0.022£0.0568 tok/son request
llama3.2:latest£0.023£0.05445 tok/son request
mistral:7b£0.027£0.05422 tok/son request
lfm3-vl:latest£0.029£0.05834 tok/son request
bespoke-minicheck:7b£0.03£0.067 tok/son request
bonsai-2:4b£0.03£0.069 tok/son request
deepseek-r1:14b£0.03£0.0625 tok/son request
falcon3:1b£0.03£0.0624 tok/son request
falcon3:3b£0.03£0.0616 tok/son request
falcon3:7b£0.03£0.066 tok/son request
gemma3:12b£0.03£0.0626 tok/son request
gemma4:e2b£0.03£0.0615 tok/son request
glm-ocr:latest£0.03£0.0626 tok/son request
granite3.1-moe:1b£0.03£0.0655 tok/son request
granite3.1-moe:3b£0.03£0.0634 tok/son request
granite3.3:2b£0.03£0.0612 tok/sloaded
granite3.3:8b£0.03£0.065 tok/son request
granite4.2:3b£0.029£0.0613 tok/sloaded
llama3.1:8b£0.01£0.0626 tok/son request
mimo-v2.6:9b£0.03£0.064 tok/son request
olmo2:7b£0.03£0.065 tok/son request
phi4-mini:3.8b£0.03£0.068 tok/sloaded
qwen3.5:2b£0.03£0.0611 tok/son request
qwen3.5:4b£0.03£0.068 tok/son request
qwen3:14b£0.03£0.0624 tok/son request
starcoder2:3b£0.03£0.0611 tok/sloaded
qwen2.5-coder:7b£0.032£0.06321 tok/son request
gemma4:e4b£0.038£0.07564 tok/son request
qwen3:8b£0.045£0.08119 tok/son request
duckdb-nsql:7b£0.043£0.0865 tok/son request
granite4.1-guardian:8b£0.086£0.0863 tok/son request
shieldgemma:9b£0.086£0.0866 tok/son request
llama-guard3:8b£0.095£0.09510 tok/son request
aya-expanse:8b£0.058£0.125 tok/son request
deepseek-ocr:latest£0.058£0.1219 tok/son request
dolphin3:8b£0.058£0.126 tok/son request
gemma4:12b£0.038£0.124 tok/son request
granite4.2:8b£0.045£0.125 tok/son request
hermes3:8b£0.058£0.125 tok/son request
medgemma:4b£0.058£0.129 tok/son request
sqlcoder:7b£0.058£0.126 tok/son request
nemotron-3.5-bolt:e4b£0.053£0.154 tok/son request
qwen2.5:7b£0.075£0.156 tok/son request
richardyoung/olmocr2:7b-q8£0.072£0.153 tok/son request
qwen2.5:14b£0.09£0.183 tok/son request
qwen3.5:9b£0.13£0.194 tok/son request
qwen3.5:0.8b£0.05£0.2022 tok/son request
shieldgemma:2b£0.049£0.2023 tok/son request
bonsai-2:2b£0.057£0.2320 tok/son request
qwen3-vl:2b£0.063£0.2518 tok/son request
qwen3-vl:4b£0.14£0.538 tok/son request
qwen3-vl:8b£0.19£0.564 tok/son request
translategemma:4b£0.15£0.598 tok/son request
olmoe-7b-colibri£0.17£0.657 tok/sloaded
qwen3.6-colibri£0.22£0.8510 tok/sloaded

Also live: embeddings and speech

Live from the marketplace, refreshed every 30 seconds. Prices per 1M tokens.

Failover is automatic.

Residential nodes come and go; your requests shouldn't. Two layers of failover keep a single flaky provider from becoming your error.

Before dispatch

If the chosen node is unreachable when your request goes out, the proxy retries the next-best node automatically. You see a brief latency spike, not an error.

Mid-stream

If a node fails before it has sent any of the reply, the proxy sends your request to the next-best node and you see a brief pause. If it fails after the reply has started arriving, the stream ends with a clear error event instead of being replayed — a replay would send you the start of the reply twice — and nothing is charged for it; retry and the request routes around the failed node.

Free models

Start without a card.

Models marked Free* on the marketplace cost nothing. Call them with any API key, exactly like a paid model — same endpoints, same SDKs.

  • Limits per model: up to 50 calls an hour for each account, and a reply-length and prompt-size cap shown on the model's listing. 100 free calls a day across all free models.
  • Past a limit, nothing breaks: the call goes through billed at the listed price if you have credit, or comes back 402 with that price if you don't.
  • Every response says which: X-Inpherio-Free-Tier: free, or paid; reason=… with the limit that applied.
  • Free calls are content-checked, and a paid request for the same hardware comes first. Free usage terms.

Every response tells you what happened.

Response headers make each request auditable in real time — which model actually served you, what it cost, and whether failover kicked in.

Response headers
X-Inpherio-ModelThe exact model that served the request.
X-Inpherio-Charged-GBPWhat this request cost, in pounds (8 decimal places). Non-streamed responses only — a stream's charge is settled when it ends; see the receipt in your dashboard or /api/consumer/usage.
X-Inpherio-Charged-CentsThe same charge in micro-pence (1p = 1,000,000) — the unit the ledger stores, for exact integer accounting.
X-Inpherio-Billed-TokensTokens counted for billing (input + output).
X-Inpherio-StreamWhether the response was streamed.
X-Inpherio-Failovertrue if the request was re-routed after a node failed. Non-streamed responses only — a stream's headers are sent before any failover can happen.
X-Inpherio-Failed-NodesHow many nodes failed before one completed the request.
Speech response headers
X-Inpherio-Audio-SecondsLength of the audio: the speech produced (/v1/audio/speech) or the recording transcribed (/v1/audio/transcriptions). This, not processing time, is what you pay for.
X-Inpherio-Billed-SecondsSeconds charged: the audio length rounded up to the second, at least 1.
X-Inpherio-Audio-Measuredtrue when the length was read from the audio itself; false when the format couldn't be read and it was estimated (TTS: from the text at a typical speaking rate; STT: from the node's measured speed).

When something goes wrong.

Errors use standard HTTP status codes, so your existing retry logic works. These are the ones you'll actually see.

Error responses
400Malformed request — invalid JSON, a missing model, or a field of the wrong type.
401Missing, invalid, revoked or expired API key.
402Your balance can't cover the prompt (or a speech or embedding request in full) — nothing ran. Top up and retry; you're never put into debt. (A reply your balance only partly covers isn't refused: it stops where your credit ends — see above.)
403A disabled account, or a key without the scope this request needs. (A prompt refused by content moderation comes back as an ordinary reply explaining the refusal, and isn't charged.)
404No provider is currently serving that model, or the model ID is wrong. Check /v1/models.
429Rate limit (60 requests/minute per key), a daily/monthly spend cap, or the key's own spend budget was hit (Retry-After says when it resets). Back off and retry.
502The serving node errored and failover couldn't recover the request. Safe to retry.
503The grid is temporarily at capacity for that model. Retry with backoff.

Priced by competition, not by rate card.

Providers undercut each other under every listing, so the price of an open model falls as supply grows — and you pay the provider's rate with no platform markup on top. See what each model costs right now on the marketplace, or read the full fee schedule and worked examples on the pricing page.

Stop paying rate-card prices for open models.

Sign up, add credit, and make your first request in minutes — no waitlist.