Supported engines

llama.cpp on Inpherio.

Earn from the llama.cpp you already run. The Inpherio connector finds it automatically and offers the text & chat models you choose to developers through one OpenAI-compatible API — at prices you set.

What it is.

llama.cpp runs GGUF-format language models efficiently on CPUs, NVIDIA, AMD and Apple Silicon. Its built-in llama-server exposes an OpenAI-compatible API.

Several engines use port 8080 by default (llama.cpp, MLX, TGI, TEI, whisper.cpp). If the connector picks the wrong one, set a port override for llama.cpp in your connector settings.

llama.cpp at a glance
ServesText & chat
Detected on127.0.0.1, port 8080 or 8081
Text & chat/v1/chat/completions (also /v1/messages and /api/chat)
Setup changes neededNone — the connector wraps it

Set it up in three steps.

  1. Run llama.cpp

    Start llama-server with your GGUF model, e.g. llama-server -m model.gguf --port 8080.

  2. Install the connector

    Run the Inpherio install command from your dashboard on the same machine. The connector finds llama.cpp on port 8080 or 8081 by itself.

  3. Set your prices

    Each model is benchmarked on your hardware, then goes live on the marketplace at the platform's default prices. Change any price, or switch a model off, from your dashboard.

The install command is one line on Linux, macOS, Windows, Raspberry Pi or Docker — see the connector guide.

Control it from your dashboard.

What the Inpherio dashboard can do with llama.cpp on your machine, remotely and securely:

  • See live engine detail — loaded models, memory and version

Questions.

Does Inpherio work with llama.cpp?

Yes. The Inpherio connector detects llama.cpp running on the same machine automatically and lists its text & chat models in your dashboard — the ones you leave switched on are offered on the marketplace, at prices you set. You don't change your llama.cpp setup.

Do all my llama.cpp models go on the marketplace?

Only the ones you choose. Every model the connector finds appears in your dashboard, switched on by default; switch any model off (or the whole engine) and it is never routed to, rented or listed — the change applies straight away, no restart. A newly found model also has to pass a benchmark on your machine before anyone can use it.

Which port does Inpherio look for llama.cpp on?

Port 8080 or 8081 on 127.0.0.1. If yours runs elsewhere, add a port override for llama.cpp in your connector settings.

Can people reach my llama.cpp server directly?

No. The connector only talks to your engine on 127.0.0.1 and dials out to Inpherio over an encrypted link — nothing on your machine listens for outside connections.

How are llama.cpp models priced?

You set the price for each model in pounds — per million tokens for text and embeddings, per hour of audio for speech. Inpherio takes a flat 15% from what you earn, and each payout carries a payout fee of £2 + 1% per payout to a UK bank account, £2 + 2% to any other country. New providers pay no platform fee for their first 12 months or until they have earned £250 fee-free, whichever comes first; the payout fee still applies.