For operators

The connector.

The connector is the small program that puts your machine on the grid. It finds the AI engine you already run, connects out to Inpherio over an encrypted link, and serves paying requests to your models — then gets out of the way. One command installs it; it keeps itself up to date.

How it works

Outbound only. Nothing to open.

  1. It finds your engine

    On start it scans this machine (localhost only) for a running inference engine — Ollama, vLLM, llama.cpp, LM Studio and many more — and reads which models each one has. It also reads your GPUs, memory and temperatures.

  2. It dials out

    It opens one encrypted WebSocket (wss://) out to Inpherio and proves who it is with your node's key. Nothing listens on your machine, no ports are opened or forwarded, and your router is never touched.

  3. Your models are benchmarked

    Every model is measured on your hardware — cold load, time to first token, tokens per second — and joins the marketplace the moment it passes, at the prices you set.

  4. It serves requests

    When a customer's request is routed to you, it arrives over the same link; the connector passes it to your engine on 127.0.0.1 and streams the answer back. If the customer disconnects, it stops your engine straight away — your GPU never finishes work nobody is waiting for.

  5. It stays out of the way

    A heartbeat every few seconds keeps your node marked online. If the link drops it reconnects by itself, and new versions install themselves while the node is idle.

Install

One command, from your dashboard.

Sign in, open Nodes → Add node, name the machine and copy the command it gives you. The command carries a single-use install token (valid 24 hours), so your password never touches the machine. Run it on the computer you want to add.

Linux · macOS · Raspberry Pi — terminal
curl -fsSL "https://inpherio.co.uk/api/connector/install.sh?token=inf_install_…&name=My%20PC" | bash
Windows 10 / 11 — PowerShell (it asks for administrator rights itself)
irm "https://inpherio.co.uk/api/connector/install.ps1?token=inf_install_…&name=My%20PC" | iex

Use the exact command from your dashboard — the token in it is yours and works once. Each step is reported live to your dashboard while it runs.

What the installer does

  • Checks the machine and that it can reach Inpherio.
  • Finds an AI engine — or, if you have none, installs Ollama and a small starter model (skip with --skip-ollama / -SkipOllama).
  • Sets up a private Python just for the connector (3.10 or newer; it installs its own copy if needed — nothing system-wide changes).
  • Downloads the connector and checks it against its published checksum.
  • Registers the node to your account and stores its key in a file only the service can read.
  • Starts it as a background service that comes back by itself after a crash or a restart (see platforms).
  • Waits until the node is really connected and tells you if it isn't, and why.

Re-running the command on the same machine is safe: it picks up where it left off, or replaces the old registration instead of creating a duplicate node.

Platforms

Where it runs.

Supported operating systems and how the connector runs on each
SystemRuns asAfter a restart
Linux (Ubuntu, Debian, Fedora, Arch …)A hardened systemd service under its own unprivileged user, with read-only access to the rest of the systemStarts at boot, restarts itself if it ever stops
Linux without systemdA background process; a cron @reboot entry is added when cron is availableStarts at boot when cron runs; in containers use Docker; on WSL turn on systemd (the installer tells you how)
macOS (Apple Silicon and Intel)A background service for your user accountStarts when you log in — see unattended Macs
Windows 10 / 11A scheduled task that runs from bootServes even when nobody is signed in — the installer also keeps Ollama running from boot
Raspberry Pi and other ARM64 boardsSame as LinuxStarts at boot
DockerA container with host networking (recipe below)Docker's restart policy brings it back

GPUs it recognises

  • NVIDIA — every card nvidia-smi reports, including unified-memory systems (GB10 / DGX Spark class): VRAM, free memory and temperature.
  • Apple Silicon — M-series chips with their unified memory.
  • AMD on Linux — Radeon and Instinct cards through the amdgpu driver (VRAM and temperature; the card's name too when ROCm is installed).
  • No GPU? CPU-only machines work too and are listed with their system memory — good for small models, embeddings and speech.

Intel Arc GPUs aren't recognised yet — the node is listed as CPU-only, but your engine can still use the card.

Security

Built so it can't be turned against you.

  • Outbound only. The connector never listens for connections. There is no inbound port on your machine for anyone to find.
  • Encrypted, always. It only talks to Inpherio over wss:// (TLS) and refuses an unencrypted server address.
  • Every message is signed. Each connection gets its own secret; every request from Inpherio is signed with it and time-stamped, and a message that's been seen before is refused — so nobody can replay or forge one.
  • Only your engines, only on this machine. Requests are forwarded only to engines the connector itself found on 127.0.0.1. It never forwards to another computer on your network or to the internet.
  • Requests carry data, never code. A prompt is passed to your engine as text — there is no path from a request to a shell on your machine.
  • Signed updates. Every new version is signed when it's built, with a key that isn't kept on our servers. Your connector checks that signature, the checksum and that the new file runs on your machine before it replaces itself — and refuses anything unsigned.
  • Your key stays private. The node key lives in a file only the connector's service can read, and is never put on a command line.

More on how the whole platform is protected on the security page.

Updates

It keeps itself current — carefully.

  • Checked automatically at start and about every 30 minutes (spread out, so the whole network never restarts at once).
  • Installed when idle. A new version waits until your node has no requests in flight (up to 15 minutes), lets anything running finish, then restarts in a few seconds.
  • Rolled back by itself if a new version crashes or can't connect — and that version is never installed again on your machine.
  • Your choice. Turn automatic updates off in your node's connector settings, and update with one click from the dashboard when you're ready.

Settings

Everything you can change from the dashboard.

Open Nodes → your node → Connector. Changes go straight to the machine over the secure link and apply without a restart. Every value is also checked on the machine itself, so a mistake can't break it.

Connector settings
SettingDefaultWhat it does
Max concurrent requests16 (1–64)How many requests the connector sends to your engines at once. Lower it on a single small GPU.
Request timeout300 s (30–1800)How long one request may run on your engine.
Default keep-alive (Ollama)Engine defaultHow long Ollama keeps a model in memory after use — 5 minutes, 30 minutes, 1 hour or pinned.
Automatic updatesOnInstall new versions by themselves (see updates).
Extra portsNoneOther ports on this machine to look for an OpenAI-compatible server on.
Port overridesNoneTell the connector which port an engine uses, when it isn't the standard one.
Disabled enginesNoneEngines on this machine you don't want offered.
Heartbeat / engine scan / warm check10 s / 60 s / 15 sHow often it reports in, looks for engine changes, and checks which models are loaded.

Keeping models warm

A model that's already loaded answers in milliseconds; a cold one has to load first. From the dashboard you can load a model and choose how long it stays in memory — 5 minutes, 30 minutes, an hour, or pinned (never unloaded). Pinned models are remembered by the connector and loaded again automatically after Ollama or the machine restarts. You can also unload one or all models, pull a new model, delete one, and — on engines that support it — free memory or stop a running job.

Also in the dashboard

  • Temperature alerts — get an alert when a GPU stays above a temperature you choose (30–120 °C) for a set time.
  • Pause or stop serving for a while — for gaming, maintenance or a heatwave — and it comes back by itself.
  • Restart or update the connector remotely, and see its version, engines and live status.

Docker

Run it in a container.

If your engines already run in containers, or the machine has no service manager, run the connector in Docker on a Linux host. It uses the stock Python image — nothing to build. Replace the token with the one from your dashboard (it's only used the first time; the node's key is kept in the volume, so restarts never create a second node).

docker · Linux host
docker run -d --name inpherio-connector --restart unless-stopped --network host \
  -v inpherio-connector:/state -e HOME=/state python:3.12-slim sh -c '
pip install -q --root-user-action=ignore "websockets>=12" "psutil>=5.9" "httpx>=0.27" &&
mkdir -p /state/app && cd /state/app &&
{ [ -f inpherio_connector.py ] || python -c "import urllib.request as u; u.urlretrieve(\"https://inpherio.co.uk/api/connector/download/latest\", \"inpherio_connector.py\")"; } &&
exec python -u inpherio_connector.py --token inf_install_… --node-name "My Docker node"'
  • Host networking lets the connector reach engines on the host's 127.0.0.1 (e.g. Ollama on port 11434). It still only connects out.
  • NVIDIA GPUs: add --gpus all (with the NVIDIA Container Toolkit) so the connector can read the card; otherwise the node is listed as CPU-only even though your engine uses the GPU.
  • Updates land in the volume, so the container keeps the newest version across restarts.

macOS

Unattended Macs.

On macOS, Ollama and the connector run as apps for your user account, so after a restart — a macOS update overnight, a power cut — nothing serves until someone logs in. If your Mac is a dedicated node (a Mac mini or Studio in a cupboard), let it log in by itself:

  • System Settings → Users & Groups → Automatically log in as → choose the account the connector was installed for.
  • Automatic login needs FileVault turned off (System Settings → Privacy & Security → FileVault). Decide whether that's right for the machine and where it's kept.
  • Also set System Settings → Energy → Start up automatically after a power failure, and stop it sleeping.

The installer checks this and reminds you if automatic login is off.

Troubleshooting

If something isn't right.

Status and logs on each system
SystemIs it running?Logs
Linuxsudo systemctl status inpherio-connectorsudo journalctl -u inpherio-connector -f
macOSlaunchctl list | grep inpheriotail -f ~/.inpherio/connector.log
WindowsTask Scheduler → InpherioConnectorC:\Program Files\Inpherio\connector.log
Dockerdocker psdocker logs -f inpherio-connector
  • "No inference engines found" — start your engine (e.g. open Ollama). The connector looks again every few seconds; you don't need to reinstall. Engine on an unusual port? Add it under Port overrides.
  • Node shows offline — check the service is running and the machine can reach inpherio.co.uk. The dashboard shows the connector's last reported status.
  • "Another connector is using this node key" — the same node is running twice (a copy on another machine, or started by hand). Stop the extra one.
  • "The server doesn't recognise this node key" — the node was deleted from your dashboard. Generate a new command and run it again.
  • Still stuck? [email protected], or open a ticket from your dashboard.

Removing it

This removes the connector and its service, and takes the node off your dashboard (add --remove-ollama / -RemoveOllama to remove Ollama and its models too):

Linux · macOS
curl -fsSL https://inpherio.co.uk/api/connector/install.sh | bash -s -- --uninstall
Windows · PowerShell
& ([scriptblock]::Create((irm https://inpherio.co.uk/api/connector/install.ps1))) -Uninstall

Docker: docker rm -f inpherio-connector && docker volume rm inpherio-connector, then delete the node in your dashboard.

Your machine, on the grid in minutes.

Sign up, copy the command, and your idle hours start earning.