Local AI, open models and the hardware that runs them
New models and how to run them, the hardware that runs them, and what's changing in local and self-hosted AI.
Listen to the latest in local AI. New models, GPUs and how-tos — every post is also read aloud, so you can catch up on the commute, at the gym or with your phone locked. Change the speed, pick up where you left off.
Accessibility: every post has a spoken version with a keyboard-friendly player, plus clear headings and a table of contents.
The smolmaxxing repo and its Hacker News discussion (475 points) blew up last week because "will it run on a 2009 ThinkPad?" never stops being asked. Here's the practical answer: models, quants, flags and realistic speeds.
Datacentre A100s are flooding the second-hand market as Blackwell and Rubin push them out, and 80GB is edging towards hobby budgets. Here's the honest buyers guide.
llama.cpp has hit 1.0.0 — not with a speed jump or a new format, but with a stability promise for everything built on top of it. Here's what that means for the tools you actually use.
A 4B model that writes 3D assets as code is the breakout niche download of the past month — but nobody has written down how to actually run it. Here's the full route: Ollama, llama.cpp, and a suggested pipeline pairing it with image-to-3D model idol.
Ricursive Intelligence's Ripple-65B Preview landed on the Ollama library on 9 November — a 65B dense model trained for agentic coding, with up to 1M tokens of context. Here's what it needs and how to run it.
Kimi-K3.1-Pro is the model getting the attention this fortnight — but the download numbers say people are actually running something else. Here's the honest answer to "best LLM right now", with the numbers behind it.
The honest answer to "vLLM vs Ollama" depends less on benchmarks than on the hardware sitting on your desk. Here's how to choose — and how to test the claims on your own machine.
AutoTrust's autotrust-jev is on the Ollama library: a decision model that returns choices, probabilities and ratings instead of prose. Here's what each size needs and how to run it.
Perplexity's 4B decision model is on the Ollama library as of today, with its own page, a 5.6GB download and a 256K context window. Here's what it is, what hardware it wants, and how to call it.
Xiaomi MiMo's MiMo-V2.6-Distill-Qwen-9B has landed on the Ollama library as mimo-v2.6 — a 9B fine-tune of Qwen3.5-9B for code and agent workflows. Here's what it needs and how to run it.
vLLM Semantic Router's d3 landed on the Ollama library today: four decision models that score your options instead of chatting — and, as far as we've seen, the first of this autumn's decision-model wave with vision at every size.