Kev on Ollama: Jared Palmer's decision model family, hands on
Kev is a new "decision model" on Ollama — it doesn't chat, it scores answers to structured questions in a single forward pass. Here's what each size needs and how to call it.
The Inpherio blog
Everything we've written about hardware.
Listen to the latest in local AI. New models, GPUs and how-tos — every post is also read aloud, so you can catch up on the commute, at the gym or with your phone locked. Change the speed, pick up where you left off.
Accessibility: every post has a spoken version with a keyboard-friendly player, plus clear headings and a table of contents.
15 posts · page 2 of 2
Kev is a new "decision model" on Ollama — it doesn't chat, it scores answers to structured questions in a single forward pass. Here's what each size needs and how to call it.
vLLM is the engine you pick when you need throughput, but the install is where most people give up. Here's the whole path: pip vs Docker, the Windows reality, a working `vllm serve` command, and a five-minute benchmark to prove it.
Prism ML's Bonsai 2 arrived on Ollama today with an unusual pitch: ternary weights that roughly halve file sizes versus 4-bit quants. Here's what each size needs and how to run it.
Have a GPU, a Mac or a spare PC?
Run open models for people who need them. You set the price and keep everything but the 15% fee.
Become a hostBuilding with AI?
OpenAI-compatible. Priced by the people who run the hardware.
Start free See the marketplace