# Nemotron 3.5 Bolt on Ollama: NVIDIA's local coding and vision model, first look

NVIDIA's compact vision-language model for local coding, reasoning and tool use has landed on the Ollama library in two very different sizes — one that fits a 12GB GPU, one that realistically needs more than 24GB of VRAM.

Published 2026-10-23 · https://inpherio.co.uk/blog/nemotron-3-5-bolt-on-ollama-nvidia-s-local-coding-and-vision-model-fir

Key points:
- Nemotron 3.5 Bolt is NVIDIA's compact vision-language model for local coding, reasoning and tool use, on Ollama from 23 October 2026.
- Two variants: e4b (8B params, 4B effective, 7.2–7.9GB, 128K context) and the default latest (74B-A9B, 9B active, 25GB, 256K context).
- Budget roughly the download size in RAM/VRAM plus context headroom: e4b fits a 12GB GPU or 16GB Mac; the 74B's 25GB file realistically needs more than 24GB of VRAM.
- Apple Silicon/Metal is not currently supported for the 74B — Mac users are limited to e4b.
- Run with `ollama run nemotron-3.5-bolt` (big) or `ollama run nemotron-3.5-bolt:e4b` (small); Ollama is the only documented engine as of release day.

NVIDIA has put **Nemotron 3.5 Bolt** on the [Ollama library](https://ollama.com/library/nemotron-3.5-bolt) as of today, 23 October 2026. It's the company's compact vision-language model for local coding, reasoning and tool use, and it arrives in two variants aimed at very different machines: a 7.2–7.9GB small model that fits a 12GB GPU or a 16GB Mac, and a 25GB mixture-of-experts model that realistically needs more than 24GB of VRAM. Here's what each one is, what it costs you in memory, and how to run it.

## What is Nemotron 3.5 Bolt?

Bolt is NVIDIA's take on a model you'd actually run at home rather than in a data centre — the Ollama page pitches it as "NVIDIA's compact vision-language model for local coding, reasoning, and tool use". There are two variants, and they share surprisingly little.

**Bolt E4B** is an 8-billion-parameter model with 4 billion effective parameters, post-trained by NVIDIA from **Gemma 4 E4B** — so it shares its lineage with the Gemma 4 family we covered when [EmbeddingGemma 2 landed on Ollama](/blog/embeddinggemma-2-on-ollama-google-s-multimodal-embedder-that-runs-on-a). It's designed for fast local conversations, instruction following and multi-turn tool use. It also ships with a companion **DFlash draft model** that, per the readme, accelerates generation through speculative decoding. The page doesn't detail the mechanism beyond that, so we won't speculate on how much speed it buys — worth testing on your own machine.

**Bolt 74B-A9B** is the unusual one. It has 74 billion total parameters with only 9 billion active per token, and it combines a **Mamba-2/Transformer mixture-of-experts backbone** with a **C-RADIO vision encoder**. NVIDIA built it from Nemotron 3.5 Super through model compression and further training — so it's a distilled, shrunk-down version of a much larger model, not something trained from scratch at this size.

The capabilities listed on Ollama are **vision, tools, thinking and audio**. The "thinking" one matters in practice: reasoning can be toggled on or off, so you can have the model work through hard problems step by step, or skip it for fast chat. The readme is explicit that vision covers images, screenshots, charts and documents, and that tools can be called and their results used in multi-turn tasks.

It's all released under the **OpenMDW License Agreement, version 1.1** — an open licence aimed at model sharing. Read the terms yourself before commercial use; for running it at home it's an open licence.

## Sizes, downloads and memory

Here are the numbers from the Ollama page:

| Tag | Download | Context | Inputs |
|---|---|---|---|
| `nemotron-3.5-bolt:latest` (74B-A9B) | 25GB | 256K | Text, Image |
| `nemotron-3.5-bolt:e4b` | 7.2–7.9GB | 128K | Text, Image |

Our rule of thumb throughout this blog — the one we set out in [LM Studio vs Ollama](/blog/lm-studio-vs-ollama-which-one-should-you-actually-install) — is to budget roughly the download size in RAM or VRAM, plus headroom for context. Bolt's readme backs that up with a warning worth quoting in spirit: usable context depends on available memory, and images, conversation history, tool results and output all share that budget. It also notes that memory requirements depend on context size, visual inputs and speculative decoding.

The 25GB file for a 74B model raises an obvious question: how? The answer is on the page. The published GGUF uses **quantisation-aware distillation for IQ2_XXS expert weights**, alongside **NVFP4 and Q8_0 for other layers**. IQ2_XXS is a very aggressive quantisation level — roughly 2 bits per weight for the experts — and it's the reason 74 billion parameters squeeze into a 25GB file. That's a different route to small files than [Bonsai 2's ternary weights](/blog/bonsai-2-on-ollama-a-27b-model-that-fits-in-8-3gb), but the goal is the same: big-model behaviour at a fraction of the disk and memory.

## What hardware you need

Mapping the two variants to real machines, using our rule of thumb:

- **E4B (7.2–7.9GB):** by the same maths, a 12GB gaming GPU should hold it with room for context, and a 16GB Mac likewise — the readme confirms E4B is supported on Windows, Linux and Apple Silicon. With 128K of nominal context, you won't use the full window on 12GB, but ordinary conversations and modest tool use should fit.
- **74B-A9B (25GB):** this is where the rule of thumb bites. A 25GB file does not fit on a 24GB card — there's no headroom at all, let alone for the KV cache. Realistically you want 32GB+ of VRAM (or system RAM with offload, at a speed cost), or a 24GB card with partial CPU offload if you're willing to trade tokens per second for having the model at all. And there's a second catch for one whole platform.

The second catch: **Apple Silicon/Metal is not currently supported for the 74B-A9B.** It's stated plainly in the readme's compatibility section. So despite the 25GB file being within reach of a 32GB or 48GB Mac's unified memory, Mac users can't run the big one today. If you're on Apple Silicon, e4b is your only option until that changes.

And even with enough VRAM: don't expect to use the full 256K context window on a modest setup. The readme is explicit that usable context depends on memory, and images, history, tool results and output all share that budget. Long-context work on this model wants generous memory headroom above the 25GB file itself.

## How to run it locally

The commands are simple:

```bash
# The big one — 74B-A9B, 25GB, 256K context
ollama run nemotron-3.5-bolt

# The small one — E4B, 7.2–7.9GB, 128K context
ollama run nemotron-3.5-bolt:e4b
```

One gotcha worth flagging: **`latest` is the 74B, not the E4B.** If you type `ollama run nemotron-3.5-bolt` on a laptop expecting the small model, you'll start a 25GB download. That tag asymmetry is easy to hit by accident — we'd rather the default were the one most machines can run, but it isn't, so specify `:e4b` if that's what you want.

As of release day, the Ollama library page is the only run documentation we've seen — the same situation we found with [kev](/blog/kev-on-ollama-jared-palmer-s-decision-model-family-hands-on) and [pplx-decider](/blog/pplx-decider-on-ollama-perplexity-s-27b-model-that-never-writes-a-word) earlier this month. There are no llama.cpp, LM Studio or vLLM steps on the page. If you want the raw weights, NVIDIA's model cards on Hugging Face cover the E4B in BF16 and Q4_K_M, the DFlash draft in NVFP4, and the 74B-A9B as a 24GB GGUF — but expect to do your own engine plumbing with those for now.

New to Ollama altogether? Our [engine guide](/engines/ollama) covers the setup.

## What it's good for — and where it's limited

The stated design goals are local coding, reasoning and multi-turn tool use, and the capability set matches: vision over screenshots, charts and documents; function calling where results feed back into the conversation; and toggleable thinking. That combination — vision plus tools plus switchable reasoning — is genuinely useful for agent-style workflows, where a model reads a screenshot, calls a tool, and reasons about the result.

The toggleable thinking is the feature we'd actually use day to day. Reasoning on for a tricky debugging question; off for quick chat. E4B is explicitly pitched for fast local conversations and instruction following, and the DFlash draft model is meant to help on the speed front.

Now the honest limits:

- **No independent benchmarks yet.** Every quality claim comes from NVIDIA's model cards. Treat them as vendor numbers until third-party results appear — the same caveat we gave for Bonsai 2 and pplx-decider this month.
- **IQ2_XXS expert weights are a very aggressive quantisation.** Two-ish bits per expert weight is at the far end of what quantisation can do before quality degrades. NVIDIA used quantisation-aware distillation to mitigate that, but some quality loss versus higher-precision checkpoints is possible. If you have the memory for a higher-precision model of similar capability, it may be the better trade.
- **No Apple Silicon for the 74B**, as above.
- **The audio capability is listed but undocumented.** Ollama's capability row includes audio, but the readme gives no run instructions for it — it's unclear how to exercise it locally. Don't buy hardware on the strength of that line item.

## How it compares with what you might already run

We don't have head-to-head numbers, so this is positioning, not benchmarking.

**E4B vs the 8B class.** Because E4B is post-trained from Gemma 4 E4B, [gemma4:e4b](/models/gemma4:e4b) is the natural baseline — same effective size class, same architecture family. If Bolt E4B beats its own parent on your tasks, that's interesting; if not, you've learned something equally useful. The other usual suspects in this class are [qwen3:8b](/models/qwen3:8b) and llama3.1:8b. Bolt E4B's differentiators are vision, toggleable thinking and the DFlash speculative-decoding draft.

**E4B vs small vision models.** For vision in a small local package, [qwen3-vl:8b](/models/qwen3-vl:8b) is the closest comparable in size and capability, and [moondream:1.8b](/models/moondream:1.8b) covers the tiny end. Bolt's pitch over those is the tool-use training and the DFlash draft; on the big variant, the context window is much longer too — 256K versus typical.

**74B-A9B vs the big-local class.** Nine billion active parameters per token puts it in the same spirit as [gpt-oss:20b](/models/gpt-oss:20b) — both are sparse designs with a large total parameter count and a small active subset. But the comparison cuts both ways: gpt-oss:20b's download is in the ~13GB class, so Bolt's 25GB file needs roughly twice the memory, and its expert weights are quantised far more aggressively. What you get for that is vision, 256K context and the Mamba-2 hybrid backbone.

The honest advice, as ever: run your own quick eval on the tasks you actually do. Ten minutes with your real prompts tells you more than any vendor table.

## Should you try it?

By hardware class:

- **12GB GPU or 16GB Mac:** e4b is the easy entry point — and the only Mac option. It's a sensible try if you want vision and tools in the Gemma 4 size class.
- **32GB+ VRAM, or a 24GB card willing to offload:** the 74B is worth the 25GB download if you want a local agent/coding model with a 256K context window. Just go in knowing the usable context depends on your memory headroom, and that the quantisation is aggressive. On a 24GB card alone, it won't fit — plan for offload or more VRAM.
- **Apple Silicon wanting the big one:** wait. Metal support isn't there yet, and there's no timeline on the page.

If you'd rather sample the size class before committing 8GB of disk, Bolt E4B's parent — [gemma4:e4b](/models/gemma4:e4b) — is on the Inpherio marketplace. It's the Gemma 4 E4B baseline Bolt was post-trained from, so it's a cheap way to feel out the size class, though not a preview of Bolt's own tuning:

(Live on Inpherio: https://inpherio.co.uk/models/gemma4:e4b)

And if you've got a spare 32GB GPU sitting idle, this is exactly the kind of model worth serving — new hosts on Inpherio get 12 months fee-free, up to £250 of earnings. See [how hosting works](/providers).

## FAQ

**How much RAM do I need for Nemotron 3.5 Bolt?** Budget roughly the download size: about 8GB of RAM/VRAM for e4b (7.2–7.9GB download) and at least 25GB plus context headroom for the 74B-A9B — realistically more than a 24GB card can hold. The readme warns usable context depends on available memory.
**Does Nemotron 3.5 Bolt work on Mac?** Partly. E4B is supported on Apple Silicon, but the 74B-A9B explicitly does not support Apple Silicon/Metal yet — Mac users can only run e4b today.
**What's the difference between Bolt E4B and Bolt 74B-A9B?** E4B is an 8B-parameter model (4B effective) post-trained from Gemma 4 E4B for fast local chat and tool use. The 74B-A9B has 74B total parameters with 9B active per token, built from Nemotron 3.5 Super via compression.
**What is DFlash speculative decoding?** DFlash is a small companion draft model for Bolt E4B. It accelerates generation through speculative decoding, per the readme — the readme doesn't detail exactly how the draft and main model interact.
**Is Nemotron 3.5 Bolt better than Qwen or Gemma at the same size?** Nobody knows yet — as of release day all quality claims come from NVIDIA's own model cards, with no independent benchmarks. Run your own quick eval on the tasks you actually do.
