Set up your local AI model (Runtime)
5 min · 5 steps
Pick + install a Small Language Model — Avery's Runtime profile uses local by default.
Avery splits AI into two profiles. Runtime (what runs your agents) is local-by-default — pick a Small Language Model and Avery runs every agent node against it on your machine. Build (what generates agents + apps from prompts) is cloud-only — see the 'Connect a cloud provider' article. The first-run wizard probes your hardware (RAM, GPU, OS) and recommends a Runtime variant.
Steps
- Open Settings → AI → AI Model for Runtime.
Or the first-run wizard if this is a fresh install. The 'Local' option in the mode toggle picks your installed SLM.
- Pick a variant.
Avery filters the catalog to variants that fit your hardware tier. The recommended one is at the top.
- Click 'Install'.
Streams the model files (3-8 GB depending on quantisation). Once done, Avery smoke-tests the install with a tiny prompt.
- If the install or smoke test fails.
Avery doesn't dead-end. It works out why — the model was too big for your machine (out of memory / not enough disk), a download hiccup, Ollama isn't installed, or the smoke test came back wrong — and offers the right one-click recovery: 'Try a smaller model' installs the next-smaller variant that still fits your hardware, or 'Switch to cloud' drops you into the cloud-provider setup instead (Build is cloud-only anyway, so this gets you running fast). Each choice is recorded in the Activity stream.
- Set as active.
Mark the new model as the default for Runtime `/llm.*` requests.
Live recipes need the desktop
This article is a static preview. The in-app Help sidecar inside Avery NXR can fire each step against your live project — install the desktop to use it interactively.