AI Mode

AI Mode is the local-AI layer built into Genesi OS. It combines three parts:

Adaptive performance tuning through the genesi-aid daemon.
A local model workspace in the Genesi AI Mode Monitor.
An agent and global Quick Chat that can answer questions and operate the system.

Models and conversations stay on the machine. The assistant uses local Ollama or Genesi Turbo endpoints and does not require a cloud account.

Adaptive performance tuning

The genesi-aid daemon detects local inference workloads, tunes the system while they are active, and restores every previous value when the workload ends. The changes are fully reversible and survive a daemon restart through a state snapshot under /run.

Setting	Normal	AI Mode
CPU governor	`powersave`	`performance`
Swappiness	60	10 (keep weights in RAM)
Transparent huge pages	`madvise`	`always`
Inference process priority	0	-5 (higher)

On machines with a GPU, AI Mode detects available VRAM through NVIDIA tools, DRM sysfs, or a vendor-neutral Vulkan probe. Profiles are RAM/VRAM-aware and degrade gracefully instead of forcing settings that can push a small machine into OOM.

Supported runtimes

Ollama, llama.cpp (llama-server and llama-cli), vLLM, LocalAI, text-generation-webui, KoboldCPP and Oobabooga are detected automatically.

Performance controls

The Monitor and Plasma widget expose two independent controls:

Activation: Force ON, Auto, or Force OFF.
Profile: Maximum, Balanced, Battery, or Auto.

Auto is the recommended default. It lets Genesi decide when acceleration is useful and how aggressive it should be for the current hardware and power state.

Based on CPU-only Ollama testing, AI Mode commonly improves tokens per second by 15-25%, reduces model load time by 30-40%, and avoids unnecessary swapping. GPU gains vary, but CPU scheduling and memory tuning still help.

Choosing a local model for your hardware

The single biggest factor for running a local model is memory — VRAM on the GPU, or system RAM when running on CPU. The model's weights have to fit, plus a smaller amount for the context (KV cache) and runtime overhead.

Genesi AI Mode already reads your RAM and VRAM and degrades gracefully, but this table helps you pick a model size before downloading it. Sizes are expressed in parameters (B = billion), which is how local models are labelled — a "7B model", a "14B model", and so on.

How much memory a model needs

Weights are almost always distributed quantized (compressed). The quantization level sets how many bits each parameter uses, which decides the file size and how much memory you need. Q4_K_M is the common sweet spot: near full quality at roughly a quarter of the size.

Quantization	Bits / param	Size vs FP16	Quality	When to use
`Q2_K`	~2.6	~16%	Noticeably degraded	Last resort to fit a bigger model
`Q3_K_M`	~3.4	~21%	Usable, some loss	Squeeze a tier up on tight memory
`Q4_K_M`	~4.5	~28%	Recommended default	Best quality-to-size balance
`Q5_K_M`	~5.5	~34%	Very close to full	If you have memory to spare
`Q6_K`	~6.6	~41%	Near-lossless	High fidelity, still smaller than FP16
`Q8_0`	~8.5	~53%	Effectively lossless	Small models where size is a non-issue
`FP16`	16	100%	Full precision	Training / research, rarely needed locally

A quick estimate for a Q4_K_M model:

memory needed ≈ (parameters in B × 0.65 GB)  +  1–2 GB context & overhead

So a 14B model at Q4_K_M needs roughly 14 × 0.65 ≈ 9 GB for weights, plus overhead — about 10–12 GB total. Longer context windows increase the second term; see Context length below.

Model size vs hardware

Figures below assume Q4_K_M, a normal 4K–8K context, and leave headroom for the desktop. "Min VRAM" = fully on the GPU (fastest). "Min RAM (CPU)" = running on the processor with no GPU offload.

Model tier	Weights (Q4)	Min VRAM (all-GPU)	Min RAM (CPU-only)	Good CPU cores	Feel on CPU	Best for	Example open-weight families
0.5–1B	~0.4–0.8 GB	2 GB	3–4 GB	2–4	Very fast	Autocomplete, classification, tiny helpers	Qwen 0.5B, Gemma 1B
1.5–3B	~1.2–2.2 GB	4 GB	6 GB	4	Fast	Quick chat, summaries, simple agents	Gemma 2B, Phi-mini, Llama 3.2 3B
3.5–4B	~2.4–3 GB	4–6 GB	8 GB	4–6	Comfortable	Everyday assistant on light machines	Phi-3, Qwen 4B
7–8B	~4.5–5.5 GB	6–8 GB	8–16 GB	6–8	Usable	The mainstream default — chat, coding help, tool use	Llama 3.1 8B, Qwen 7B, Mistral 7B
12–14B	~8–9 GB	10–12 GB	16 GB	8+	Slower, patient	Better reasoning & code, still single-GPU friendly	Qwen 14B, Phi-4 14B
27–34B	~16–20 GB	24 GB	32 GB	12+	Slow on CPU	Strong reasoning, serious local work	Gemma 27B, Qwen 32B, Yi 34B
70B	~38–42 GB	48 GB (or 2×24 GB)	64 GB	16+	Very slow on CPU	Near-frontier quality, workstation-class	Llama 3.x 70B, Qwen 72B
100B+ (MoE)	60 GB+	80 GB+ (multi-GPU)	96–128 GB	16+	Impractical on CPU	Research rigs & servers	Mixtral 8×22B, large MoE

MoE models are lighter than they look

Mixture-of-Experts models (e.g. an 8×7B) list a huge total parameter count but only activate a fraction per token. They still need memory for all experts, but they run much faster than a dense model of the same total size.

Quick pick by GPU VRAM

If you know your graphics card's VRAM, start here. This is the largest tier that runs fully on the GPU at Q4_K_M with room for the desktop.

GPU VRAM	Largest comfortable model	Also great	Typical cards
4 GB	3B	1–2B fast	GTX 1650, MX-series
6 GB	7–8B (tight)	3–4B smooth	RTX 2060, RTX 3050
8 GB	7–8B	12–14B partial offload	RTX 3050 8G, 3060 Ti, 4060
12 GB	12–14B	7–8B at Q6/Q8	RTX 3060 12G, 4070
16 GB	14B (comfortable)	27–34B partial offload	RTX 4060 Ti 16G, 4080
24 GB	27–34B	70B partial offload	RTX 3090, 4090
48 GB	70B (Q4)	34B at high quant	RTX 6000, 2×24 GB
80 GB+	70B at high quant / 100B+ MoE	multi-model serving	Data-center GPUs

You don't need to fit it all on the GPU

With llama.cpp and Ollama you can partially offload — put as many layers as fit on the GPU and run the rest on CPU. An 8 GB card can still help a 14B model; it just won't be as fast as a full-GPU fit. Genesi Turbo enables GPU offload when a compatible backend is installed.

Quick pick for CPU-only (system RAM)

No dedicated GPU? Models run on the CPU using system RAM. It works well for smaller models; expect fewer tokens per second as size grows. More/faster cores and dual-channel memory help noticeably.

System RAM	Comfortable model	Stretch (slower)	Notes
8 GB	3B	7–8B	Close other apps; keep context short
16 GB	7–8B	12–14B	The practical sweet spot for most laptops
32 GB	12–14B	27–34B	34B is usable but slow on CPU
64 GB	27–34B	70B (very slow)	70B fits but is patience-testing on CPU
128 GB	70B	100B+ MoE	Server/workstation territory

Context length costs extra memory

Everything above assumes a modest context. The KV cache grows with how many tokens are in play, and it comes out of the same VRAM/RAM budget as the weights.

Context window	Extra memory (7–8B)	Extra memory (14B)
4K (default)	~0.5 GB	~1 GB
8K	~1 GB	~2 GB
16K	~2 GB	~4 GB
32K	~4 GB	~8 GB

If you push a long context, either leave more free memory or drop one model tier. Some backends support KV-cache quantization to soften this cost.

Let Genesi decide when unsure

Set AI Mode's profile to Auto and pick a model one tier below your maximum for the best balance of speed and quality. The daemon is RAM/VRAM-aware and won't push a small machine into swapping or OOM. Use Turbo for GPU offload and speculative decoding (a small draft model + your main model) to gain speed without moving up a tier.

Mixture-of-Experts: bigger brains on a small card

Everything above assumes a dense model, where every parameter is read for every token. A Mixture-of-Experts (MoE) model is built differently: it holds many "expert" sub-networks but activates only a couple per token.

That changes the arithmetic completely. Decode speed is limited by how many bytes the machine reads per token, not by how many parameters the model has — so a 30B MoE with 3B active per token decodes at roughly 3B speed while answering with something much closer to 30B quality.

Genesi Turbo exploits this directly. When an MoE model is too big for your VRAM, Turbo does not spill random dense layers over the slow PCIe bus. It keeps all attention layers on the GPU and pushes only the expert tensors into system RAM (--n-cpu-moe). Because only a fraction of the experts fire per token, streaming them from RAM costs far less than spilling a dense model would.

	Dense 30B	MoE 30B (3B active)
Bytes read per token (Q4)	~18 GB	~2 GB
Runs on an 8 GB card?	Barely, very slowly	Yes, via expert offload
Limiting resource	VRAM	System RAM (to hold the experts)

For MoE, RAM is the requirement — not VRAM

You need enough system RAM to hold all the experts (roughly the model's file size plus ~15%), and enough VRAM for attention and the KV cache (about 4 GB is a practical floor). Fast dual-channel memory matters a lot here: it becomes the real bottleneck, and DDR5 roughly doubles the ceiling over DDR4.

Best models per hardware

The practical picks, assuming Q4_K_M. Speeds are rough single-stream estimates for planning, not benchmarks — your CPU, RAM speed and backend (CUDA is typically 1.5–2× Vulkan) all move these numbers. Measure your own with the Monitor's benchmark.

Your hardware	Best all-GPU pick (fastest)	Best "punch above your weight" pick	Approx. speed
4 GB VRAM	a 3B dense model	a 4B dense model	25–45 tok/s
6 GB VRAM	a 7–8B dense model	a ~16B MoE with expert offload	20–35 tok/s
8 GB VRAM + 16 GB RAM	a 7–8B dense model	a 20B-class MoE (~3.6B active)	25–40 dense · 10–20 MoE
8 GB VRAM + 32 GB RAM	a 7–8B dense model	a 30B-class MoE (~3B active)	25–40 dense · 10–25 MoE
12 GB VRAM + 32 GB RAM	a 12–14B dense model	a 30B-class MoE	20–30 dense · 15–30 MoE
16 GB VRAM + 32 GB RAM	a 14B dense model	a 30B-class MoE (mostly on GPU)	25–35 dense · 25–40 MoE
24 GB VRAM + 64 GB RAM	a 27–34B dense model	a 100B+ MoE with expert offload	15–25 dense · 10–20 MoE
No GPU, 16 GB RAM	a 7–8B dense model	a 20B-class MoE	4–9 dense · 8–15 MoE
No GPU, 32 GB RAM	a 12–14B dense model	a 30B-class MoE	2–5 dense · 6–12 MoE

Some widely used open-weight families in each tier, to make the sizes concrete:

Tier	Dense examples	MoE examples
1–4B	Llama 3.2 (1B/3B), Gemma 3 (1B/4B), Qwen3 4B	—
7–8B	Llama 3.1 8B, Qwen3 8B, Mistral 7B	—
12–16B	Qwen3 14B, Phi-4 14B, Gemma 3 12B	DeepSeek-V2-Lite (16B, ~2.4B active)
20–34B	Mistral Small 24B, Gemma 3 27B, Qwen3 32B	gpt-oss-20b (21B, ~3.6B active), Qwen3-30B-A3B (30B, ~3.3B active), Mixtral 8×7B (47B, ~13B active)
70B+	Llama 3.x 70B, Qwen2.5 72B	Llama 4 Scout (109B, ~17B active), gpt-oss-120b (117B, ~5.1B active)

The honest expectation on a small card

MoE is a large, real win — but it is not unlimited. On an 8 GB card a 30B-class MoE lands in the tens of tokens per second, not the hundreds: the experts are streamed from system RAM, so your memory bandwidth sets the ceiling. Compared with a dense model of the same size (often under 5 tok/s), it is the difference between unusable and comfortable.

Using any GGUF model

AI Mode is not limited to what ollama pull provides. Any .gguf file on the machine is a first-class model — including one downloaded straight from Hugging Face. There is no conversion step and no Modelfile to write.

This matters most for MoE models: the quantizations that make them usable on a small card are published as GGUF files, and Genesi Turbo is what applies the expert-offload flag that runs them fast.

Open the Models page in the Monitor. The Local GGUF models card lets you:

Add a GGUF file with the file picker.
Drag and drop a .gguf onto the window straight from your file manager.
Paste a direct link to a .gguf to download it with progress.
Rescan — files already in ~/Downloads or ~/Models are found automatically.

Imported files are copied into ~/.local/share/genesi/models. Files discovered elsewhere are listed but never moved or deleted.

Knowing how it will run before you load it

Every model in the list carries a badge computed from your actual VRAM and RAM, so you learn what to expect without a trial-and-error load:

Badge	Meaning
FITS GPU	Fits entirely in VRAM — full GPU speed
MoE OFFLOAD	Too big for VRAM, but attention stays on the GPU and experts stream from RAM
SLOW	A dense model that spills VRAM — part of it runs on the CPU
CPU	No usable GPU; runs on the processor

The card also shows parameter count, quantization, file size and architecture, all read from the GGUF header — no weights are loaded to display this.

GGUF models run through Turbo

Selecting a local GGUF switches Turbo on automatically. Turbo drives llama-server, which loads a .gguf path directly and reads the architecture, context length and chat template from the file header. Ollama's chat API only knows models in its own registry, so it cannot serve an arbitrary file.

From the command line

The same library is available without the GUI:

genesi-ai-turbo gguf-list                    # JSON of every GGUF found
genesi-ai-turbo gguf-info ~/Downloads/m.gguf # metadata + how it will run here
genesi-ai-turbo gguf-import <path-or-url>    # add one to the library

Serving takes a path or a library name wherever it takes an Ollama tag:

genesi-ai-turbo serve ~/Downloads/qwen3-30b-a3b-q4_k_m.gguf
genesi-ai-turbo serve gguf:qwen3-30b-a3b-q4_k_m

Every Turbo feature — MoE expert-offload, Genesi Fit, speculative decoding and autotuned profiles — applies to these files exactly as it does to Ollama models.

Sharded and multi-part files

Large models are often split into model-00001-of-00003.gguf and so on. Keep every shard in the same folder and point Genesi at the first one — the rest load automatically. The Monitor lists only the first shard, so the set appears as a single model.

AI Mode Monitor

Open Genesi AI Mode Monitor from the application launcher, the Plasma widget, or the cross-desktop tray icon. The Monitor provides:

Live AI Mode state, detected workloads and applied optimizations.
Local model discovery and selection.
AI Chat with in-conversation history.
Force/profile controls shared with the daemon.
Turbo and speculative-decoding controls.
Optional Memory Palace integration for long-term local recall.
A shortcut to the global Quick Chat.

You can also launch it from a terminal:

genesi-ai-monitor

Quick Chat

Press Ctrl+Alt+Space from anywhere to open the always-ready Quick Chat. It is registered for Plasma, GNOME/Budgie, Cinnamon, XFCE, Hyprland/Caelestia, Niri, COSMIC and LXDE sessions. The same surface can be opened from the Monitor, Plasma widget, tray menu, or with:

genesi-ai-quick --show

Quick Chat uses the same selected local model and performance stack as the Monitor. Its controls let you choose the model, AI Mode activation, performance profile, Turbo and speculative decoding without opening the full dashboard.

The conversation is displayed as one chronological timeline: user messages, approval requests, running tools, results and final answers appear in the order they happened. A Stop button cancels the current model request or captured background command.

Quick Chat service

Quick Chat starts as a user service and keeps a single lightweight process ready in the background. Opening it again shows the existing instance instead of starting another model session.

Agent modes

The assistant has three interaction modes in the full Monitor:

Mode	Behavior
Chat	Answers questions without operating the computer.
Approval	Shows a human-readable action card and waits for permission.
Automatic	Runs supported actions without asking for each one.

Quick Chat exposes Ask first and Automatic in its AI controls. Ask first is the safer default. Automatic is useful for trusted local workflows, but it should only be enabled when you are comfortable letting the selected model act without per-action confirmation.

Approval cards explain the action in plain language, show the target, command, path, package or PID where relevant, and keep raw technical arguments behind a details control. Desktop notifications are sent when approval is needed while the window is not focused.

What the agent can do

The local tool layer can:

Inspect system information and running processes.
List, search and read local files.
Create directories and write text files.
Run non-interactive commands and return their output to the model.
Open a visible terminal for TUI, interactive, sudo, installer or long-running commands.
Discover and launch Genesi apps and applications installed through .desktop files.
Open files, folders and URLs with their default application.
Stop a user-owned process by numeric PID.
Install and remove official Arch/CachyOS packages through pacman.
Fall back to paru or yay in a visible terminal for AUR packages.

Common intents such as opening Genesi Sandboxes, running a command in Terminal, installing a package, or removing an installed application are resolved deterministically. This keeps small local models reliable for operating-system tasks instead of depending on them to guess executable names or tool arguments.

Safety boundaries

Approval mode never executes an action before the user accepts it.
Privileged package operations still use the operating system authorization flow.
Process termination requires a real numeric PID and is never treated as uninstalling software.
Destructive disk commands such as filesystem formatting or wiping devices are blocked.
Agent loops are bounded to prevent repeated or runaway tool calls.
Duplicate actions are detected and are not executed twice.

Local models still matter

Larger models generally understand ambiguous requests better, but tool safety and common system actions do not rely on model size alone. Genesi validates every tool call and uses deterministic routes for frequent desktop, terminal and package tasks.

Turbo and speculative decoding

Turbo routes generation through Genesi's optimized llama.cpp server and can offload work to the GPU when a compatible backend is installed. It reuses the selected model and does not change the normal Ollama chat path when disabled.

Speculative decoding pairs the main model with a smaller draft model. The draft proposes tokens and the main model verifies them, which can improve output speed on supported hardware. The Monitor prevents incompatible combinations and shows backend installation guidance when CUDA or Vulkan support is missing.

Command-line control

AI Mode activation can be inspected or changed without the GUI:

genesi-ai-mode status     # current state and detected processes
genesi-ai-mode on         # force on
genesi-ai-mode off        # force off
genesi-ai-mode auto       # return to automatic detection

Inspect the daemon directly:

sudo systemctl status genesi-aid
sudo journalctl -u genesi-aid -f

Troubleshooting Quick Chat

Check and restart the user service:

systemctl --user status genesi-ai-quick.service
systemctl --user restart genesi-ai-quick.service

Repair the global shortcut registration for the current desktop session:

genesi-ai-quick-shortcuts

If a model is not listed, open the Monitor and refresh Models, then verify Ollama:

systemctl status ollama
ollama list

AI Mode

On this page