Tune Performance, Hardware, and Network
Use Settings → Performance to tune speed, memory use, hardware acceleration, Offline Mode, and the cloud fallback switch.
Follow the bold button names and numbered steps. Technical names are included only when you need to recognize a model or file inside the app.
Open Settings → Performance (the tab used to be called Hardware) to inspect storage, check what can run right now, choose compute behavior, set the local context size, reduce resource use, and control network routing.
Choose a compute device and verify acceleration
Goal
Use the best available hardware and confirm whether the active local model is running on the GPU or CPU.
Before you start
- Send at least one message so the local model is loaded before checking acceleration.
- A GPU is optional on Windows; Local Waifu can fall back to CPU.
- macOS builds require Apple Silicon and use unified memory.
Steps: macOS
- Open Settings → Performance.
- Under Compute Device, choose:
- Auto-detect (Recommended): lets the local engine use Metal acceleration on Apple Silicon.
- CPU Only: forces CPU-only inference. In 1.8.0 this really runs the model on the CPU. An earlier version ignored the setting.
- Restart Local Waifu when prompted.
Steps: Windows
- Open Settings → Performance.
- Leave Compute Device on Auto-detect (Recommended) unless troubleshooting.
- Send a message to load the model.
- Under Local AI acceleration, select Check.
- Read the result:
- Running on GPU: the model is in VRAM.
- Split CPU/GPU: only part of the model fits in VRAM.
- Running on CPU: the GPU is not engaged.
- No model loaded yet: send a message, then check again.
- If NVIDIA or AMD hardware is detected but not fully used, select Retry GPU setup. The app may download a matching runtime and restart the engine.
AMD note: current Windows builds read dedicated AMD VRAM for the model-tier recommendation and report ROCm/Vulkan readiness in Settings → Performance. ROCm v7/HIP7 is preferred where the driver exposes it; Ollama’s Vulkan runner is the supported fallback on cards such as some RX 6000 systems. A 16 GB card alone still maps to Light; if Qwen 3.5 9B fits your VRAM, select it manually in Settings → Models and confirm acceleration with Check. Lower Context Size if the result is Split CPU/GPU.
Expected result
macOS uses Metal unless CPU Only was selected. Windows reports the real placement of the loaded model and, where available, the percentage held in VRAM.
If something goes wrong
- Restart the app after changing Compute Device.
- Try a smaller or more heavily quantized model if Windows reports Split CPU/GPU.
- Update GPU drivers, then use Retry GPU setup again.
- CPU fallback is functional but can be much slower.
- There is no separate Apple Neural Engine inference path in the macOS 1.8.0 build. A stored Metal choice is shown as Auto-detect, because the two were the same setting.
Check runtime readiness
Goal
Confirm which local features can run now.
Before you start
This panel is on both platforms. It shows what can run right now, which saves guessing when a feature is silent.
Steps
- Open Settings → Performance.
- Under Runtime status, select Check.
- Review Local AI engine, Memory model, Chat model, Voice output, Sees images, and Image generation. Each reads Ready or Not available.
- For an item marked Not available, open the relevant model section and install or select the missing component.
Expected result
Each feature you intend to use is marked Ready.
If something goes wrong
Restart Local Waifu after installing a runtime or model, then run Check again. If the local engine remains unavailable, verify Ollama Host and review App Logs as described in Chapter 41.
Set the context size and thinking mode
Goal
Balance conversation context, memory use, and response speed for local chat models.
Before you start
Larger context windows require more RAM or unified memory and may substantially slow replies. Cloud providers use their own limits.
Steps
- Open Settings → Performance and find Context Size.
- Start with 8192 (Standard).
- Increase to 16384 (Large Context) or 32768 (Max Context) only if your model and available memory can support it. A stored 4096 legacy value is automatically protected to 8192.
- Use Custom only when you know the model’s context limit. Both platforms accept 8,192 to 262,144 tokens, although the active model may support less.
- Turn on Deeper thinking only when you prefer more considered but slower responses.
- Optionally turn on Suggest replies to show three tappable replies under her newest message. It is off by default because it costs output tokens on every turn.
Expected result
Local replies use the requested context size within the active model’s capabilities. In 1.8.0 local chat uses at least 8K and can grow to 16K to keep the character prompt, except on machines where long prompts are already slow. Deeper thinking trades speed for more reasoning.
If something goes wrong
- Reduce Context Size if responses become slow, memory pressure rises, or the model fails to load.
- A high value is a request, not a guarantee; the active model may support less.
- Turn off Deeper thinking for faster, more responsive chat.
Reduce background resource use
Goal
Make Local Waifu release model memory sooner and perform fewer background wake-ups.
Before you start
Low Power Mode can make the next reply slower because the local model is unloaded sooner between chats.
Steps
- Open Settings → Performance.
- Turn on Low Power Mode.
- Restart Local Waifu.
Expected result
Background work occurs less frequently and the local model leaves memory sooner after inactivity.
If something goes wrong
If first-response latency becomes inconvenient, turn off Low Power Mode and restart the app.
Configure proxy, offline mode, or a remote Ollama server
Goal
Control network behavior without disabling local AI unnecessarily.
Before you start
- Use System Proxy is enabled by default and changes require a restart.
- Offline Mode blocks cloud providers but does not block local Ollama.
- A remote Ollama Host exposes prompts to that other computer and to the intervening network; use only a host you trust.
Steps
- Open Settings → Performance → Network.
- Leave Use System Proxy enabled if your network depends on operating-system proxy settings. Restart after changing it.
- Turn on Offline Mode to prevent cloud chat providers from running.
- To use Ollama on another computer, enter its URL in Ollama Host, for example
http://192.168.1.50:11434. - To return to the bundled local engine, select Reset beside Ollama Host.
- Decide on Use the cloud when the local model can’t answer. It is off by default. When it is on and the local engine is down, or she receives a photo and no local model can see, the message goes to one of your saved cloud keys, and you see a note when that happens. Leave it off to keep everything local. Chapter 29 explains it.
Expected result
Local Ollama continues to work in Offline Mode. Cloud models refuse to run. A valid custom Ollama Host routes local-model requests to the specified server.
If something goes wrong
- If cloud chat reports that it is blocked, turn off Offline Mode only if sending data to the configured provider is intentional.
- If local chat goes offline after editing Ollama Host, select Reset.
- If downloads fail behind a corporate proxy, enable Use System Proxy and restart.
- Check firewall access to the remote Ollama host and port; do not expose Ollama directly to the public internet.
Storage note: Local models can occupy tens of gigabytes. The Change Path control is disabled in the current build. Do not rely on it for moving models; keep the default model folder and plan free disk space before downloading a large model.