Install a GGUF Model from Hugging Face

Local Waifu can search Hugging Face for chat models packaged as single GGUF files.

Find and install a model

Goal

Install a compatible third-party GGUF quant without leaving Local Waifu.

Before you start

• Know your available RAM/unified memory and free disk space.
• Prefer model repositories that publish a single file for each quantization.
• Third-party models have their own licenses and behavior.

Steps

1. Open Settings → Models → Chat Models.
2. Under Browse Hugging Face, enter:
   • a search term;
   • owner/repository; or
   • a full Hugging Face model URL.
3. Open a result.
4. Select a single runnable quant such as a Q4 variant.
5. Review the download size and memory-fit estimate.
6. Install the quant.
7. When installation finishes, select Use for chat.
8. Test the model.

Expected result

The Hugging Face model appears under installed models and can be selected for chat.

If something goes wrong

• No GGUF models found: use a repository that contains GGUF files.
• No runnable quants (sharded files only): Local Waifu 1.7.3 does not support a GGUF split across several shard files.
• SafeTensors repositories are not installed through the chat-model search panel.
• Couldn’t reach Hugging Face: check internet, proxy/VPN, and firewall.
• A fit estimate is guidance, not a guarantee. Reduce context or choose a smaller quant if runtime memory is insufficient.