Best GPU for a Local AI Companion on Windows

Every “best GPU for AI” guide I’ve read talks about CUDA cores and clock speeds, which is the wrong axis entirely for a local companion app. The number that actually decides what you can run is VRAM, in gigabytes, full stop.

Why VRAM is the only spec that matters here

The short version: A local language model has to fit into GPU memory to run fast. If it doesn’t fit, it either won’t load or it spills into slow system RAM, and either way the card’s raw compute power becomes irrelevant.

Local Waifu (and most local-model tools generally) auto-detects your hardware and picks a model sized to what will actually fit and run well. That detection weighs total available memory, not GPU speed. A GPU with fewer CUDA cores but more VRAM will run a bigger, better model than a faster card with less memory. This is the single most common confusion in GPU-shopping-for-AI advice, and it’s backwards in almost every guide that isn’t specific to this use case. A card built for high frame rates at 1080p and a card built for VRAM headroom are optimizing for different things, and gaming benchmarks tell you almost nothing about which one wins here.

The four tiers, and which cards land in each

The short version: Local Waifu’s own hardware detection splits into four bands: 8 to 18 GB, 19 to 34 GB, 35 to 56 GB, and 57 GB or more, each opening the door to a progressively larger model.

Here’s how consumer GPUs map onto that, using real, publicly documented VRAM specs:

Every one of those cards, from the humblest RTX 4060 up through the 32 GB RTX 5090, lands in one of the two lower tiers. That’s not a knock on any of them, both tiers are genuinely usable, responsive companions. It’s just the honest shape of where a GPU-only build tops out today.

AMD cards work the same way

The short version: The VRAM math doesn’t care about brand. An 8 GB AMD card lands in the same tier as an 8 GB Nvidia card.

Most local-AI guides write as if Nvidia is the only option, largely because CUDA has historically had the deepest software support. That’s changing. AMD’s Radeon RX 7000 line covers the same tiers on VRAM alone: the RX 7600 at 8 GB and RX 7700 XT at 12 GB land in Light, the RX 7800 XT at 16 GB lands in Balanced, and the RX 7900 XTX at 24 GB reaches the same Balanced ceiling a 24 GB Nvidia card does. Local Waifu’s model runner provisions the right backend automatically depending on which vendor’s card it detects, so this isn’t a manual setup step you need to research yourself, the same auto-detection that picks your model tier also picks the matching GPU runtime behind the scenes.

The tier no single GPU reaches alone

The short version: The top two tiers start at 35 GB and 57 GB. Nothing on the current consumer GPU market, including the 32 GB RTX 5090, crosses even the lower of those two lines by itself.

This is the fact most buying guides never say out loud, because it’s an awkward one for anyone trying to sell you the most expensive card as the automatic best choice. If you specifically want the top model tier, a Windows gaming PC’s GPU is not currently how you get there, because the ceiling on a single consumer card tops out well below the 35 GB threshold. What actually crosses that line on Windows is a generous pool of system RAM, since the detection takes whichever resource, RAM or VRAM, is bigger. A machine with 64 GB of system RAM and a modest GPU reaches further into the tier list than a machine with the single best consumer GPU money can buy and 16 GB of system RAM, which is the opposite of what most shopping advice implies.

What this means for buying decisions

The short version: If you already own an RTX 4060 or better, you don’t need to upgrade for this. If you’re building new specifically for a local AI companion, put the marginal dollar into RAM before you put it into a bigger GPU.

An RTX 4070 at 12 GB and 32 GB of system RAM will run a genuinely good local companion today, on hardware plenty of people already own. Chasing a 24 GB or 32 GB card only matters if you’re already at the Light tier’s ceiling and want the extra context headroom Balanced gives you, not because a 12 GB card is locked out of some categorically smarter tier.

If you’re buying new hardware specifically for this and have a fixed budget, the ranked priority is: enough system RAM to cover the tier you want on its own, then whatever GPU fits what’s left of the budget, rather than the reverse. A GPU upgrade from 12 GB to 16 GB moves you exactly one tier. A RAM upgrade from 16 GB to 64 GB can move you three tiers by itself, RAM is simply the cheaper lever for this specific workload, dollar for dollar, than the GPU market currently offers.

The used-market option nobody mentions

The short version: A three or four year old 12 or 16 GB card, bought used, reaches the same tier as a new one at a fraction of the price.

Because the tier system cares about VRAM capacity and not benchmark scores, an older card that shipped with generous memory for its era is still fully competitive today for this specific task, even if it would lose badly in a modern gaming benchmark. A used RTX 3060 with its 12 GB of VRAM costs a small fraction of anything currently on shelves and lands in exactly the same Light tier as a brand-new card three generations later. If your only goal is running a local companion, this is the cheapest realistic path into a capable tier, worth checking before assuming a new purchase is necessary. Just confirm the actual VRAM figure for the specific card you’re looking at, since some product lines (the RTX 4060 Ti is the clearest example) ship two different VRAM configurations under the same name, and the cheaper one is easy to buy by mistake if you’re going on the model name alone.

Why bigger models actually need more memory

The short version: A language model’s size is measured in parameters, billions of them, and each one has to sit in memory as a number while the model is running. More parameters means a bigger number to store, which means more VRAM before the model even generates a single word.

This is worth understanding at a basic level because it explains why the tiers jump the way they do rather than scaling smoothly. A model with a few billion parameters, compressed down (quantized) to fit efficiently, might need only a few gigabytes of memory just to be loaded, before it’s done any actual work. A model with tens of billions of parameters needs proportionally more, regardless of how fast your GPU’s compute cores are. The compute cores decide how quickly the model produces each word once it’s loaded; the VRAM decides whether it can be loaded at all. A extremely fast GPU with too little memory simply can’t hold the bigger model, full stop, no amount of raw speed compensates for a model that doesn’t fit.

This is also why the tier bands in Local Waifu’s hardware detection aren’t arbitrary round numbers, they’re set at the actual memory footprints of specific real models (the same ones listed in the tier table above), verified against what those models need to load and run with reasonable headroom for the conversation context itself, not just guessed at.

Gaming laptops follow the same rules, with one caveat

The short version: A laptop RTX 4070 or 4080 has less VRAM than its desktop namesake, so check the laptop-specific spec sheet, not the desktop card’s number.

If your only machine is a gaming laptop, the same tier table applies, with one important catch: laptop GPUs sharing a desktop card’s name sometimes ship with less VRAM to fit thermal and power constraints. A laptop RTX 4070 is not guaranteed to carry the same 12 GB as its desktop counterpart. Before assuming your laptop lands in a particular tier, check the specific VRAM figure listed for your exact laptop model, usually in the manufacturer’s spec sheet or via Windows’ own Task Manager performance tab under the GPU section, rather than assuming the desktop card’s number applies.

Multi-GPU setups don’t help the way you’d expect

The short version: Running two GPUs doesn’t simply add their VRAM together for a single model the way it might for other workloads.

Worth a quick mention since it comes up: unlike some other GPU-bound tasks that split cleanly across multiple cards, a single local language model generally needs specific software support to split itself across more than one GPU’s memory pool, and that isn’t the default, automatic behavior most local-AI tools ship with today. If you’re weighing two mid-range cards against one card with more VRAM for this specific use case, the single larger-VRAM card is the safer, simpler bet, and it’s what the tier detection is built around.

See what your card gets you

Local Waifu detects your GPU and RAM automatically on first launch and picks the right model tier without you having to do this math yourself. Seven days free on Windows, no card required to try it.

Questions people ask

Back to the blog