Best local AI models for NVIDIA GTX 980

The largest models that fit fully

The 30 largest of the 81 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The NVIDIA GTX 980 is equipped with 4 GB of GDDR5 video memory. This hardware limit determines which local artificial intelligence models can run directly on your graphics card. To run a model entirely in video memory, the model size and its runtime overhead must fit within this 4 GB boundary. If a model exceeds this limit, your system must use alternative execution strategies.

Quantization is a method that reduces model size by lowering the precision of the weights. The quantization column shows the best format that fits your hardware. For example, the 5B Lumina Next and CogVideoX models fit using the Q4_K_M quantization, which uses 3.7 GB of video memory. Models like DeepSeek-VL2 at 4.5B and DeepFloyd IF at 4.3B run using the Q5_K_M quantization. Smaller models like the 4B Qwen3, Gemma 3, Gemma 4 E4B, MiniCPM 3, Danube 3, and Fish Speech 1.5 can use the higher quality Q6_K quantization while staying under 3.9 GB.

Other models also fit within the 4 GB limit of the card. The 3.8B Phi-4-mini-instruct, Phi-3.5 Mini, and OmniGen models use 3.7 GB with Q6_K quantization. Image generation models like SD Cascade at 3.6B, SDXL Turbo at 3.5B, and SDXL Lightning at 3.5B fit within 3.5 GB or 3.4 GB. Audio models like MusicGen at 3.3B use 3.2 GB. Highly quantized 3B models like SmolLM3, Replit Code v1.5, Kandinsky 3.1, Voxtral Mini, Orpheus TTS, and Higgs Audio v2 can run at Q8_0 quantization using 3.8 GB of video memory.

When a model is too large for the 4 GB video memory, you can use CPU offloading. This process splits the model between your graphics card and your system memory. CPU offloading allows you to run larger models, but it reduces processing speed because system RAM is slower than video memory. For these cases, we assume your computer has 32 GB of system RAM to handle the shared workload.

Several popular models can run using CPU offloading. The 7B models like Mistral, Qwen2.5, OLMo 2, Falcon 3, Command R7B, and OpenHermes 2.5 require 5.1 GB to 5.7 GB of memory at Q4_K_M quantization, which uses 7.1 GB to 7.7 GB of system RAM. Magicoder-S-DS 6.7B needs 4.9 GB of memory and 6.9 GB of system RAM. The 5.6B Phi-4-multimodal needs 4.1 GB of memory and 6.1 GB of system RAM. Stable Diffusion XL needs 4.1 GB at FP8 precision and uses 6.1 GB of system RAM.

You must consider the context limit when running these models. The memory usage figures are calculated using a basic 4k context window. If you increase the context window to process longer documents or longer conversations, the memory requirements will rise. This extra memory usage might force a model that normally fits in your 4 GB video memory to require CPU offloading instead.