Best local AI models for NVIDIA GTX 980M

The largest models that fit fully

The 30 largest of the 123 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The NVIDIA GTX 980M is a mobile graphics card equipped with 8 GB of GDDR5 video memory. This onboard memory determines the maximum size of the artificial intelligence models you can run entirely on the hardware. When a model fits completely within this limit, it executes at the maximum speed the graphics processor can deliver.

To fit larger models into the available memory, developers use quantization. The quant column indicates the compression level applied to the model weights. For example, a Q4_K_M quant uses approximately four bits per weight, while a Q6_K quant uses six bits. Higher quantization levels like Q6_K preserve more model accuracy but require more memory. Lower quantization levels like Q4_K_M reduce memory usage at the cost of some precision.

With 8 GB of video memory, you can run several capable models locally. The Mochi 1 10B model fits at a Q4_K_M quant using 7.3 GB of memory. You can also run Gemma 2 9B at Q4_K_M using exactly 8 GB of memory. Other options include Nemotron Nano 9B, GLM-4 9B, Yi-Coder 9B, and GLM-4V-9B, which all run at Q5_K_M using 7.7 GB of memory. Llama 3.1 8B fits at Q5_K_M using 7.4 GB of memory. Models like Granite 3.3 8B, Ministral 8B, and Stable Diffusion 3.5 Large fit at Q6_K using 7.9 GB of memory. Popular 7B models like Mistral 7B and Qwen2.5 7B fit easily at Q6_K, using 7.4 GB and 6.9 GB of memory respectively.

If you want to run models that exceed the 8 GB video memory limit, you must use CPU offload. This process splits the model layers between your graphics card and your system RAM. Assuming you have 32 GB of system RAM, you can run larger models with a performance penalty. For example, FLUX.1 dev requires 14.4 GB of memory at FP8, which uses 16.4 GB of system RAM. Gemma 3 12B, Mistral NeMo 12B, and FLUX.1 schnell require 8.8 GB of memory at Q4_K_M, which uses 10.8 GB of system RAM. Vicuna 13B requires 9.5 GB of memory at Q4_K_M, which uses 11.5 GB of system RAM. Offloading layers to the CPU slows down processing speed significantly because system RAM is much slower than GDDR5 video memory.

When planning your memory budget, you must consider the context window. The memory figures listed are calculated using a standard 4k context window. If you increase the context window to process longer documents or chat histories, the memory usage will increase. This extra memory requirement can push a model past the 8 GB limit of your graphics card, forcing the system to use slower CPU offload.