Best local AI models for NVIDIA GTX 880M

The largest models that fit fully

The 30 largest of the 123 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The NVIDIA GTX 880M is a mobile graphics card equipped with 8 GB GDDR5 memory. This physical memory size determines the maximum size of the artificial intelligence models you can run locally. To fit inside this limit, models must reside entirely within the onboard graphics memory. If a model exceeds this capacity, execution will either fail or slow down significantly.

Quantization is a method that compresses model files to save space. The quant column shows the best available quantization level that still fits within your hardware limits. For example, Gemma 2 9B fits at the Q4_K_M quantization level which uses exactly 8 GB of graphics memory. Other models like Llama 3.1 8B can run at a higher quality Q5_K_M quantization level while using 7.4 GB of memory.

Smaller models can run at even higher quantization levels on this hardware. Granite 3.3 8B, Ministral 8B, and InternLM 3 8B all run at the Q6_K quantization level using 7.9 GB of memory. Popular 7B models like Mistral 7B use 7.4 GB at Q6_K. Other options like Qwen2.5 7B, OLMo 2 7B, and Falcon 3 7B use 6.9 GB at the same Q6_K level.

You can run larger models by offloading parts of the workload to your system RAM. This process requires a computer with at least 32 GB of system memory. Offloading allows you to run Gemma 3 12B or Mistral NeMo 12B at Q4_K_M. These models require 8.8 GB of graphics memory and 10.8 GB of system RAM. Offloading makes larger models run but it reduces processing speed because system RAM is slower than graphics memory.

Context window size also affects memory consumption. The memory figures listed here assume a standard 4k context window. If you increase the context window to process longer documents, the model will require more memory. You may need to select a smaller model or a lower quantization level to prevent your graphics card from running out of memory during long conversations.