Best local AI models for NVIDIA GTX 870M

The largest models that fit fully

The 30 largest of the 76 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The NVIDIA GTX 870M is a mobile graphics card equipped with 3 GB of GDDR5 video memory. This memory size determines which artificial intelligence models can run directly on your hardware. To fit inside this limit, models must use quantization. The quantization column shows the compression level used to shrink the model files. A lower quantization like Q4_K_M reduces the memory footprint but slightly lowers output quality, while a higher quantization like Q8_0 or Q6_K preserves more original accuracy.

For local text generation, several 4B models fit within the 3 GB limit. Qwen3 4B, Gemma 3 4B, Gemma 4 E4B, MiniCPM 3 4B, and Danube 3 4B all run at the Q4_K_M quantization level using 2.9 GB of video memory. You can also run Phi-4-mini-instruct and Phi-3.5 Mini, which are 3.8B models that use 2.8 GB of memory at Q4_K_M quantization. When running these models, you must watch your context length. Running a 4k context window requires extra memory for the key value cache, which can easily exceed your 3 GB limit and cause errors.

Audio and speech generation models are also highly compatible with this hardware. Fish Speech 1.5 / OpenAudio S1 is a 4B model that uses 2.9 GB of memory at Q4_K_M quantization. MusicGen small/medium/large is a 3.3B model requiring 2.8 GB of memory at Q5_K_M quantization. Voxtral Mini / Small, Orpheus TTS, and Higgs Audio v2 are 3B models that use 3 GB of memory at Q6_K quantization. For translation tasks, SeamlessM4T v2 is a 2.3B model that fits within 2.9 GB of memory at Q8_0 quantization, while Parler-TTS is a 2.2B model using 2.8 GB of memory at Q8_0 quantization.

Image generation models can run locally on your system. SDXL Turbo, SDXL Lightning, and ACE-Step are 3.5B models that use exactly 3 GB of video memory at Q5_K_M quantization. SD Cascade (Würstchen v3) is a 3.6B model using 2.6 GB of memory at Q4_K_M quantization. You can also run Kandinsky 3.1 at Q6_K quantization using 3 GB of memory, or Playground v2.5 at Q6_K quantization using 2.6 GB of memory. Stable Diffusion 3.5 Medium is a 2.5B model that fits within 2.5 GB of memory at Q6_K quantization.

If you want to run larger models, you must use CPU offload to share the workload with your system RAM. This process allows you to run models that exceed 3 GB, but it lowers your generation speed. For example, running Mistral 7B at Q4_K_M quantization requires 5.7 GB of video memory and 7.7 GB of system RAM. Similarly, CogVideoX 2B / 5B requires 3.7 GB of video memory and 5.7 GB of system RAM at Q4_K_M quantization. These offload cases assume your computer has at least 32 GB of system RAM installed.