Best local AI models for NVIDIA GTX 860M

The largest models that fit fully

The 30 largest of the 56 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The NVIDIA GTX 860M is an older mobile graphics card equipped with 2 GB of GDDR5 memory. This memory limit is the most important factor when you run local AI models. The memory size determines the maximum model size that can fit entirely on your hardware. To run these models without errors you must choose small architectures and use quantized versions.

Quantization is a method that compresses the weights of an AI model. The quant column shows the best compression level for each model. For example a Q4_K_M quant uses four bit quantization to shrink the model size. This allows larger models like the Allegro 2.8B or the Open-Sora Plan 2.7B to fit into the 2 GB memory of your card. Higher quants like Q8_0 provide better quality but they require more memory space.

When a model is slightly too large for the 2 GB video memory you must use CPU offload. This technique splits the model weights between your graphics card and your system RAM. We assume you have 32 GB of system RAM for these setups. Offloading allows you to run larger models like the SmolLM3 3B or the Kandinsky 3.1. However CPU offload costs performance because transferring data between system RAM and video memory is slow.

If you use CPU offload for image generation models like Stable Diffusion XL you will experience longer generation times. Stable Diffusion XL has 3.417B parameters and needs 4.1 GB of memory at FP8 or optimized settings. This requires 6.1 GB of system RAM to offload the extra weights. Other models like SDXL Turbo and SDXL Lightning need 2.6 GB of video memory at Q4_K_M and 4.6 GB of system RAM.

You must also consider the 4k context caveat when running text models. Running a model with a larger context window increases memory usage during inference. A model like TinyLlama 1.1B uses 1.4 GB of memory at Q8_0 which fits easily. But if you increase the context length to 4k tokens or higher the active memory will quickly exceed your 2 GB limit and cause a crash.