Best local AI models for NVIDIA GTX 965M

The largest models that fit fully

The 30 largest of the 56 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The NVIDIA GTX 965M is a mobile graphics card equipped with 2 GB of GDDR5 video memory. This hardware limit determines which artificial intelligence models can run directly on the graphics processor. When a model fits entirely within this video memory space, it executes at the maximum speed supported by the hardware. If a model exceeds this limit, it cannot run unless some of its workload is transferred to the system memory.

The quantization column indicates the compression level applied to each model. Quantization reduces the size of model weights to save video memory. For example, a Q4_K_M quant represents a medium four bit quantization that allows larger models like the Allegro 2.8B or Open-Sora Plan 2.7B to fit within 2 GB of video memory. Higher quants like Q8_0 preserve more original model quality but require more memory space, limiting their use to smaller models like the TinyLlama 1.1B.

To run models that exceed the 2 GB video memory limit, you must use CPU offload. This technique splits the model between your graphics card and your system RAM. For instance, running the SmolLM3 3B or Kandinsky 3.1 requires 2.2 GB of video memory at Q4_K_M and an additional 4.2 GB of system RAM. While CPU offload allows you to run larger models like Stable Diffusion XL, it introduces a performance cost because system RAM is much slower than GDDR5 video memory.

When running large language models on this hardware, you must also consider the context window. Running a model at its maximum context length, such as 4k context, requires additional video memory to store the active conversation history. This extra memory requirement can easily exceed the remaining space on a 2 GB card. You may need to reduce your context window size or use a smaller model to prevent out of memory errors.

Several compact models are optimized to fit within the limits of this graphics card. The SmolLM2 1.7B model fits within 1.7 GB of video memory using a Q6_K quant. For audio tasks, Whisper Large v3 fits within 2 GB of video memory using a Q8_0 quant. By matching the model size and quantization level to your available hardware, you can achieve stable local execution without overloading your system.