Best local AI models for NVIDIA GT 645M

The largest models that fit fully

The 30 largest of the 56 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The NVIDIA GT 645M is an entry level mobile graphics card equipped with 2 GB of DDR3 video memory. This memory size is the absolute limit for running local AI models entirely on the graphics hardware. To run a model successfully on this card, the model files must fit within this 2 GB boundary. Because this video memory is limited, selecting the correct model size and quantization level is critical to prevent out of memory errors.

Quantization is a method that compresses AI models to make them smaller. In the model lists, the best quant column shows the optimal compression format for each model on this hardware. Formats like Q4_K_M or Q5_K_M reduce the precision of the model weights to save space. This compression allows larger models to fit into the 2 GB video memory, though it may slightly reduce the accuracy of the outputs.

For models that fit entirely within the graphics memory, the Allegro 2.8B model is the largest option at Q4_K_M quantization, using exactly 2 GB. Other viable options include the Open-Sora Plan 2.7B model at Q4_K_M using 2 GB, and the LFM2 2.6B model at Q4_K_M using 1.9 GB. You can also run the Stable Diffusion 3.5 Medium 2.5B model at Q4_K_M using 1.8 GB, or the SmolVLM 2B model at Q6_K using 2 GB.

Smaller models allow you to use higher quality quantization levels on this card. For example, the StableLM 2 1.6B model, the Sana 1.6B model, and the Whisper Large v3 1.55B model can all run at the high quality Q8_0 quantization level while using 2 GB of video memory. Very small models like the TinyLlama 1.1B model and the SantaCoder 1.1B model run at Q8_0 quantization while using only 1.4 GB of video memory.

When a model is too large for the 2 GB video memory, you must use CPU offload. This process splits the model between your graphics card and your system RAM. For these cases, we assume your computer has 32 GB of system RAM. For example, the SmolLM3 3B model needs 2.2 GB of video memory at Q4_K_M quantization and requires an additional 4.2 GB of system RAM. The MusicGen model at 3.3B needs 2.4 GB of video memory at Q4_K_M and 4.4 GB of system RAM.

CPU offload allows you to run larger models like Stable Diffusion XL at 3.417B, which needs 4.1 GB of video memory at FP8 or optimized settings along with 6.1 GB of system RAM. However, offloading data to system RAM over the system bus is much slower than running models directly on the graphics card. You must also remember the 4k context caveat, because increasing the context window length requires extra memory and will quickly exceed your limits.