Best local AI models for NVIDIA GT 625M

The largest models that fit fully

The 30 largest of the 56 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The NVIDIA GT 625M is an entry level mobile graphics card equipped with 2 GB of DDR3 video memory. This dedicated memory size determines which artificial intelligence models can run directly on your hardware. Because the onboard memory is limited to 2 GB, choosing the correct model size and quantization level is essential to prevent out of memory errors during inference.

Quantization is a compression method that reduces the precision of model weights to save space. The quant column indicates the optimal format for each model to fit your hardware. For example, the Allegro 2.8B model requires a Q4_K_M quantization to fit within 2 GB of used video memory. Smaller models like Moondream 2 can run at a higher Q6_K quantization while using 1.9 GB of video memory.

When a model exceeds the physical memory of your graphics card, you must use CPU offload. This technique splits the workload between your video memory and your system RAM. If you have a system with 32 GB of system RAM, you can run larger models by offloading the extra data. For instance, SmolLM3 3B needs 2.2 GB of video memory at Q4_K_M quantization and requires an additional 4.2 GB of system RAM to function.

Using CPU offload allows you to run advanced models like Stable Diffusion XL which needs 4.1 GB at FP8 or optimized settings along with 6.1 GB of system RAM. Similarly, SDXL Turbo and SDXL Lightning both require 2.6 GB of video memory at Q4_K_M quantization and 4.6 GB of system RAM. Offloading data to system RAM prevents crashes but it significantly reduces processing speeds because system RAM is much slower than video memory.

You must also consider the memory cost of context length when running text models. The listed memory figures represent the base model size at startup. Running a model with a standard 4k context window requires additional video memory to store the active conversation history. With only 2 GB of video memory available on the GT 625M, you may need to reduce the context window or use a smaller model like TinyLlama 1.1B at Q8_0 quantization which uses 1.4 GB of video memory.