Best local AI models for NVIDIA GT 640M

The largest models that fit fully

The 30 largest of the 56 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The NVIDIA GT 640M is a mobile graphics card equipped with 2 GB of DDR3 memory. This dedicated video memory determines the size of the artificial intelligence models you can run directly on the hardware. To run a model entirely on the graphics processor, the model files and the active memory space must fit within this 2 GB limit.

The quantization level indicates how much the model weights are compressed. Quantization levels like Q4_K_M or Q6_K reduce the memory footprint of a model so it can fit into the 2 GB space. Higher quantization levels like Q8_0 offer better accuracy but require more memory. Lower quantization levels allow larger models to fit but may reduce output quality.

Several models can fit entirely within the 2 GB memory limit of the card. The Allegro 2.8B model fits using the Q4_K_M quant and uses 2 GB of memory. The Open-Sora Plan 2.7B model also fits using the Q4_K_M quant with 2 GB used. You can also run LFM2 2.6B, Playground v2.5 2.6B, Stable Diffusion 3.5 Medium 2.5B, or Canary 1B / Qwen-2.5B 2.5B using the Q4_K_M quant. Other options include SeamlessM4T v2 2.3B, Parler-TTS 2.2B, and Kimi K3 DSpark 2.2B using the Q5_K_M quant.

Smaller models can use higher quality quantizations. SmolVLM 256M / 500M / 2B, Stable Diffusion 3 Medium 2B, Pyramid Flow 2B, and Wav2Vec2 / XLS-R 2B all fit using the Q6_K quant with 2 GB used. Moondream 2 1.9B, Qwen3 1.7B, and SmolLM2 135M / 360M / 1.7B also run well within the memory limit. Models like StableLM 2 1.6B, Sana 0.6B / 1.6B, Zonos 0.1 1.6B, Dia 1.6B, and Whisper Large v3 1.55B can use the high quality Q8_0 quant.

When a model is too large for the 2 GB video memory, you can offload parts of it to your system RAM. This offloading process allows you to run larger models but slows down processing speeds. For example, SmolLM3 3B, Replit Code v1.5 3B, Kandinsky 3.1 3B, Voxtral Mini / Small 3B, Orpheus TTS 3B, and Higgs Audio v2 3B require 2.2 GB of video memory at Q4_K_M and 4.2 GB of system RAM. MusicGen small/medium/large 3.3B requires 2.4 GB of video memory at Q4_K_M and 4.4 GB of system RAM.

Very large models require even more system RAM. Stable Diffusion XL 3.417B needs 4.1 GB of video memory at FP8 / optimized and 6.1 GB of system RAM. SDXL Turbo 3.5B and SDXL Lightning 3.5B both need 2.6 GB of video memory at Q4_K_M and 4.6 GB of system RAM. These requirements assume your computer has 32 GB of total system RAM installed.

Running models at a standard 4k context window increases memory usage significantly. The memory figures listed represent the base model requirements. Generating long text or processing large inputs will require additional memory beyond these base numbers. You may need to reduce the context window size to avoid running out of memory on this hardware.