Best local AI models for NVIDIA GTX 675M

The largest models that fit fully

The 30 largest of the 56 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The NVIDIA GTX 675M is a mobile graphics card equipped with 2 GB of GDDR5 memory. This memory size is the absolute limit for running local AI models entirely on the graphics hardware. When a model runs inside this VRAM limit, execution is as fast as the hardware allows. If a model exceeds this capacity, it cannot load or must offload parts of its workload to system memory.

The quantization column shows the best compression format for each model. Quantization reduces the precision of model weights to save space. For example, the Allegro 2.8B model fits in 2 GB of VRAM using a Q4_K_M quantization. Smaller models like SmolLM2 1.7B can run at a higher Q6_K quantization while using 1.7 GB. TinyLlama 1.1B can run at Q8_0 quantization using 1.4 GB of VRAM.

When you run models that exceed 2 GB, you must use CPU offloading. This process splits the model between your graphics card and your system RAM. A system with 32 GB of system RAM can handle these larger files. For example, the MusicGen model family at 3.3B parameters needs 2.4 GB of VRAM at Q4_K_M quantization and requires 4.4 GB of system RAM to function.

Offloading comes with a performance cost. System RAM is much slower than the GDDR5 memory on your graphics card. Running models like Stable Diffusion XL at 3.417B parameters requires 4.1 GB of VRAM at FP8 and 6.1 GB of system RAM. This split will slow down generation speeds significantly compared to running smaller models entirely within the onboard VRAM.

Context window size also impacts memory usage. Running a model with a standard 4k context window increases the memory footprint during operation. If you generate long texts or process large prompts, the active memory will quickly exceed the 2 GB limit. You must budget your VRAM carefully and choose smaller models like Qwen3 1.7B or Sana 1.6B to keep some memory free for active context.