Best local AI models for NVIDIA GTX 675MX

The largest models that fit fully

The 30 largest of the 56 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The NVIDIA GTX 675MX is a mobile graphics card equipped with 2 GB of GDDR5 memory. This physical memory size is the absolute limit for running local AI models entirely on the graphics hardware. When a model is loaded, its weights must fit within this 2 GB space to run at hardware speed. If a model exceeds this capacity, it cannot run on the graphics card alone.

To fit larger models into this limited memory, developers use quantization. The quant column shows the specific compression level used for each model. For example, Q4_K_M represents a four bit quantization that reduces model size while keeping acceptable accuracy. Higher quants like Q6_K and Q8_0 offer better quality but require more memory space. A model like Allegro 2.8B fits in 2 GB using a Q4_K_M quant, while the smaller TinyLlama 1.1B can use a high quality Q8_0 quant and consume only 1.4 GB.

When a model size exceeds the 2 GB limit, you must use CPU offload. This process splits the model between your graphics card and your system RAM. For example, running SmolLM3 3B at Q4_K_M requires 2.2 GB of graphics memory and 4.2 GB of system RAM. While offloading allows you to run larger models like MusicGen or Stable Diffusion XL, it comes with a heavy speed cost. Data must travel between the system RAM and the graphics card, which slows down generation speeds significantly.

Users must also consider the context window when running text models. The memory figures listed are for the base model weights only. Running a model with a standard 4k context window requires additional memory to store the active conversation history. On a card with 2 GB of memory, active context will quickly consume the remaining space, which may force the system to offload data to system RAM and slow down performance.

For image generation, models like Stable Diffusion 3 Medium fit within the 2 GB limit using a Q6_K quant. Larger options like SDXL Turbo require CPU offloading, needing 2.6 GB of graphics memory and 4.6 GB of system RAM. By balancing model size, quantization levels, and offloading, you can still run a variety of audio, image, and text models on this legacy hardware.