Best local AI models for NVIDIA GTX 670MX

The largest models that fit fully

The 30 largest of the 76 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The NVIDIA GTX 670MX is a mobile graphics card equipped with 3 GB of GDDR5 memory. This dedicated memory size determines which artificial intelligence models can run entirely on your hardware. To load a model successfully, the model files and the active processing data must fit within this 3 GB limit. Running out of video memory will cause the software to crash or slow down significantly.

Quantization is a method that compresses model files to save space. The quant column shows the specific compression level used to fit each model. For example, a Q4_K_M quant represents a four bit compression that keeps good accuracy while lowering memory use. A Q6_K or Q8_0 quant offers higher precision but requires more memory. Choosing the right quant is necessary to balance output quality and your limited video memory.

Several capable models fit completely inside your 3 GB limit. The Qwen3 4B, Gemma 3 4B, Gemma 4 E4B, MiniCPM 3 4B, Danube 3 4B, and Fish Speech 1.5 / OpenAudio S1 models all use a Q4_K_M quant and require 2.9 GB of memory. You can also run Phi-4-mini-instruct, Phi-3.5 Mini, and OmniGen / OmniGen2 at Q4_K_M quants using 2.8 GB of memory. Image generation is possible with SD Cascade (Würstchen v3) using 2.6 GB of memory at a Q4_K_M quant.

Other options include SDXL Turbo, SDXL Lightning, and ACE-Step which use a Q5_K_M quant and require exactly 3 GB of memory. MusicGen small/medium/large fits with a Q5_K_M quant using 2.8 GB. You can run SmolLM3 3B, Replit Code v1.5 3B, Kandinsky 3.1, Voxtral Mini / Small, Orpheus TTS, and Higgs Audio v2 at a Q6_K quant using 3 GB. Smaller models like Stable Diffusion 3.5 Medium require 2.5 GB at a Q6_K quant, while SmolVLM 256M / 500M / 2B uses 2.5 GB at a Q8_0 quant.

If you want to run larger models, you must use CPU offload. This technique shares the workload between your graphics card and your system RAM. We assume your system has 32 GB of system RAM for these cases. For example, Mistral 7B needs 5.7 GB at a Q4_K_M quant and requires 7.7 GB of system RAM. Phi-4-multimodal needs 4.1 GB at a Q4_K_M quant and requires 6.1 GB of system RAM. Offloading allows you to run these larger models but it reduces processing speed.

When running local text models, you must consider the context window. The standard 4k context window refers to the amount of text the model can remember during a conversation. Keeping a full 4k context requires extra video memory for the system state. If your model already uses close to 3 GB of memory, you may need to reduce the context length to prevent memory errors.