Best local AI models for NVIDIA GTX 670

The largest models that fit fully

The 30 largest of the 56 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The NVIDIA GTX 670 is a legacy graphics card equipped with 2 GB of GDDR5 video memory. This memory size is the absolute limit for running local AI models entirely on the graphics hardware. To load a model successfully, the model files and the active memory space must fit within this 2 GB boundary. If a model exceeds this limit, the system will fail to run it or experience severe performance drops.

To fit modern AI models into this limited space, you must use quantized models. The quantization level indicates how much the model weights are compressed. For example, a Q4_K_M quantization compresses weights to roughly four bits, which allows larger models like the 2.8B Allegro or the 2.7B Open-Sora Plan to fit into exactly 2 GB of used VRAM. Higher quants like Q6_K or Q8_0 offer better accuracy but require more memory per parameter, limiting you to smaller models.

When using Q6_K quantization, you can run models such as the 2B SmolVLM, 2B Stable Diffusion 3 Medium, or 1.9B Moondream 2. These models use up to 2 GB of VRAM. If you choose Q8_0 quantization, the maximum model size drops further. Under Q8_0, you can run the 1.6B StableLM 2, the 1.55B Whisper Large v3, or the 1.1B TinyLlama, which uses 1.4 GB of VRAM.

You can run larger models by offloading parts of the workload to your system RAM. If you have a 32 GB system RAM setup, you can run models that exceed the 2 GB VRAM limit. For example, the 3.5B SDXL Turbo requires 2.6 GB of VRAM at Q4_K_M and 4.6 GB of system RAM. Similarly, the 3.417B Stable Diffusion XL requires 4.1 GB at FP8 and 6.1 GB of system RAM. Offloading allows these models to run, but it costs significant processing speed because system RAM is much slower than GDDR5 video memory.

When running local text models, you must also consider the context window. The memory figures listed here represent the model at its base state. Generating long responses or inputting large prompts increases memory usage. Running a model near the 2 GB limit means you must keep your context window short, typically around 4k tokens, to prevent out of memory errors during generation.