Best local AI models for NVIDIA GTX 660

The largest models that fit fully

The 30 largest of the 56 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The NVIDIA GTX 660 graphics card features 2 GB of GDDR5 video memory. This hardware specification determines the size of the artificial intelligence models you can run locally. To load and execute a model entirely on the graphics hardware, the total memory footprint of the model must remain under this 2 GB limit. Running models locally ensures data privacy and eliminates subscription fees.

The quant column indicates the quantization level used to compress the model weights. Quantization reduces the precision of the numerical values in a model to save space. For example, the Allegro 2.8B model fits in 2 GB of video memory when compressed to the Q4_K_M quantization level. Similarly, the SmolVLM 2B model fits in 2 GB of video memory using the Q6_K quantization level. Higher quantization levels like Q8_0 preserve more original quality but require more memory.

Smaller models can run entirely inside the video memory of the graphics card. The TinyLlama 1.1B model uses 1.4 GB of video memory at the Q8_0 quantization level. The Qwen3 1.7B model uses 1.7 GB of video memory at the Q6_K quantization level. The Canary 1B / Qwen-2.5B model uses 1.8 GB of video memory at the Q4_K_M quantization level. These configurations run at maximum speed because the graphics processor does not have to wait for system memory.

When a model exceeds the 2 GB video memory limit, you must use CPU offload. This technique stores part of the model in your system RAM. For example, the SmolLM3 3B model needs 2.2 GB of video memory at the Q4_K_M quantization level and requires 4.2 GB of system RAM. The Stable Diffusion XL model needs 4.1 GB of video memory at the FP8 / optimized level and requires 6.1 GB of system RAM. CPU offload allows you to run larger models but reduces processing speed significantly.

You must also consider the memory cost of the context window. The listed memory usage figures assume a standard 4k context window. If you increase the context window to process longer texts, the memory usage will rise. This extra memory requirement can push a model past the 2 GB limit of your graphics card. You may need to use a smaller model or a lower quantization level to keep the context window functional.