Best local AI models for NVIDIA GTX 660 Ti

The largest models that fit fully

The 30 largest of the 56 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The NVIDIA GTX 660 Ti features 2 GB of GDDR5 video memory. This memory capacity determines which AI models can run locally on your hardware. Models exceeding 2 GB require system RAM to function. You must ensure your computer has at least 32 GB of system RAM to support these larger configurations.

The quant column refers to quantization. This process reduces the precision of model weights to save space. A lower precision allows larger models to fit into your limited video memory. For example, the Allegro 2.8B model uses the Q4_K_M quant to fit within 2 GB. Higher quants like Q8_0 provide better accuracy but require more memory.

Memory size indicates the amount of video memory a model occupies during operation. Models like SmolVLM 2B or Stable Diffusion 3 Medium use 2 GB of memory at the Q6_K quant. You must keep total usage below the 2 GB limit to avoid performance drops. If a model exceeds this limit, the system moves data to your slower system RAM.

CPU offload allows you to run models that are too large for your video card. Models such as SmolLM3 3B or Kandinsky 3.1 require 2.2 GB of memory at Q4_K_M. Because this exceeds your 2 GB limit, the system offloads the remainder to your 32 GB of system RAM. This process increases the time required for each inference step.

The 4k context caveat is important for all models. Context length refers to the amount of text or data a model can process at once. Increasing the context length consumes additional memory beyond the base model size. If you push the context limit while running a model near the 2 GB threshold, you will likely exceed your available video memory.

You can run various models depending on your specific needs. For image generation, options include Stable Diffusion 3.5 Medium at 1.8 GB or Playground v2.5 at 1.9 GB. For audio tasks, you might choose Parler TTS at 1.9 GB or Whisper Large v3 at 2 GB. Always verify the quant and memory usage before starting a new model.