Best local AI models for NVIDIA GTX 660M

The largest models that fit fully

The 30 largest of the 56 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The NVIDIA GTX 660M is an older mobile graphics card equipped with 2 GB of GDDR5 memory. This dedicated video memory determines the maximum size of the artificial intelligence models you can run directly on the hardware. Because the physical memory limit is strict, running local models requires careful selection of model parameters and quantization levels to avoid out of memory errors.

Quantization is a compression method that reduces the precision of model weights to save memory. The quant column indicates the optimal format for each model on this hardware. For example, the Allegro 2.8B model fits within the limit using the Q4_K_M quantization which uses exactly 2 GB of video memory. Other models like the SmolVLM 2B use a higher quality Q6_K quantization to occupy 2 GB of memory.

When a model exceeds the physical video memory of your graphics card, you must use CPU offload. This technique splits the model layers between your video memory and your system RAM. If you have 32 GB of system RAM, you can run larger models such as the SmolLM3 3B or Replit Code v1.5 3B. These models require 2.2 GB of video memory at Q4_K_M quantization and an additional 4.2 GB of system RAM.

Using CPU offload comes with a performance cost. Transferring data between the system RAM and the graphics card over the system bus is much slower than accessing the GDDR5 memory directly. While offloading allows you to run larger models like the 3.5B parameter SDXL Turbo or SDXL Lightning, the generation speed will be significantly lower than running a smaller model entirely on the graphics card.

You must also consider the context window when running local text models. The memory figures listed are for the base model weights only. Running a model with a long context window like 4k tokens requires extra memory for the key value cache. If you generate long responses or input large prompts, you may need to choose a smaller model like the TinyLlama 1.1B which uses 1.4 GB of video memory at Q8_0 quantization to leave room for the context.