Best local AI models for NVIDIA GTX 1660 Ti MAX-Q
6 GB GDDR6. At a 4k context, 114 of the 233 models in our catalog with verified parameter counts fit fully, up to Granite 3.3 2B / 8B at 8B parameters.
Check your own machine against every model →The largest models that fit fully
The 30 largest of the 114 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.
| Model | Parameters | Best quant that fits | Memory used at 4k |
|---|---|---|---|
| Granite 3.3 2B / 8B | 8B | Q4_K_M | 5.9 GB |
| Ministral 3B / 8B | 8B | Q4_K_M | 5.9 GB |
| InternLM 3 8B | 8B | Q4_K_M | 5.9 GB |
| OpenCoder 1.5B / 8B | 8B | Q4_K_M | 5.9 GB |
| Seed-Coder 8B | 8B | Q4_K_M | 5.9 GB |
| MiniCPM-V 2.6 / MiniCPM-o 2.6 | 8B | Q4_K_M | 5.9 GB |
| Idefics 3 8B | 8B | Q4_K_M | 5.9 GB |
| Fuyu-8B | 8B | Q4_K_M | 5.9 GB |
| Emu3 | 8B | Q4_K_M | 5.9 GB |
| Stable Diffusion 3.5 Large / Turbo | 8B | Q4_K_M | 5.9 GB |
| EXAONE 3.5 2.4B / 7.8B | 7.8B | Q4_K_M | 5.7 GB |
| Mistral 7B | 7B | Q4_K_M | 5.7 GB |
| Qwen2.5 0.5B / 1.5B / 3B / 7B | 7B | Q5_K_M | 6 GB |
| OLMo 2 1B / 7B | 7B | Q5_K_M | 6 GB |
| Falcon 3 1B / 3B / 7B | 7B | Q5_K_M | 6 GB |
| Command R7B | 7B | Q5_K_M | 6 GB |
| OpenHermes 2.5 | 7B | Q5_K_M | 6 GB |
| Zephyr 7B Beta | 7B | Q5_K_M | 6 GB |
| OpenChat 3.5 | 7B | Q5_K_M | 6 GB |
| Starling LM 7B | 7B | Q5_K_M | 6 GB |
| Codestral Mamba 7B | 7B | Q5_K_M | 6 GB |
| CodeGemma 2B / 7B | 7B | Q5_K_M | 6 GB |
| aiXcoder-7B | 7B | Q5_K_M | 6 GB |
| Nxcode / CodeQwen 1.5 7B | 7B | Q5_K_M | 6 GB |
| Janus-Pro 1B / 7B | 7B | Q5_K_M | 6 GB |
| Ruyi-Mini-7B | 7B | Q5_K_M | 6 GB |
| Qwen2-Audio 7B | 7B | Q5_K_M | 6 GB |
| Qwen2.5-Omni 3B / 7B | 7B | Q5_K_M | 6 GB |
| YuE | 7B | Q5_K_M | 6 GB |
| Magicoder-S-DS 6.7B | 6.7B | Q5_K_M | 5.7 GB |
Close, but only with CPU offload
These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.
| Model | Parameters | Memory at Q4_K_M | System RAM at 4k |
|---|---|---|---|
| Llama 3.1 8B | 8B | 6.4 GB needed | 8.4 GB |
| Chroma | 8.9B | 6.5 GB needed | 8.5 GB |
| Gemma 2 9B | 9B | 8 GB needed | 10 GB |
| Nemotron Nano 4B / 9B | 9B | 6.6 GB needed | 8.6 GB |
| GLM-4 9B / GLM-4.5-Air | 9B | 6.6 GB needed | 8.6 GB |
| Yi-Coder 1.5B / 9B | 9B | 6.6 GB needed | 8.6 GB |
| GLM-4-9B-Chat / CodeGeeX4 | 9B | 6.6 GB needed | 8.6 GB |
| GLM-4V-9B / GLM-4.1V-Thinking | 9B | 6.6 GB needed | 8.6 GB |
| Mochi 1 | 10B | 7.3 GB needed | 9.3 GB |
| Open-Sora 2.0 | 11B | 8.1 GB needed | 10.1 GB |
How to read this
The NVIDIA GTX 1660 Ti MAX-Q is a laptop graphics card equipped with 6 GB of GDDR6 memory. This memory size determines which artificial intelligence models can run directly on your hardware. If a model fits entirely within this video memory, it will process your prompts quickly. If a model exceeds this limit, you must use system memory to run it.
The quantization column shows the compression level used to fit these models. A quant like Q4_K_M or Q5_K_M represents the balance between model smarts and memory footprint. For example, Granite 3.3 8B, Ministral 8B, InternLM 3 8B, OpenCoder 8B, and Seed-Coder 8B all fit within 5.9 GB of video memory using the Q4_K_M quant. MiniCPM-V 2.6, Idefics 3 8B, Fuyu-8B, Emu3, and Stable Diffusion 3.5 Large or Turbo also fit within 5.9 GB at this same compression level.
Slightly smaller models like EXAONE 3.5 7.8B and Mistral 7B use 5.7 GB of video memory at Q4_K_M. Magicoder-S-DS 6.7B also uses 5.7 GB at the Q5_K_M quant. Other models like Qwen2.5 7B, OLMo 2 7B, Falcon 3 7B, Command R7B, OpenHermes 2.5, Zephyr 7B Beta, OpenChat 3.5, Starling LM 7B, Codestral Mamba 7B, CodeGemma 7B, aiXcoder-7B, Nxcode or CodeQwen 1.5 7B, Janus-Pro 7B, Ruyi-Mini-7B, Qwen2-Audio 7B, Qwen2.5-Omni 7B, and YuE utilize exactly 6 GB of video memory at the Q5_K_M quant.
When a model is too large for the 6 GB of video memory, you must offload parts of it to your system RAM. This offload process slows down generation speeds significantly. For this setup, we assume you have 32 GB of system RAM. Llama 3.1 8B requires 6.4 GB at Q4_K_M, which means you need 8.4 GB of system RAM to run it. Chroma requires 6.5 GB at Q4_K_M and needs 8.5 GB of system RAM. Gemma 2 9B requires 8 GB at Q4_K_M and needs 10 GB of system RAM.
Other offload options include Nemotron Nano 9B, GLM-4 9B, Yi-Coder 9B, GLM-4-9B-Chat or CodeGeeX4, and GLM-4V-9B or GLM-4.1V-Thinking. These models all require 6.6 GB at Q4_K_M and need 8.6 GB of system RAM. Mochi 1 requires 7.3 GB at Q4_K_M and needs 9.3 GB of system RAM. Open-Sora 2.0 requires 8.1 GB at Q4_K_M and needs 10.1 GB of system RAM.
Keep in mind that these memory figures are calculated using a standard 4k context window. If you increase the context window to process longer documents, the memory usage will rise. This extra memory demand can push a model that normally fits in your 6 GB of video memory into system RAM offloading.