Best local AI models for NVIDIA GTX 1660 Ti
6 GB GDDR6. At a 4k context, 114 of the 233 models in our catalog with verified parameter counts fit fully, up to Granite 3.3 2B / 8B at 8B parameters.
Check your own machine against every model →The largest models that fit fully
The 30 largest of the 114 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.
| Model | Parameters | Best quant that fits | Memory used at 4k |
|---|---|---|---|
| Granite 3.3 2B / 8B | 8B | Q4_K_M | 5.9 GB |
| Ministral 3B / 8B | 8B | Q4_K_M | 5.9 GB |
| InternLM 3 8B | 8B | Q4_K_M | 5.9 GB |
| OpenCoder 1.5B / 8B | 8B | Q4_K_M | 5.9 GB |
| Seed-Coder 8B | 8B | Q4_K_M | 5.9 GB |
| MiniCPM-V 2.6 / MiniCPM-o 2.6 | 8B | Q4_K_M | 5.9 GB |
| Idefics 3 8B | 8B | Q4_K_M | 5.9 GB |
| Fuyu-8B | 8B | Q4_K_M | 5.9 GB |
| Emu3 | 8B | Q4_K_M | 5.9 GB |
| Stable Diffusion 3.5 Large / Turbo | 8B | Q4_K_M | 5.9 GB |
| EXAONE 3.5 2.4B / 7.8B | 7.8B | Q4_K_M | 5.7 GB |
| Mistral 7B | 7B | Q4_K_M | 5.7 GB |
| Qwen2.5 0.5B / 1.5B / 3B / 7B | 7B | Q5_K_M | 6 GB |
| OLMo 2 1B / 7B | 7B | Q5_K_M | 6 GB |
| Falcon 3 1B / 3B / 7B | 7B | Q5_K_M | 6 GB |
| Command R7B | 7B | Q5_K_M | 6 GB |
| OpenHermes 2.5 | 7B | Q5_K_M | 6 GB |
| Zephyr 7B Beta | 7B | Q5_K_M | 6 GB |
| OpenChat 3.5 | 7B | Q5_K_M | 6 GB |
| Starling LM 7B | 7B | Q5_K_M | 6 GB |
| Codestral Mamba 7B | 7B | Q5_K_M | 6 GB |
| CodeGemma 2B / 7B | 7B | Q5_K_M | 6 GB |
| aiXcoder-7B | 7B | Q5_K_M | 6 GB |
| Nxcode / CodeQwen 1.5 7B | 7B | Q5_K_M | 6 GB |
| Janus-Pro 1B / 7B | 7B | Q5_K_M | 6 GB |
| Ruyi-Mini-7B | 7B | Q5_K_M | 6 GB |
| Qwen2-Audio 7B | 7B | Q5_K_M | 6 GB |
| Qwen2.5-Omni 3B / 7B | 7B | Q5_K_M | 6 GB |
| YuE | 7B | Q5_K_M | 6 GB |
| Magicoder-S-DS 6.7B | 6.7B | Q5_K_M | 5.7 GB |
Close, but only with CPU offload
These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.
| Model | Parameters | Memory at Q4_K_M | System RAM at 4k |
|---|---|---|---|
| Llama 3.1 8B | 8B | 6.4 GB needed | 8.4 GB |
| Chroma | 8.9B | 6.5 GB needed | 8.5 GB |
| Gemma 2 9B | 9B | 8 GB needed | 10 GB |
| Nemotron Nano 4B / 9B | 9B | 6.6 GB needed | 8.6 GB |
| GLM-4 9B / GLM-4.5-Air | 9B | 6.6 GB needed | 8.6 GB |
| Yi-Coder 1.5B / 9B | 9B | 6.6 GB needed | 8.6 GB |
| GLM-4-9B-Chat / CodeGeeX4 | 9B | 6.6 GB needed | 8.6 GB |
| GLM-4V-9B / GLM-4.1V-Thinking | 9B | 6.6 GB needed | 8.6 GB |
| Mochi 1 | 10B | 7.3 GB needed | 9.3 GB |
| Open-Sora 2.0 | 11B | 8.1 GB needed | 10.1 GB |
How to read this
The NVIDIA GTX 1660 Ti graphics card features 6 GB of GDDR6 memory. This memory size determines which local AI models you can run entirely on your hardware. To fit a model into this space, you must look at the model size and the quantization level. Quantization reduces the precision of model weights to save space. The quant column shows the best option that fits your hardware limits.
For models up to 8B parameters, the Q4_K_M quantization is the best choice to fit inside your 6 GB limit. Granite 3.3 8B, Ministral 8B, InternLM 3 8B, OpenCoder 8B, Seed-Coder 8B, MiniCPM-V 2.6, MiniCPM-o 2.6, Idefics 3 8B, Fuyu-8B, Emu3, and Stable Diffusion 3.5 Large use 5.9 GB of memory at this quantization. EXAONE 3.5 7.8B and Mistral 7B also fit well at Q4_K_M using 5.7 GB of memory.
You can run 7B models at a higher quality Q5_K_M quantization. Qwen2.5 7B, OLMo 2 7B, Falcon 3 7B, Command R7B, OpenHermes 2.5, Zephyr 7B Beta, OpenChat 3.5, Starling LM 7B, Codestral Mamba 7B, CodeGemma 7B, aiXcoder-7B, Nxcode, CodeQwen 1.5 7B, Janus-Pro 7B, Ruyi-Mini-7B, Qwen2-Audio 7B, Qwen2.5-Omni 7B, and YuE use exactly 6 GB of memory at Q5_K_M. Magicoder-S-DS 6.7B fits at Q5_K_M using 5.7 GB of memory.
If a model exceeds your 6 GB memory limit, you must use CPU offload. This process splits the model between your graphics card and your system RAM. CPU offload allows you to run larger models, but it significantly reduces your generation speed. We assume you have 32 GB of system RAM available for these offload scenarios.
Several models require this offload setup. Llama 3.1 8B needs 6.4 GB at Q4_K_M and 8.4 GB of system RAM. Chroma needs 6.5 GB at Q4_K_M and 8.5 GB of system RAM. Gemma 2 9B needs 8 GB at Q4_K_M and 10 GB of system RAM. Nemotron Nano 9B, GLM-4 9B, Yi-Coder 9B, GLM-4-9B-Chat, CodeGeeX4, GLM-4V-9B, and GLM-4.1V-Thinking all need 6.6 GB at Q4_K_M and 8.6 GB of system RAM. Mochi 1 needs 7.3 GB at Q4_K_M and 9.3 GB of system RAM. Open-Sora 2.0 needs 8.1 GB at Q4_K_M and 10.1 GB of system RAM.
Be aware of the 4k context limit when running these models. The memory numbers listed here are calculated for a standard 4k context window. If you increase the context window to process longer texts, the model will require more memory. This extra memory demand can cause your system to slow down or run out of memory entirely.