Best local AI models for NVIDIA GTX 960A
2 GB GDDR5. At a 4k context, 56 of the 233 models in our catalog with verified parameter counts fit fully, up to Allegro at 2.8B parameters.
Check your own machine against every model →The largest models that fit fully
The 30 largest of the 56 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.
| Model | Parameters | Best quant that fits | Memory used at 4k |
|---|---|---|---|
| Allegro | 2.8B | Q4_K_M | 2 GB |
| Open-Sora Plan | 2.7B | Q4_K_M | 2 GB |
| LFM2 1.2B / 2.6B | 2.6B | Q4_K_M | 1.9 GB |
| Playground v2.5 | 2.6B | Q4_K_M | 1.9 GB |
| Stable Diffusion 3.5 Medium | 2.5B | Q4_K_M | 1.8 GB |
| Canary 1B / Qwen-2.5B | 2.5B | Q4_K_M | 1.8 GB |
| SeamlessM4T v2 | 2.3B | Q5_K_M | 2 GB |
| Parler-TTS | 2.2B | Q5_K_M | 1.9 GB |
| Kimi K3 DSpark | 2.2B | Q5_K_M | 2 GB |
| SmolVLM 256M / 500M / 2B | 2B | Q6_K | 2 GB |
| Stable Diffusion 3 Medium | 2B | Q6_K | 2 GB |
| Pyramid Flow | 2B | Q6_K | 2 GB |
| Wav2Vec2 / XLS-R | 2B | Q6_K | 2 GB |
| Moondream 2 | 1.9B | Q6_K | 1.9 GB |
| Qwen3 1.7B | 1.7B | Q6_K | 1.7 GB |
| SmolLM2 135M / 360M / 1.7B | 1.7B | Q6_K | 1.7 GB |
| StableLM 2 1.6B | 1.6B | Q8_0 | 2 GB |
| Sana 0.6B / 1.6B | 1.6B | Q8_0 | 2 GB |
| Zonos 0.1 | 1.6B | Q8_0 | 2 GB |
| Dia 1.6B | 1.6B | Q8_0 | 2 GB |
| Whisper Large v3 | 1.55B | Q8_0 | 2 GB |
| ControlNet / T2I-Adapter / IP-Adapter | 1.5B | Q8_0 | 1.9 GB |
| Hunyuan-DiT | 1.5B | Q8_0 | 1.9 GB |
| Stable Video Diffusion | 1.5B | Q8_0 | 1.9 GB |
| Whisper Large v2 / turbo | 1.5B | Q8_0 | 1.9 GB |
| AudioGen | 1.5B | Q8_0 | 1.9 GB |
| AudioLDM 2 | 1.5B | Q8_0 | 1.9 GB |
| Tango 2 | 1.4B | Q8_0 | 1.8 GB |
| TinyLlama 1.1B | 1.1B | Q8_0 | 1.4 GB |
| SantaCoder 1.1B | 1.1B | Q8_0 | 1.4 GB |
Close, but only with CPU offload
These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.
| Model | Parameters | Memory at Q4_K_M | System RAM at 4k |
|---|---|---|---|
| SmolLM3 3B | 3B | 2.2 GB needed | 4.2 GB |
| Replit Code v1.5 3B | 3B | 2.2 GB needed | 4.2 GB |
| Kandinsky 3.1 | 3B | 2.2 GB needed | 4.2 GB |
| Voxtral Mini / Small | 3B | 2.2 GB needed | 4.2 GB |
| Orpheus TTS | 3B | 2.2 GB needed | 4.2 GB |
| Higgs Audio v2 | 3B | 2.2 GB needed | 4.2 GB |
| MusicGen small/medium/large | 3.3B | 2.4 GB needed | 4.4 GB |
| Stable Diffusion XL | 3.417B | 4.1 GB needed | 6.1 GB |
| SDXL Turbo | 3.5B | 2.6 GB needed | 4.6 GB |
| SDXL Lightning | 3.5B | 2.6 GB needed | 4.6 GB |
How to read this
The NVIDIA GTX 960A is an entry level graphics card with 2 GB GDDR5 memory. This memory size is the main limit for running local AI models. The model must fit inside this memory to run fast. If a model is too large it will not fit on the card. You must use smaller models or use system memory to help.
The quant column shows the compression level of the model. Quantization reduces the size of the model so it uses less memory. A Q4_K_M quant uses less space than a Q8_0 quant. For example Allegro 2.8B fits in 2 GB of memory using the Q4_K_M quant. Smaller models like TinyLlama 1.1B can run at the higher quality Q8_0 quant using 1.4 GB of memory.
You can run larger models by using CPU offload if your computer has 32 GB system RAM. Offload means some parts of the model run on your system memory instead of the graphics card. This process lets you run models that exceed the 2 GB limit. However offload makes the model run much slower because system RAM is slower than graphics card memory.
Several models can run with CPU offload. MusicGen small/medium/large is a 3.3B model that needs 2.4 GB at Q4_K_M and uses 4.4 GB system RAM. Stable Diffusion XL is a 3.417B model that needs 4.1 GB at FP8 / optimized and uses 6.1 GB system RAM. SDXL Turbo is a 3.5B model that needs 2.6 GB at Q4_K_M and uses 4.6 GB system RAM.
When you run these models you must watch the context window size. Running a model with a 4k context window uses more memory than running it with a shorter context. If you increase the context length the model might exceed the 2 GB memory limit of your card. This will cause the model to slow down or fail to load.