Best local AI models for NVIDIA RTX 4080 SUPER

The largest models that fit fully

The 30 largest of the 155 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The NVIDIA RTX 4080 SUPER features 16 GB of GDDR6X memory. This dedicated video memory determines which local AI models you can run entirely on your graphics hardware. To run a model at full speed, both the model weights and the active context window must fit within this 16 GB limit. When a model fits completely in your video memory, you get the fastest possible generation speeds.

The quant column shows the quantization level used to compress each model. Raw models are often too large for consumer hardware, so they are compressed to smaller bit widths. For example, the Q4_K_M quant uses approximately four bits per weight, while Q8_0 uses eight bits. Higher quants like Q6_K and Q8_0 preserve more original model accuracy but require more memory. Lower quants like Q4_K_M allow larger models to fit into your 16 GB limit.

The largest models that fit completely within your video memory include gpt-oss-20b and Reka Flash 3. Both are 21B models that use 15.4 GB of memory at the Q4_K_M quant. You can also run Qwen-Image and Qwen-Image-Edit at Q4_K_M, which use 14.6 GB. The 19B CogVLM2 model fits at Q4_K_M using 13.9 GB. HunyuanImage 2.1 / 3.0 fits at Q5_K_M using 14.5 GB, and Ling-Coder-Lite fits at Q5_K_M using 14.3 GB.

For maximum precision, you can run slightly smaller models at higher quants. DeepSeek-Coder-V2 16B / 236B and Kimi-VL A3B fit at Q6_K using 15.7 GB. Apriel-1.5-15B-Thinker and StarCoder2 3B / 7B / 15B fit at Q6_K using 14.8 GB. Qwen2.5 14B fits at Q6_K using 15.3 GB. Phi-3 Medium, Phi-4, Phi-4-reasoning / -plus, Wan 2.2 T2I, Wan 2.1 (1.3B / 14B), and SkyReels V2 all fit at Q6_K using 13.8 GB. Vicuna 13B, HunyuanVideo, HunyuanVideo-Avatar, LTX-Video / LTX-2, and FramePack fit at Q6_K using 12.8 GB.

Popular 12B models can run at the high Q8_0 quant. Gemma 3 12B, Gemma 4 12B, Mistral NeMo 12B, Pixtral 12B, FLUX.1 schnell, and FLUX.1 Kontext dev all use 15.3 GB at Q8_0. FLUX.1 dev fits at an optimized FP8 quant using 14.4 GB. Note that these memory figures are calculated with a standard 4k context window. If you increase the context window to process longer documents, the system will require additional video memory.

If you want to run larger models, you can offload some layers to your system RAM. This requires at least 32 GB of system memory. Offloading allows you to run Solar Pro or Codestral 22B, which need 16.1 GB at Q4_K_M and 18.1 GB of system RAM. Mistral Small 3.2, Magistral Small, and Devstral Small 1.1 need 17.6 GB at Q4_K_M and 19.6 GB of system RAM. Aria needs 18.3 GB at Q4_K_M and 20.3 GB of system RAM.

The largest offload options include Gemma 4 26B-A4B and Gemma 4 (all sizes), which need 19 GB at Q4_K_M and 21 GB of system RAM. Gemma 3 27B and Gemma 3 4B/12B/27B (vision) need 19.8 GB at Q4_K_M and 21.8 GB of system RAM. While offloading lets you run these larger models, it comes with a cost. Moving data between system RAM and video memory is slow, which significantly reduces your generation speed.