Best local AI models for NVIDIA RTX 2080 SUPER MAX-Q

The largest models that fit fully

The 30 largest of the 123 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The NVIDIA RTX 2080 SUPER MAX-Q is a mobile graphics card equipped with 8 GB of GDDR6 memory. This dedicated video memory determines the size of the artificial intelligence models you can run locally. To fit a model entirely on your hardware, its active size must remain under this 8 GB limit. Running models fully within your video memory ensures the fastest processing speeds for text generation and image creation.

The quantization column shows the compression level applied to each model. Quantization reduces the precision of model weights to save space. For example, a Q4_K_M quant uses a four bit format to compress weights, while a Q6_K quant uses a six bit format. Higher quants like Q6_K retain more original model quality but require more memory. Lower quants like Q4_K_M allow larger models to fit inside your 8 GB limit at a small cost to accuracy.

Many capable models fit completely within your video memory. The Mochi 1 10B model fits at a Q4_K_M quant using 7.3 GB of memory. You can run Gemma 2 9B at Q4_K_M using exactly 8 GB. Several 9B models like Nemotron Nano 4B / 9B, GLM-4 9B / GLM-4.5-Air, Yi-Coder 1.5B / 9B, GLM-4-9B-Chat / CodeGeeX4, and GLM-4V-9B / GLM-4.1V-Thinking fit at Q5_K_M using 7.7 GB. Chroma 8.9B fits at Q5_K_M using 7.6 GB, and Llama 3.1 8B fits at Q5_K_M using 7.4 GB.

Other 8B models fit at a higher Q6_K quant using 7.9 GB of video memory. These include Granite 3.3 2B / 8B, Ministral 3B / 8B, InternLM 3 8B, OpenCoder 1.5B / 8B, Seed-Coder 8B, MiniCPM-V 2.6 / MiniCPM-o 2.6, Idefics 3 8B, Fuyu-8B, Emu3, and Stable Diffusion 3.5 Large / Turbo. The EXAONE 3.5 2.4B / 7.8B model fits at Q6_K using 7.7 GB. Mistral 7B fits at Q6_K using 7.4 GB. You can also run Qwen2.5 0.5B / 1.5B / 3B / 7B, OLMo 2 1B / 7B, Falcon 3 1B / 3B / 7B, Command R7B, OpenHermes 2.5, Zephyr 7B Beta, OpenChat 3.5, Starling LM 7B, and Codestral Mamba 7B at Q6_K using 6.9 GB.

When a model is too large for your 8 GB video memory, you can offload parts of it to your system RAM. This offloading requires a system with 32 GB of system RAM. Offloading allows you to run Open-Sora 2.0 11B at Q4_K_M, which needs 8.1 GB of video memory and 10.1 GB of system RAM. You can run FLUX.1 dev 12B at FP8 / optimized using 14.4 GB of video memory and 16.4 GB of system RAM. Gemma 3 12B, Gemma 4 12B, Mistral NeMo 12B, Pixtral 12B, FLUX.1 schnell 12B, FLUX.1 Kontext dev 12B, and FLUX.1 Krea dev 12B all need 8.8 GB of video memory and 10.8 GB of system RAM at Q4_K_M. Vicuna 13B needs 9.5 GB of video memory and 11.5 GB of system RAM at Q4_K_M.

Offloading comes with a performance cost. Moving data between your system RAM and your graphics card is much slower than keeping everything in video memory. This transfer speed bottleneck will significantly reduce your generation speed. Additionally, all memory calculations assume a standard 4k context window. If you increase the context window to process longer texts, the memory usage will rise and may cause the model to exceed your available video memory.