Best local AI models for NVIDIA RTX 2080 Ti

The largest models that fit fully

The 30 largest of the 144 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The NVIDIA RTX 2080 Ti features 11 GB GDDR6 memory. This onboard memory determines the maximum size of the artificial intelligence models you can run locally. To fit a model entirely on this graphics card, the model files and the active memory must stay under this 11 GB limit. Running models completely inside your video memory ensures the fastest possible processing speeds.

Quantization is a method that compresses model files so they require less memory. The best quantization column shows the optimal balance of size and quality for this hardware. For example, the 15B models Apriel-1.5-15B-Thinker and StarCoder2 15B fit within 11 GB used when using the Q4_K_M quantization. Other models like Phi-3 Medium, Phi-4, Phi-4-reasoning / -plus, Wan 2.2 T2I, Wan 2.1 14B, and SkyReels V2 use 10.2 GB of memory at this same Q4_K_M level.

Smaller models can run with higher precision quantizations because they have lower memory requirements. The 12B models Gemma 3 12B, Gemma 4 12B, Mistral NeMo 12B, Pixtral 12B, FLUX.1 schnell, FLUX.1 Kontext dev, and FLUX.1 Krea dev use 10.2 GB of memory with the Q5_K_M quantization. Models like Open-Sora 2.0 use 10.8 GB at Q6_K, while Llama 3.1 8B can run at the high quality Q8_0 quantization using 10.7 GB of video memory.

When a model exceeds the 11 GB video memory limit, you must use CPU offloading. This process splits the model workload between your graphics card and your system memory. Offloading allows you to run larger models, but it costs significant processing speed because system RAM is much slower than GDDR6 video memory. For these cases, we assume your computer has 32 GB of system RAM available.

Several larger models require this CPU offload setup to function. FLUX.1 dev needs 14.4 GB at FP8 or optimized settings, which requires 16.4 GB of system RAM. The Qwen2.5 14B model needs 11.6 GB at Q4_K_M and 13.6 GB of system RAM. Other offload options include DeepSeek-Coder-V2 16B, Kimi-VL A3B, Ling-Coder-Lite, HunyuanImage 2.1 / 3.0, CogVLM2, Qwen-Image, Qwen-Image-Edit, and gpt-oss-20b.

You must also consider the memory cost of text history. The memory figures listed here are measured using a standard 4k context window. If you increase the context window to remember longer conversations, the system will require more video memory. This extra memory demand can force a model that normally fits on the card to require CPU offloading instead.