Best local AI models for NVIDIA RTX 3080 12GB

The largest models that fit fully

The 30 largest of the 147 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The NVIDIA RTX 3080 12GB graphics card features 12 GB of fast GDDR6X memory. This onboard memory determines the maximum size of the artificial intelligence models you can run entirely on your hardware. To run a model smoothly at native speeds, both the model weights and the active working memory must fit completely inside this physical limit.

The quantization column indicates the compression level used to shrink these models. Quantization reduces the precision of model weights to save space. For example, a Q4_K_M quant represents a medium four bit quantization. A Q6_K quant represents a six bit quantization, and a Q8_0 quant represents an eight bit quantization. Higher quantization numbers preserve more original model intelligence but require more memory.

With 12 GB of VRAM, you can run several highly capable models locally. DeepSeek-Coder-V2 16B and Kimi-VL A3B fit at Q4_K_M quantization using 11.7 GB of memory. Apriel-1.5-15B-Thinker and StarCoder2 15B fit at Q4_K_M using 11 GB. Qwen2.5 14B fits at Q4_K_M using 11.6 GB. Phi-3 Medium, Phi-4, Phi-4-reasoning / -plus, Wan 2.2 T2I, Wan 2.1 (1.3B / 14B), and SkyReels V2 all fit at Q5_K_M using 11.9 GB. Vicuna 13B, HunyuanVideo, HunyuanVideo-Avatar, LTX-Video / LTX-2, and FramePack fit at Q5_K_M using 11.1 GB.

Slightly smaller models can run at even higher precision levels on this card. Gemma 3 12B, Gemma 4 12B, Mistral NeMo 12B, Pixtral 12B, FLUX.1 schnell, FLUX.1 Kontext dev, and FLUX.1 Krea dev fit at Q6_K using 11.8 GB. Open-Sora 2.0 fits at Q6_K using 10.8 GB, while Mochi 1 fits at Q6_K using 9.8 GB. Gemma 2 9B fits at Q6_K using 10.3 GB. Nemotron Nano 4B / 9B, GLM-4 9B / GLM-4.5-Air, Yi-Coder 1.5B / 9B, and GLM-4-9B-Chat / CodeGeeX4 fit at Q8_0 using 11.4 GB.

When a model exceeds 12 GB, you must offload the remaining data to your system RAM. This offload process allows you to run larger models but severely reduces processing speed. For these cases, we assume your system has 32 GB of system RAM. FLUX.1 dev needs 14.4 GB at FP8 / optimized and 16.4 GB of system RAM. Ling-Coder-Lite needs 12.3 GB at Q4_K_M and 14.3 GB of system RAM. HunyuanImage 2.1 / 3.0 needs 12.4 GB at Q4_K_M and 14.4 GB of system RAM. CogVLM2 needs 13.9 GB at Q4_K_M and 15.9 GB of system RAM.

Other offload options include Qwen-Image and Qwen-Image-Edit, which need 14.6 GB at Q4_K_M and 16.6 GB of system RAM. The gpt-oss-20b and Reka Flash 3 models need 15.4 GB at Q4_K_M and 17.4 GB of system RAM. Solar Pro and Codestral 22B both need 16.1 GB at Q4_K_M and 18.1 GB of system RAM.

Be aware of the context limit caveat when running these configurations. The memory numbers listed here are calculated using a basic 4k context window. If you increase the context window to process longer documents or longer conversations, the model will require significantly more memory. This extra memory demand can cause the model to spill over your 12 GB limit and slow down.