Best local AI models for NVIDIA RTX 2000 Ada Generation

The largest models that fit fully

The 30 largest of the 155 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The NVIDIA RTX 2000 Ada Generation features 16 GB GDDR6 memory. This dedicated video memory determines the size of the AI models you can run locally. To run a model entirely on the graphics card, the model files and the active context data must fit within this 16 GB limit. Keeping the entire model on the GPU ensures the fastest possible processing speeds.

Quantization is a method that compresses model weights to save memory. The quantization column shows the best balance of quality and size for each model. For example, the 21B gpt-oss-20b and Reka Flash 3 models fit in 15.4 GB of memory using the Q4_K_M quantization. Other models like Qwen-Image and Qwen-Image-Edit use 14.6 GB of memory at the same Q4_K_M level. CogVLM2 fits in 13.9 GB using Q4_K_M.

As models get smaller, you can use higher quality quantizations. HunyuanImage 2.1 / 3.0 fits in 14.5 GB using Q5_K_M. Ling-Coder-Lite uses 14.3 GB at Q5_K_M. DeepSeek-Coder-V2 16B / 236B, Kimi-VL A3B, Apriel-1.5-15B-Thinker, and StarCoder2 3B / 7B / 15B all fit using the Q6_K quantization. This Q6_K level is also used for Qwen2.5 14B, Phi-3 Medium, Phi-4, Phi-4-reasoning / -plus, Wan 2.2 T2I, Wan 2.1 (1.3B / 14B), SkyReels V2, Vicuna 13B, HunyuanVideo, HunyuanVideo-Avatar, LTX-Video / LTX-2, and FramePack.

For even smaller models, you can run the highest quality Q8_0 or FP8 quantizations. FLUX.1 dev fits in 14.4 GB using FP8 / optimized. Gemma 3 12B, Gemma 4 12B, Mistral NeMo 12B, Pixtral 12B, FLUX.1 schnell, and FLUX.1 Kontext dev all run at Q8_0 quantization using 15.3 GB of video memory.

If a model is too large for the 16 GB video memory, you can offload parts of it to your system RAM. This CPU offload requires a system with at least 32 GB of system RAM. For example, Solar Pro and Codestral 22B need 16.1 GB of memory at Q4_K_M, which requires 18.1 GB of system RAM. Mistral Small 3.2, Magistral Small, and Devstral Small 1.1 need 17.6 GB at Q4_K_M, requiring 19.6 GB of system RAM. Aria needs 18.3 GB at Q4_K_M, requiring 20.3 GB of system RAM.

Larger offload options include Gemma 4 26B-A4B and Gemma 4 (all sizes), which need 19 GB at Q4_K_M and require 21 GB of system RAM. Gemma 3 27B and Gemma 3 4B/12B/27B (vision) need 19.8 GB at Q4_K_M, requiring 21.8 GB of system RAM. Offloading to system RAM allows you to run these larger models, but it significantly reduces the generation speed because system RAM is much slower than GDDR6 video memory.

All memory calculations assume a standard 4k context window. If you increase the context window to process longer documents or conversations, the active memory usage will grow. Running close to the 16 GB limit of the RTX 2000 Ada Generation with a large model may cause out of memory errors if the context window expands beyond this baseline.