Best local AI models for NVIDIA RTX 3090 Ti

The largest models that fit fully

The 30 largest of the 173 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The NVIDIA RTX 3090 Ti features 24 GB of GDDR6X onboard memory. This dedicated video memory determines the size of the artificial intelligence models you can run entirely on your graphics card. When a model fits completely within this limit, you get the fastest possible processing speeds. If a model exceeds this capacity, you must use CPU offloading to system memory, which slows down performance.

The quantization column shows the compression level applied to each model. Quantization reduces the precision of model weights to save memory. For example, the Q4_K_M quant represents a four bit compression level. This allows larger models like Qwen3 32B, Qwen3.5 32B, Aya Expanse 32B, Granite 4.0 32B, and Qwen2.5-Coder 32B to fit into 23.4 GB of video memory. Similarly, Qwen3-30B-A3B and Qwen3-Coder 30B-A3B fit into 22 GB using Q4_K_M.

Higher precision quants require more memory but preserve more original model quality. The Q5_K_M quant is used for Gemma 3 27B, Gemma 3 27B vision, and Wan 2.2 / 2.5, which all consume 23 GB. Gemma 4 26B-A4B and Gemma 4 all sizes use Q5_K_M to fit in 22.2 GB, while Aria uses it to fit in 21.3 GB. The Q6_K quant allows Mistral Small 3.2, Magistral Small, and Devstral Small 1.1 to run at 23.6 GB. Solar Pro and Codestral 22B use Q6_K at 21.6 GB, while gpt-oss-20b and Reka Flash 3 use it at 20.7 GB. Qwen-Image and Qwen-Image-Edit fit in 19.7 GB, and CogVLM2 fits in 18.7 GB using Q6_K.

The highest precision Q8_0 quant is ideal for smaller models. HunyuanImage 2.1 / 3.0 uses Q8_0 at 21.6 GB. Ling-Coder-Lite fits in 21.4 GB, while DeepSeek-Coder-V2 16B and Kimi-VL A3B use Q8_0 at 20.4 GB. Apriel-1.5-15B-Thinker and StarCoder2 15B require 19.1 GB, while Qwen2.5 14.7B uses 19.5 GB under Q8_0. These configurations maximize output quality within your hardware limits.

When a model is too large for the 24 GB limit, you can offload layers to your system RAM. Assuming you have 32 GB of system RAM, you can run OTel 2.0 LLM 31B IT at Q4_K_M, which needs 27.5 GB of memory and 29.5 GB of system RAM. DeepSeek-Coder 33B and WizardCoder 33B require 24.2 GB of memory and 26.2 GB of system RAM. Yi 1.5 34B, Granite Code 34B, LLaVA 1.5 / 1.6 34B, and Ovis 2 need 24.9 GB of memory and 26.9 GB of system RAM. Qwen3.6-35B-A3B and Command R 35B require 25.6 GB of memory and 27.6 GB of system RAM, while Seed-OSS 36B needs 26.4 GB of memory and 28.4 GB of system RAM.

All memory calculations in this guide assume a standard 4k context window. The context window is the amount of text the model can process at one time. If you increase the context window beyond 4k tokens, the memory usage will rise. This extra memory demand can push a model past the 24 GB limit of your graphics card and force slow CPU offloading.