Best local AI models for NVIDIA RTX 4080

The largest models that fit fully

The 30 largest of the 155 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The NVIDIA RTX 4080 graphics card features 16 GB of GDDR6X memory. This video memory determines the maximum size of the artificial intelligence models you can run entirely on your local hardware. When a model fits completely within this 16 GB limit, it runs at maximum speed because the processor has direct, high speed access to all model weights.

To fit larger models into your video memory, developers use quantization. The quantization column shows the optimal compression level for each model. For example, the 21B gpt-oss-20b and Reka Flash 3 models fit within 15.4 GB of video memory using the Q4_K_M quantization. Smaller models like the 12B Gemma 3 12B, Gemma 4 12B, and Mistral NeMo 12B can run at a higher Q8_0 quantization while using 15.3 GB of video memory.

If a model requires more than 16 GB of memory, you must offload some layers to your system RAM. This process requires at least 32 GB of system RAM to function. For instance, running the 22B Solar Pro or Codestral 22B at Q4_K_M requires 16.1 GB of memory, which uses 18.1 GB of system RAM. The 27B Gemma 3 27B requires 19.8 GB of memory and uses 21.8 GB of system RAM. Offloading allows you to run these larger models, but it significantly reduces processing speed because system RAM is much slower than GDDR6X video memory.

You must also consider the memory cost of your context window. The memory figures listed for these models assume a standard 4k context window. If you increase the context window to process longer documents or chat histories, the model will require more video memory. This extra memory usage might force you to use a lower quantization or offload layers to system RAM.

The 16 GB memory limit also supports various vision and media generation models. You can run the 20B Qwen-Image at Q4_K_M using 14.6 GB of video memory. Image and video generation models like FLUX.1 dev run at FP8 or optimized settings using 14.4 GB of video memory, while HunyuanVideo runs at Q6_K using 12.8 GB of video memory.