Best local AI models for AMD RX 9070 XT

The largest models that fit fully

The 30 largest of the 155 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The AMD RX 9070 XT graphics card features 16 GB of GDDR6 memory. This memory size determines which local AI models you can run entirely on your hardware. For the best performance, the entire model must fit inside this onboard memory. If a model exceeds this limit, your system must use slower system memory to process the remaining data.

The quantization column shows the compression level used to fit these models into memory. Quantization reduces the precision of model weights to save space. For example, the 21B gpt-oss-20b and Reka Flash 3 models fit within 15.4 GB of memory using the Q4_K_M quantization. Larger models like Qwen-Image and Qwen-Image-Edit use 14.6 GB of memory at the same Q4_K_M level. This compression allows you to run larger parameter sizes on consumer hardware.

As model sizes decrease, you can use higher precision levels for better output quality. The 17B HunyuanImage 2.1 / 3.0 uses 14.5 GB at Q5_K_M quantization. The 16B DeepSeek-Coder-V2 16B / 236B and Kimi-VL A3B models fit within 15.7 GB using a higher Q6_K quantization. You can also run the 14B Phi-4, Phi-4-reasoning / -plus, Wan 2.2 T2I, and SkyReels V2 models at Q6_K quantization using 13.8 GB of memory.

For maximum precision, you can run 12B models at Q8_0 quantization. The Gemma 3 12B, Gemma 4 12B, Mistral NeMo 12B, and Pixtral 12B models use 15.3 GB of memory at this level. The FLUX.1 dev model can run at FP8 / optimized quantization using 14.4 GB of memory. Running these models at higher quantization levels preserves original model accuracy.

When a model is too large for the 16 GB GDDR6 memory, you can offload parts of it to your system RAM. This requires at least 32 GB of system RAM. For example, Solar Pro and Codestral 22B need 16.1 GB at Q4_K_M quantization and require 18.1 GB of system RAM. The Mistral Small 3.2, Magistral Small, and Devstral Small 1.1 models need 17.6 GB at Q4_K_M and require 19.6 GB of system RAM. Offloading allows you to run the Gemma 3 27B model which needs 19.8 GB at Q4_K_M and requires 21.8 GB of system RAM. This process makes larger models run but reduces generation speed.

All listed memory figures assume a standard 4k context window. Generating longer responses or processing larger inputs increases memory usage. If you increase the context window beyond 4k tokens, the model will require more memory than the listed figures. You may need to use a lower quantization level to avoid running out of memory during long conversations.