Best local AI models for AMD RX 9070 GRE

The largest models that fit fully

The 30 largest of the 147 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The AMD Radeon RX 9070 GRE is equipped with 12 GB of GDDR6 memory. This memory capacity determines the maximum size of the artificial intelligence models you can run entirely on your graphics card. When a model fits completely within this video memory, it processes tokens at maximum speed. If a model exceeds this limit, you must offload parts of it to your system memory, which reduces performance.

The quantization column indicates the compression level used to fit these models into memory. Quantization reduces the precision of model weights to save space. For example, a Q4_K_M quantization represents a medium four bit precision format, while Q5_K_M, Q6_K, and Q8_0 represent progressively higher precision levels. Higher precision levels require more memory but preserve more of the original output quality of the model.

With 12 GB of video memory, you can run several large models locally without offloading. The DeepSeek-Coder-V2 16B and Kimi-VL A3B models fit at Q4_K_M quantization, using 11.7 GB of memory. You can also run the Apriel-1.5-15B-Thinker and StarCoder2 15B models at Q4_K_M quantization using 11 GB. The Qwen2.5 14B model fits at Q4_K_M using 11.6 GB of memory.

For higher precision, several 14B and 13B models run at Q5_K_M quantization. The Phi-4, Phi-4-reasoning / -plus, Wan 2.2 T2I, Wan 2.1 (1.3B / 14B), and SkyReels V2 models use 11.9 GB of memory. The Vicuna 13B, HunyuanVideo, HunyuanVideo-Avatar, LTX-Video / LTX-2, and FramePack models use 11.1 GB. You can also run 12B models like Gemma 3 12B, Gemma 4 12B, Mistral NeMo 12B, Pixtral 12B, FLUX.1 schnell, FLUX.1 Kontext dev, and FLUX.1 Krea dev at Q6_K quantization using 11.8 GB.

Smaller models can run at even higher precision levels. The Open-Sora 2.0 model uses 10.8 GB at Q6_K quantization, and Mochi 1 uses 9.8 GB at Q6_K. The Gemma 2 9B model uses 10.3 GB at Q6_K. You can run the Nemotron Nano 9B, GLM-4 9B / GLM-4.5-Air, Yi-Coder 9B, and GLM-4-9B-Chat / CodeGeeX4 models at Q8_0 quantization, which uses 11.4 GB of memory.

If you have 32 GB of system RAM, you can run larger models by offloading some data to your CPU. This offloading allows you to run FLUX.1 dev, which needs 14.4 GB at FP8 and uses 16.4 GB of system RAM. You can run Ling-Coder-Lite at Q4_K_M using 12.3 GB of video memory and 14.3 GB of system RAM. Other offload options include Qwen-Image, Reka Flash 3, Solar Pro, and Codestral 22B, which require up to 18.1 GB of system RAM.

When planning your local deployments, remember the 4k context caveat. The memory figures listed here represent the model weights. Running longer conversations or processing larger documents requires additional video memory to store the context window. If your context window grows large, you may need to use a lower quantization level to prevent the model from spilling over into your system memory.