Best local AI models for AMD Pro 5700 XT

The largest models that fit fully

The 30 largest of the 155 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The AMD Radeon Pro WX 5700 XT graphics card features 16 GB of GDDR6 memory. This dedicated video memory determines the size of the artificial intelligence models you can run locally. For optimal performance, the entire model must fit within this 16 GB limit. If a model exceeds this capacity, your system must use slower system memory, which decreases processing speeds.

Quantization is a method that compresses model weights to save memory. The quant column shows the best quantization level that fits your hardware. For example, you can run the 21B gpt-oss-20b or Reka Flash 3 at a Q4_K_M quantization using 15.4 GB of video memory. Smaller models like the 15B StarCoder2 or Apriel-1.5-15B-Thinker can run at a higher quality Q6_K quantization using 14.8 GB of memory.

For vision and image tasks, the 20B Qwen-Image and Qwen-Image-Edit models fit comfortably at Q4_K_M quantization using 14.6 GB of memory. The 19B CogVLM2 model uses 13.9 GB of memory at Q4_K_M. If you want to run the FLUX.1 dev model, you can use the FP8 optimized quantization which requires 14.4 GB of video memory.

You can also run highly accurate 12B models at the excellent Q8_0 quantization level. Gemma 3 12B, Gemma 4 12B, Mistral NeMo 12B, and Pixtral 12B all require 15.3 GB of memory at Q8_0. The FLUX.1 schnell and FLUX.1 Kontext dev models also run at Q8_0 quantization using 15.3 GB of video memory.

When a model is too large for the 16 GB video memory, you can offload parts of it to your system RAM. If your computer has 32 GB of system RAM, you can run larger models with a performance penalty. For example, the 22B Solar Pro and Codestral 22B require 16.1 GB at Q4_K_M quantization and need 18.1 GB of system RAM. The 27B Gemma 3 model requires 19.8 GB at Q4_K_M quantization and needs 21.8 GB of system RAM.

All memory calculations assume a standard 4k context window. If you increase the context window to process longer documents, the model will require significantly more memory. You must choose a smaller model or a lower quantization level if you plan to use long context windows.