Best local AI models for AMD Pro 580X

The largest models that fit fully

The 30 largest of the 123 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The AMD Pro 580X features 8 GB of GDDR5 memory. This memory stores the model weights during local inference. You must choose models that fit within this 8 GB limit to maintain high performance. Exceeding this limit forces the system to use slower system RAM. This causes a significant drop in generation speed.

The quant column refers to quantization. Quantization reduces the precision of model weights to save space. A Q4_K_M quant uses less memory than a Q6_K quant. Choosing the right quant allows larger models to fit into your 8 GB of video memory. You should select the highest quality quant that fits your available space.

Models like Mochi 1 at 10B use 7.3 GB. Gemma 2 9B uses 8 GB. Nemotron Nano 9B and GLM 4 9B use 7.7 GB. Yi Coder 9B and GLM 4V 9B also use 7.7 GB. Chroma uses 7.6 GB. Llama 3.1 8B uses 7.4 GB. These models fit entirely on your GPU. You will get the fastest response times with these selections.

Many 8B models fit comfortably in your 8 GB of memory. Granite 3.3 8B, Ministral 8B, InternLM 3 8B, OpenCoder 8B, and Seed Coder 8B use 7.9 GB at Q6_K. MiniCPM V 2.6, Idefics 3 8B, Fuyu 8B, Emu3, and Stable Diffusion 3.5 Large use 7.9 GB at Q6_K. EXAONE 3.5 7.8B uses 7.7 GB at Q6_K. Mistral 7B and Qwen2.5 7B use 7.4 GB or 6.9 GB depending on the quant.

You can run larger models using CPU offload. This process moves parts of the model to your 32 GB of system RAM. Open Sora 2.0 needs 8.1 GB and 10.1 GB of system RAM. Gemma 3 12B, Gemma 4 12B, Mistral NeMo 12B, Pixtral 12B, and FLUX.1 models need 8.8 GB and 10.8 GB of system RAM. Vicuna 13B needs 9.5 GB and 11.5 GB of system RAM.

Offloading carries a performance cost. Your system RAM is slower than GDDR5 memory. Inference will be slower when offloading is active. You must also account for context window size. A 4k context window consumes additional memory. This extra usage can push a model over your 8 GB limit. Monitor your memory usage closely to ensure stable operation.