Best local AI models for AMD RX 580

The largest models that fit fully

The 30 largest of the 123 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The AMD RX 580 graphics card features 8 GB GDDR5 memory. This onboard memory determines the maximum size of the AI models you can run locally. To fit a model entirely on the card, the model size and its active memory footprint must remain under this 8 GB threshold. Running models fully within the graphics memory ensures the fastest possible processing speeds.

The quantization column shows the best compression format for each model. Quantization reduces the precision of model weights to save space. For example, a Q4_K_M quant uses fewer bits per weight than a Q6_K quant. This compression allows larger models to fit into the 8 GB memory limit of your card. A higher quant like Q6_K retains more original model quality but requires more memory space.

Several high quality models fit directly into the 8 GB limit of the AMD RX 580. The Mochi 1 10B model fits at a Q4_K_M quant using 7.3 GB of memory. Gemma 2 9B fits at Q4_K_M using exactly 8 GB. You can also run Llama 3.1 8B at a Q5_K_M quant using 7.4 GB. Other options include Mistral 7B and Qwen2.5 7B which both run at a Q6_K quant using 7.4 GB and 6.9 GB respectively.

When a model exceeds the 8 GB graphics memory, you must use CPU offload. This process splits the model weights between your graphics card and your system RAM. CPU offload requires a system with sufficient RAM, such as 32 GB. Offloading allows you to run larger models, but it introduces a speed penalty because data must travel between the system RAM and the graphics card.

Several larger models are viable with CPU offload. FLUX.1 dev 12B requires 14.4 GB at FP8 or optimized settings and uses 16.4 GB of system RAM. Gemma 3 12B, Gemma 4 12B, and Mistral NeMo 12B each need 8.8 GB at a Q4_K_M quant and require 10.8 GB of system RAM. Vicuna 13B requires 9.5 GB at a Q4_K_M quant and uses 11.5 GB of system RAM.

Memory calculations assume a standard 4k context window. If you increase the context window to process longer documents, the memory usage will rise. This extra memory consumption can push a model past the 8 GB limit of your card. For stable performance, keep your context length within the standard limits or choose a smaller model.