Best local AI models for AMD RX 580X

The largest models that fit fully

The 30 largest of the 123 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The AMD RX 580X graphics card features 8 GB of GDDR5 video memory. This memory size determines which artificial intelligence models can run directly on your hardware. To run a model entirely on the graphics processor, the model files and working memory must fit within this 8 GB limit. If a model exceeds this capacity, it will require system memory offloading which slows down performance.

Quantization is a method that compresses model files to save space. The quant column shows the best compression level that fits your hardware. For example, the Mochi 1 10B model fits using the Q4_K_M quant which uses 7.3 GB of memory. Gemma 2 9B fits at the Q4_K_M quant using exactly 8 GB of memory. Models like Nemotron Nano 9B, GLM-4 9B, Yi-Coder 9B, GLM-4-9B-Chat, and GLM-4V-9B can run at the Q5_K_M quant using 7.7 GB of memory.

Smaller models can use higher quality quantizations because they require less space. Chroma 8.9B fits at Q5_K_M using 7.6 GB. Llama 3.1 8B fits at Q5_K_M using 7.4 GB. Granite 3.3 8B, Ministral 8B, InternLM 3 8B, OpenCoder 8B, Seed-Coder 8B, MiniCPM-V 2.6, Idefics 3 8B, Fuyu-8B, Emu3, and Stable Diffusion 3.5 Large can all run at the Q6_K quant using 7.9 GB. EXAONE 3.5 7.8B also runs at Q6_K using 7.7 GB.

Popular 7B models run comfortably on this hardware. Mistral 7B uses 7.4 GB at the Q6_K quant. Qwen2.5 7B, OLMo 2 7B, Falcon 3 7B, Command R7B, OpenHermes 2.5, Zephyr 7B Beta, OpenChat 3.5, Starling LM 7B, and Codestral Mamba 7B all use 6.9 GB at the Q6_K quant. Running these models at Q6_K provides excellent response quality while staying safely under the 8 GB video memory limit.

When a model is too large for the graphics card, you can offload parts of it to your system RAM. This process requires a system with 32 GB of system RAM. Open-Sora 2.0 needs 8.1 GB at Q4_K_M and uses 10.1 GB of system RAM. FLUX.1 dev needs 14.4 GB at FP8 and uses 16.4 GB of system RAM. Gemma 3 12B, Gemma 4 12B, Mistral NeMo 12B, Pixtral 12B, FLUX.1 schnell, FLUX.1 Kontext dev, and FLUX.1 Krea dev all need 8.8 GB at Q4_K_M and use 10.8 GB of system RAM. Vicuna 13B needs 9.5 GB at Q4_K_M and uses 11.5 GB of system RAM.

Offloading allows you to run these larger models but it comes with a performance cost. Moving data between the graphics card and system RAM is much slower than keeping everything in the GDDR5 memory. You must also consider the context window. Keeping a 4k context window active requires extra memory space. If you generate long answers or upload large prompts, the model might exceed the 8 GB limit and trigger slow system RAM offloading.