Best local AI models for AMD Pro 580

The largest models that fit fully

The 30 largest of the 123 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The AMD Radeon Pro 580 is equipped with 8 GB of GDDR5 graphics memory. This dedicated memory determines the maximum size of the artificial intelligence models you can run entirely on your graphics hardware. When a model fits completely within this 8 GB limit, it executes with the fastest possible processing speeds.

To fit larger models into your hardware, you must use quantized versions. The quantization column indicates the optimal compression level for each model. For example, the 10B Mochi 1 model fits in 7.3 GB of memory when using the Q4_K_M quantization. Models like Gemma 2 9B require exactly 8 GB of memory at the Q4_K_M quantization, which fully utilizes your graphics hardware capacity.

Many capable models fit within the 8 GB limit using higher quality quantizations. The Granite 3.3 8B, Ministral 8B, InternLM 3 8B, OpenCoder 8B, and Seed_Coder 8B models all run at the Q6_K quantization while using 7.9 GB of memory. Vision models such as MiniCPM_V 2.6, Idefics 3 8B, Fuyu_8B, and Emu3 also fit within 7.9 GB at the Q6_K quantization. Image generators like Stable Diffusion 3.5 Large and Stable Diffusion 3.5 Turbo utilize 7.9 GB at Q6_K as well.

If you want to run models that exceed 8 GB, you must use CPU offloading. This process splits the model between your graphics memory and your system memory. For this setup, we assume you have 32 GB of system RAM. Running FLUX.1 dev at FP8 requires 14.4 GB of memory, which uses all your graphics memory and 16.4 GB of system RAM. Gemma 3 12B, Gemma 4 12B, Mistral NeMo 12B, and Pixtral 12B require 8.8 GB of memory, which uses 10.8 GB of system RAM.

CPU offloading allows you to run larger architectures, but it comes with a performance cost. Transferring data between your system RAM and the graphics card slows down the generation speed significantly. For example, Vicuna 13B requires 9.5 GB of memory at Q4_K_M, which forces 11.5 GB of data into your system RAM and reduces processing speed.

You must also consider the memory required for context. The memory figures listed here are calculated using a standard 4k context window. If you increase the context length to process longer documents or conversations, the system will require more memory. This extra memory usage might force you to use a smaller model or a lower quantization level to prevent system slowdowns.