Best local AI models for AMD Pro 560
4 GB GDDR5. At a 4k context, 81 of the 233 models in our catalog with verified parameter counts fit fully, up to Lumina-Next / Lumina-Image 2.0 at 5B parameters.
Check your own machine against every model →The largest models that fit fully
The 30 largest of the 81 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.
| Model | Parameters | Best quant that fits | Memory used at 4k |
|---|---|---|---|
| Lumina-Next / Lumina-Image 2.0 | 5B | Q4_K_M | 3.7 GB |
| CogVideoX 2B / 5B | 5B | Q4_K_M | 3.7 GB |
| DeepSeek-VL2 | 4.5B | Q5_K_M | 3.8 GB |
| DeepFloyd IF | 4.3B | Q5_K_M | 3.7 GB |
| Phi-3.5-vision | 4.2B | Q5_K_M | 3.6 GB |
| Qwen3 4B | 4B | Q6_K | 3.9 GB |
| Gemma 3 4B | 4B | Q6_K | 3.9 GB |
| Gemma 4 E4B | 4B | Q6_K | 3.9 GB |
| MiniCPM 3 4B | 4B | Q6_K | 3.9 GB |
| Danube 3 4B | 4B | Q6_K | 3.9 GB |
| Fish Speech 1.5 / OpenAudio S1 | 4B | Q6_K | 3.9 GB |
| Phi-4-mini-instruct | 3.8B | Q6_K | 3.7 GB |
| Phi-3.5 Mini | 3.8B | Q6_K | 3.7 GB |
| OmniGen / OmniGen2 | 3.8B | Q6_K | 3.7 GB |
| SD Cascade (Würstchen v3) | 3.6B | Q6_K | 3.5 GB |
| SDXL Turbo | 3.5B | Q6_K | 3.4 GB |
| SDXL Lightning | 3.5B | Q6_K | 3.4 GB |
| ACE-Step | 3.5B | Q6_K | 3.4 GB |
| MusicGen small/medium/large | 3.3B | Q6_K | 3.2 GB |
| SmolLM3 3B | 3B | Q8_0 | 3.8 GB |
| Replit Code v1.5 3B | 3B | Q8_0 | 3.8 GB |
| Kandinsky 3.1 | 3B | Q8_0 | 3.8 GB |
| Voxtral Mini / Small | 3B | Q8_0 | 3.8 GB |
| Orpheus TTS | 3B | Q8_0 | 3.8 GB |
| Higgs Audio v2 | 3B | Q8_0 | 3.8 GB |
| Allegro | 2.8B | Q8_0 | 3.6 GB |
| Open-Sora Plan | 2.7B | Q8_0 | 3.4 GB |
| LFM2 1.2B / 2.6B | 2.6B | Q8_0 | 3.3 GB |
| Playground v2.5 | 2.6B | Q8_0 | 3.3 GB |
| Stable Diffusion 3.5 Medium | 2.5B | Q8_0 | 3.2 GB |
Close, but only with CPU offload
These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.
| Model | Parameters | Memory at FP8 / optimized | System RAM at 4k |
|---|---|---|---|
| Stable Diffusion XL | 3.417B | 4.1 GB needed | 6.1 GB |
| Phi-3 Mini | 3.8B | 4.4 GB needed | 6.4 GB |
| Phi-4-multimodal | 5.6B | 4.1 GB needed | 6.1 GB |
| Magicoder-S-DS 6.7B | 6.7B | 4.9 GB needed | 6.9 GB |
| Mistral 7B | 7B | 5.7 GB needed | 7.7 GB |
| Qwen2.5 0.5B / 1.5B / 3B / 7B | 7B | 5.1 GB needed | 7.1 GB |
| OLMo 2 1B / 7B | 7B | 5.1 GB needed | 7.1 GB |
| Falcon 3 1B / 3B / 7B | 7B | 5.1 GB needed | 7.1 GB |
| Command R7B | 7B | 5.1 GB needed | 7.1 GB |
| OpenHermes 2.5 | 7B | 5.1 GB needed | 7.1 GB |
How to read this
The AMD Radeon Pro 560 is a dedicated graphics card equipped with 4 GB of GDDR5 memory. When running artificial intelligence models locally, this onboard memory capacity is the main hardware limit. The entire model weights and the active processing data must fit within this space to run at hardware speed. If a model exceeds this limit, execution slows down or fails.
To fit larger models into the 4 GB memory space, developers use quantization. This process compresses the numerical weights of a model. The quantization column shows the best format that fits your hardware. For example, a Q4_K_M quant uses four bit compression, while Q6_K and Q8_0 quants offer higher precision but require more memory. Choosing the correct quant balance ensures the model fits within the 3.7 GB to 3.9 GB limits of your card.
For models that fit entirely on your hardware, several options are available. The Lumina-Next and Lumina-Image 2.0 models at 5B parameters fit using a Q4_K_M quant which uses 3.7 GB. The CogVideoX 2B / 5B model also fits at 5B parameters using Q4_K_M for 3.7 GB. DeepSeek-VL2 at 4.5B parameters fits using Q5_K_M and uses 3.8 GB. You can also run DeepFloyd IF at 4.3B parameters using Q5_K_M for 3.7 GB, or Phi-3.5-vision at 4.2B parameters using Q5_K_M for 3.6 GB.
Several 4B parameter models run using a Q6_K quant which uses 3.9 GB of memory. These include Qwen3 4B, Gemma 3 4B, Gemma 4 E4B, MiniCPM 3 4B, Danube 3 4B, and Fish Speech 1.5 / OpenAudio S1. For slightly smaller models, Phi-4-mini-instruct and Phi-3.5 Mini at 3.8B parameters use Q6_K for 3.7 GB. OmniGen / OmniGen2 also runs at 3.8B parameters using Q6_K for 3.7 GB. Image generators like SD Cascade (Würstchen v3) at 3.6B parameters use Q6_K for 3.5 GB, while SDXL Turbo and SDXL Lightning at 3.5B parameters use Q6_K for 3.4 GB.
When a model is too large for the 4 GB onboard memory, you can use CPU offloading if your system has 32 GB of system RAM. This method splits the workload between your graphics card and your system memory. For instance, Stable Diffusion XL at 3.417B parameters needs 4.1 GB at FP8 or optimized settings, which requires 6.1 GB of system RAM. Phi-3 Mini at 3.8B parameters needs 4.4 GB at Q4_K_M, requiring 6.4 GB of system RAM. Phi-4-multimodal at 5.6B parameters needs 4.1 GB at Q4_K_M, requiring 6.1 GB of system RAM.
Larger 7B models also run via offloading. Mistral 7B needs 5.7 GB at Q4_K_M and requires 7.7 GB of system RAM. Qwen2.5 0.5B / 1.5B / 3B / 7B, OLMo 2 1B / 7B, Falcon 3 1B / 3B / 7B, Command R7B, and OpenHermes 2.5 all run at 7B parameters using Q4_K_M, requiring 5.1 GB on the card and 7.1 GB of system RAM. Note that using a standard 4k context window increases memory usage during active generation, which can push close limits into out of memory errors.