Best local AI models for AMD RX 5600M
6 GB GDDR6. At a 4k context, 114 of the 233 models in our catalog with verified parameter counts fit fully, up to Granite 3.3 2B / 8B at 8B parameters.
Check your own machine against every model →The largest models that fit fully
The 30 largest of the 114 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.
| Model | Parameters | Best quant that fits | Memory used at 4k |
|---|---|---|---|
| Granite 3.3 2B / 8B | 8B | Q4_K_M | 5.9 GB |
| Ministral 3B / 8B | 8B | Q4_K_M | 5.9 GB |
| InternLM 3 8B | 8B | Q4_K_M | 5.9 GB |
| OpenCoder 1.5B / 8B | 8B | Q4_K_M | 5.9 GB |
| Seed-Coder 8B | 8B | Q4_K_M | 5.9 GB |
| MiniCPM-V 2.6 / MiniCPM-o 2.6 | 8B | Q4_K_M | 5.9 GB |
| Idefics 3 8B | 8B | Q4_K_M | 5.9 GB |
| Fuyu-8B | 8B | Q4_K_M | 5.9 GB |
| Emu3 | 8B | Q4_K_M | 5.9 GB |
| Stable Diffusion 3.5 Large / Turbo | 8B | Q4_K_M | 5.9 GB |
| EXAONE 3.5 2.4B / 7.8B | 7.8B | Q4_K_M | 5.7 GB |
| Mistral 7B | 7B | Q4_K_M | 5.7 GB |
| Qwen2.5 0.5B / 1.5B / 3B / 7B | 7B | Q5_K_M | 6 GB |
| OLMo 2 1B / 7B | 7B | Q5_K_M | 6 GB |
| Falcon 3 1B / 3B / 7B | 7B | Q5_K_M | 6 GB |
| Command R7B | 7B | Q5_K_M | 6 GB |
| OpenHermes 2.5 | 7B | Q5_K_M | 6 GB |
| Zephyr 7B Beta | 7B | Q5_K_M | 6 GB |
| OpenChat 3.5 | 7B | Q5_K_M | 6 GB |
| Starling LM 7B | 7B | Q5_K_M | 6 GB |
| Codestral Mamba 7B | 7B | Q5_K_M | 6 GB |
| CodeGemma 2B / 7B | 7B | Q5_K_M | 6 GB |
| aiXcoder-7B | 7B | Q5_K_M | 6 GB |
| Nxcode / CodeQwen 1.5 7B | 7B | Q5_K_M | 6 GB |
| Janus-Pro 1B / 7B | 7B | Q5_K_M | 6 GB |
| Ruyi-Mini-7B | 7B | Q5_K_M | 6 GB |
| Qwen2-Audio 7B | 7B | Q5_K_M | 6 GB |
| Qwen2.5-Omni 3B / 7B | 7B | Q5_K_M | 6 GB |
| YuE | 7B | Q5_K_M | 6 GB |
| Magicoder-S-DS 6.7B | 6.7B | Q5_K_M | 5.7 GB |
Close, but only with CPU offload
These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.
| Model | Parameters | Memory at Q4_K_M | System RAM at 4k |
|---|---|---|---|
| Llama 3.1 8B | 8B | 6.4 GB needed | 8.4 GB |
| Chroma | 8.9B | 6.5 GB needed | 8.5 GB |
| Gemma 2 9B | 9B | 8 GB needed | 10 GB |
| Nemotron Nano 4B / 9B | 9B | 6.6 GB needed | 8.6 GB |
| GLM-4 9B / GLM-4.5-Air | 9B | 6.6 GB needed | 8.6 GB |
| Yi-Coder 1.5B / 9B | 9B | 6.6 GB needed | 8.6 GB |
| GLM-4-9B-Chat / CodeGeeX4 | 9B | 6.6 GB needed | 8.6 GB |
| GLM-4V-9B / GLM-4.1V-Thinking | 9B | 6.6 GB needed | 8.6 GB |
| Mochi 1 | 10B | 7.3 GB needed | 9.3 GB |
| Open-Sora 2.0 | 11B | 8.1 GB needed | 10.1 GB |
How to read this
The AMD Radeon RX 5600M is a mobile graphics processor equipped with 6 GB of GDDR6 VRAM. This memory capacity determines the size of the artificial intelligence models you can run locally. To load a model entirely onto your graphics hardware, the model files and active memory must fit within this 6 GB limit. Running models directly on your VRAM ensures the fastest possible processing speeds.
The quantization column indicates the compression level applied to each model. Quantization reduces the precision of model weights to make the files smaller. For this hardware, the Q4_K_M and Q5_K_M quants represent the best balance of size and output quality. For example, Granite 3.3 8B, Ministral 8B, and InternLM 3 8B fit within 5.9 GB of VRAM using the Q4_K_M quant. Other models like Qwen2.5 7B, OLMo 2 7B, and Falcon 3 7B utilize the Q5_K_M quant to fit exactly 6 GB of VRAM.
When a model exceeds the 6 GB VRAM limit, you must use CPU offloading. This technique splits the model layers between your graphics card and your system RAM. We assume your computer has 32 GB of system RAM for these scenarios. For instance, running Llama 3.1 8B at Q4_K_M requires 6.4 GB of memory, which uses your VRAM and 8.4 GB of system RAM. Similarly, Gemma 2 9B at Q4_K_M requires 8 GB of memory and uses 10 GB of system RAM.
CPU offloading allows you to run larger architectures such as the 10B Mochi 1 or the 11B Open-Sora 2.0. Mochi 1 at Q4_K_M requires 7.3 GB of memory and 9.3 GB of system RAM. Open-Sora 2.0 at Q4_K_M requires 8.1 GB of memory and 10.1 GB of system RAM. While offloading enables these larger models to run, it comes with a performance cost. Moving data between system RAM and VRAM is much slower than keeping everything on the graphics card.
You must also consider the 4k context window caveat when planning your memory usage. The memory figures listed for these models are calculated using a standard 4096 token context window. If you increase the context length to process longer documents or conversations, the model will require more memory. This extra memory demand can push a model past the 6 GB VRAM limit and force your system to offload layers to the slower system RAM.