Best local AI models for AMD RX 590

The largest models that fit fully

The 30 largest of the 123 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The AMD Radeon RX 590 graphics card features 8 GB of GDDR5 video memory. This onboard memory determines the maximum size of the artificial intelligence models you can run directly on your hardware. For local execution, the model files must fit within this VRAM boundary to maintain acceptable processing speeds.

To fit larger models into the 8 GB limit, developers use quantization. The quant column indicates the compression level applied to the model weights. For example, Gemma 2 9B fits at the Q4_K_M quantization level using 8 GB of VRAM. Smaller architectures like Llama 3.1 8B can run at the higher quality Q5_K_M quantization level while consuming 7.4 GB of VRAM. Models like Mistral 7B and Qwen2.5 7B can run at the Q6_K quantization level, which uses 7.4 GB and 6.9 GB of VRAM respectively.

Other models that fit entirely within your local VRAM include Mochi 1 at Q4_K_M using 7.3 GB. You can also run Nemotron Nano 9B, GLM-4 9B, Yi-Coder 9B, GLM-4-9B-Chat, and GLM-4V-9B at Q5_K_M quantization using 7.7 GB of VRAM. Chroma fits at Q5_K_M using 7.6 GB. Granite 3.3 8B, Ministral 8B, InternLM 3 8B, OpenCoder 8B, Seed-Coder 8B, MiniCPM-V 2.6, Idefics 3 8B, Fuyu-8B, Emu3, and Stable Diffusion 3.5 Large fit at Q6_K quantization using 7.9 GB of VRAM. EXAONE 3.5 7.8B uses 7.7 GB at Q6_K. OLMo 2 7B, Falcon 3 7B, Command R7B, OpenHermes 2.5, Zephyr 7B Beta, OpenChat 3.5, Starling LM 7B, and Codestral Mamba 7B all use 6.9 GB at Q6_K.

When a model exceeds the 8 GB VRAM limit, you must offload parts of the workload to your system RAM. This process requires a system with at least 32 GB of system RAM. Offloading allows you to run larger models, but it reduces processing speed because system RAM is slower than GDDR5 video memory.

Examples of offloaded models include Gemma 3 12B, Gemma 4 12B, Mistral NeMo 12B, Pixtral 12B, FLUX.1 schnell, FLUX.1 Kontext dev, and FLUX.1 Krea dev. These models require 8.8 GB at the Q4_K_M quantization level, which uses 10.8 GB of system RAM. Open-Sora 2.0 requires 8.1 GB at Q4_K_M and uses 10.1 GB of system RAM. FLUX.1 dev requires 14.4 GB at FP8 and uses 16.4 GB of system RAM. Vicuna 13B requires 9.5 GB at Q4_K_M and uses 11.5 GB of system RAM.

VRAM consumption calculations are based on a standard context window of 4k tokens. If you increase the context window to process longer documents or chat histories, the memory requirement will increase. This extra memory usage may force you to use a lower quantization level or offload more data to your system RAM.