Best local AI models for AMD RX 5700 XT 50th Anniversary

The largest models that fit fully

The 30 largest of the 123 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The AMD RX 5700 XT 50th Anniversary edition features 8 GB of GDDR6 onboard memory. This dedicated video memory determines the maximum size of the artificial intelligence models you can run entirely on the graphics card. When a model fits completely within this limit, it runs at the maximum processing speed of your hardware.

The quantization column indicates the compression level applied to each model. Quantization reduces the precision of the model weights to save space. For example, the Q4_K_M quant offers a good balance of size and quality, while the Q5_K_M and Q6_K quants provide higher precision but require more memory. A model like Gemma 2 9B at Q4_K_M uses exactly 8 GB of video memory, which fully utilizes your hardware capacity.

Several high quality models fit comfortably within your 8 GB limit. Mochi 1 10B at Q4_K_M uses 7.3 GB of memory. Nemotron Nano 9B, GLM-4 9B, Yi-Coder 9B, GLM-4-9B-Chat, and GLM-4V-9B all fit at Q5_K_M quantization using 7.7 GB of memory. Chroma 8.9B fits at Q5_K_M using 7.6 GB. Llama 3.1 8B fits at Q5_K_M using 7.4 GB. Granite 3.3 8B, Ministral 8B, InternLM 3 8B, OpenCoder 8B, Seed-Coder 8B, MiniCPM-V 2.6, Idefics 3 8B, Fuyu-8B, Emu3, and Stable Diffusion 3.5 Large all fit at Q6_K quantization using 7.9 GB of memory.

Slightly smaller models leave more breathing room on your card. EXAONE 3.5 7.8B at Q6_K uses 7.7 GB of memory. Mistral 7B at Q6_K uses 7.4 GB. Qwen2.5 7B, OLMo 2 7B, Falcon 3 7B, Command R7B, OpenHermes 2.5, Zephyr 7B Beta, OpenChat 3.5, Starling LM 7B, and Codestral Mamba 7B all run at Q6_K quantization using 6.9 GB of memory.

When a model exceeds your 8 GB video memory, you must offload some layers to your system RAM. This process requires a system with at least 32 GB of system RAM. Offloading allows you to run larger models, but it introduces a significant speed penalty because data must travel over the system bus. For instance, FLUX.1 dev at FP8 requires 14.4 GB of memory and needs 16.4 GB of system RAM. Open-Sora 2.0 at Q4_K_M requires 8.1 GB of memory and needs 10.1 GB of system RAM.

Other offload options include Gemma 3 12B, Gemma 4 12B, Mistral NeMo 12B, Pixtral 12B, FLUX.1 schnell, FLUX.1 Kontext dev, and FLUX.1 Krea dev, which all require 8.8 GB of memory at Q4_K_M and need 10.8 GB of system RAM. Vicuna 13B at Q4_K_M requires 9.5 GB of memory and needs 11.5 GB of system RAM. Note that memory usage calculations assume a standard 4k context window, and increasing this context window will require additional memory.