Best local AI models for AMD RX 5700

The largest models that fit fully

The 30 largest of the 123 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The AMD RX 5700 graphics card features 8 GB of GDDR6 onboard memory. This memory capacity determines the size of the artificial intelligence models you can run locally. For optimal performance, the entire model should fit within this video memory. If a model exceeds this limit, your system must split the workload between the graphics card and system memory, which slows down processing speeds.

The quantization column indicates the compression level applied to each model. Quantization reduces the precision of model weights to save space. A Q4_K_M quant represents a four bit medium quantization, while Q5_K_M and Q6_K represent five bit and six bit configurations. Higher quantization levels like Q6_K preserve more original model accuracy but require more video memory. Lower levels like Q4_K_M allow larger models to fit within your hardware limits.

With 8 GB of video memory, you can run several capable models entirely on your graphics card. Gemma 2 9B fits at the Q4_K_M quantization level using exactly 8 GB of memory. Mochi 1 10B fits at Q4_K_M using 7.3 GB. Several 9B models like Nemotron Nano 4B / 9B, GLM-4 9B / GLM-4.5-Air, Yi-Coder 1.5B / 9B, GLM-4-9B-Chat / CodeGeeX4, and GLM-4V-9B / GLM-4.1V-Thinking fit at Q5_K_M using 7.7 GB. Chroma 8.9B fits at Q5_K_M using 7.6 GB. Llama 3.1 8B fits at Q5_K_M using 7.4 GB.

Many 8B models can run at the higher Q6_K quantization level using 7.9 GB of memory. These include Granite 3.3 2B / 8B, Ministral 3B / 8B, InternLM 3 8B, OpenCoder 1.5B / 8B, Seed-Coder 8B, MiniCPM-V 2.6 / MiniCPM-o 2.6, Idefics 3 8B, Fuyu-8B, Emu3, and Stable Diffusion 3.5 Large / Turbo. EXAONE 3.5 2.4B / 7.8B fits at Q6_K using 7.7 GB. You can also run 7B models like Mistral 7B at Q6_K using 7.4 GB. Other 7B models like Qwen2.5 0.5B / 1.5B / 3B / 7B, OLMo 2 1B / 7B, Falcon 3 1B / 3B / 7B, Command R7B, OpenHermes 2.5, Zephyr 7B Beta, OpenChat 3.5, Starling LM 7B, and Codestral Mamba 7B fit at Q6_K using 6.9 GB.

When a model is too large for the 8 GB video memory, you must use CPU offload. This process shares the workload with your system RAM but reduces generation speed. For these cases, we assume a system with 32 GB of system RAM. Open-Sora 2.0 11B needs 8.1 GB at Q4_K_M and 10.1 GB of system RAM. FLUX.1 dev 12B needs 14.4 GB at FP8 / optimized and 16.4 GB of system RAM. Vicuna 13B needs 9.5 GB at Q4_K_M and 11.5 GB of system RAM.

Other 12B models also require CPU offload on this hardware. Gemma 3 12B, Gemma 4 12B, Mistral NeMo 12B, Pixtral 12B, FLUX.1 schnell, FLUX.1 Kontext dev, and FLUX.1 Krea dev all need 8.8 GB at Q4_K_M and 10.8 GB of system RAM. Note that all memory calculations for these local models assume a standard 4k context window. Expanding the context window to process longer texts will require additional video memory and may force earlier CPU offloading.