Best local AI models for AMD Pro W5700

The largest models that fit fully

The 30 largest of the 123 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The AMD Radeon Pro W5700 is a professional workstation graphics card equipped with 8 GB of GDDR6 onboard memory. This dedicated video memory determines the maximum size of the artificial intelligence models you can run entirely on the hardware. To run a model smoothly, its total memory footprint must remain under this 8 GB threshold. If a model exceeds this limit, your system must use alternative execution strategies.

The quant column indicates the quantization level used to compress the model weights. Quantization reduces the precision of the model parameters to make them smaller. For example, a Q4_K_M quant uses approximately four bits per weight, while a Q6_K quant uses six bits. Choosing the best quant allows you to run larger models like Mochi 1 10B or Gemma 2 9B. Mochi 1 10B fits at Q4_K_M using 7.3 GB of video memory, while Gemma 2 9B fits at Q4_K_M using exactly 8 GB.

Several high quality models fit comfortably within the hardware limits. Nemotron Nano 9B, GLM-4 9B, Yi-Coder 9B, GLM-4-9B-Chat, and GLM-4V-9B all fit at the Q5_K_M quant using 7.7 GB of memory. Chroma 8.9B fits at Q5_K_M using 7.6 GB. Llama 3.1 8B fits at Q5_K_M using 7.4 GB. Granite 3.3 8B, Ministral 8B, InternLM 3 8B, OpenCoder 8B, Seed-Coder 8B, MiniCPM-V 2.6, Idefics 3 8B, Fuyu-8B, Emu3, and Stable Diffusion 3.5 Large fit at Q6_K using 7.9 GB.

Slightly smaller architectures provide even more safety margin. EXAONE 3.5 7.8B fits at Q6_K using 7.7 GB. Mistral 7B fits at Q6_K using 7.4 GB. Qwen2.5 7B, OLMo 2 7B, Falcon 3 7B, Command R7B, OpenHermes 2.5, Zephyr 7B Beta, OpenChat 3.5, Starling LM 7B, and Codestral Mamba 7B all fit at Q6_K using 6.9 GB. These options leave extra video memory available for other system tasks.

When a model is too large for the 8 GB video memory, you can offload parts of it to your system RAM. This process requires a system with at least 32 GB of system RAM. Offloading allows you to run Open-Sora 2.0 11B, which needs 8.1 GB at Q4_K_M and 10.1 GB of system RAM. FLUX.1 dev needs 14.4 GB at FP8 and 16.4 GB of system RAM. Gemma 3 12B, Gemma 4 12B, Mistral NeMo 12B, Pixtral 12B, FLUX.1 schnell, FLUX.1 Kontext dev, and FLUX.1 Krea dev all need 8.8 GB at Q4_K_M and 10.8 GB of system RAM. Vicuna 13B needs 9.5 GB at Q4_K_M and 11.5 GB of system RAM. Offloading prevents out of memory errors but reduces processing speed significantly.

All memory calculations assume a standard 4k context window. The context window is the amount of text the model can process at one time. If you increase the context window beyond 4k tokens, the model will require more memory. This extra memory usage can cause a model that normally fits within the 8 GB limit to overflow into system RAM.