Best local AI models for AMD Pro WX 8200

The largest models that fit fully

The 30 largest of the 123 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The AMD Radeon Pro WX 8200 workstation graphics card features 8 GB of HBM2 frame buffer memory. This high bandwidth memory determines the maximum size of the artificial intelligence models you can run locally on the hardware. To run a model entirely on the graphics processor, the model files and the active context data must fit within this 8 GB limit.

The quantization column indicates the compression level applied to each model. Quantization reduces the precision of model weights to save memory. For example, the 10B Mochi 1 model fits in 7.3 GB of memory using the Q4_K_M quantization. Models like Gemma 2 9B require exactly 8 GB of memory at the Q4_K_M quantization, which fully utilizes the onboard frame buffer.

Many popular models can run at higher precision levels on this hardware. The Llama 3.1 8B model fits in 7.4 GB of memory using the Q5_K_M quantization. Several 8B models, including Granite 3.3 8B, Ministral 8B, InternLM 3 8B, and Stable Diffusion 3.5 Large, run at the Q6_K quantization while consuming 7.9 GB of memory. Smaller 7B models like Mistral 7B use 7.4 GB at Q6_K, while Qwen2.5 7B and Falcon 3 7B use 6.9 GB at Q6_K.

When a model size exceeds the 8 GB HBM2 capacity, you must offload layers to the system memory. Offloading allows you to run larger models but reduces processing speed because system RAM is slower than graphics memory. This setup assumes your computer has 32 GB of system RAM. For instance, Gemma 3 12B and Mistral NeMo 12B require 8.8 GB of memory at Q4_K_M, which uses 10.8 GB of system RAM. The FLUX.1 dev model requires 14.4 GB at FP8, which uses 16.4 GB of system RAM.

Memory consumption calculations assume a standard 4k context window. The context window represents the total text history the model can process at one time. If you increase the context length beyond 4000 tokens, the memory usage will rise. This extra memory demand may require you to use a more compressed quantization or offload more layers to system RAM to prevent running out of memory.