Best local AI models for AMD Pro W6800X Duo

The largest models that fit fully

The 30 largest of the 183 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The AMD Pro W6800X Duo graphics card features 32 GB GDDR6 memory. This memory capacity determines the size of the artificial intelligence models you can run locally. To run a model entirely on the graphics hardware, the model files and its working memory must fit within this 32 GB limit.

The quantization column indicates the compression level applied to each model. Quantization reduces the size of a model so it uses less memory. A Q5_K_M quant represents a high quality five bit quantization. A Q6_K quant represents a six bit quantization. A Q8_0 quant represents an eight bit quantization which preserves more original model accuracy but requires more memory.

For maximum performance, you can run models like Seed-OSS 36B at Q5_K_M which uses 30.7 GB of memory. You can also run Qwen3.6-35B-A3B or Command R (35B) at Q5_K_M using 29.8 GB of memory. Other options include Yi 1.5 9B / 34B, Granite Code 3B to 34B, LLaVA 1.5 / 1.6 (7B to 34B), and Ovis 2 at Q5_K_M using 29 GB of memory.

Models like Qwen3 8B / 14B / 32B, Qwen3.5 (dense variants), Aya Expanse 8B / 32B, Granite 4.0 Small/Tiny, and Qwen2.5-Coder 0.5B to 32B run at Q6_K using 31.5 GB of memory. You can also run Gemma 3 27B, Gemma 3 4B/12B/27B (vision), and Wan 2.2 / 2.5 at Q6_K using 26.6 GB of memory. Aria runs at Q8_0 using 31.8 GB of memory.

When a model exceeds the 32 GB graphics memory, you must offload parts of the model to your system RAM and CPU. CPU offload allows you to run larger models but it reduces processing speed. For example, Mixtral 8x7B needs 34.4 GB at Q4_K_M and requires 36.4 GB system RAM. Llama 3.1 Nemotron 51B needs 37.3 GB at Q4_K_M and requires 39.3 GB system RAM. Jamba 1.5 Mini / Large needs 38.1 GB at Q4_K_M and requires 40.1 GB system RAM.

All memory calculations in this guide assume a standard 4k context window. If you increase the context window to process longer documents, the model will require more memory. This extra memory usage might require you to choose a lower quantization level or use CPU offload.