Best local AI models for AMD RX 6750 XT

The largest models that fit fully

The 30 largest of the 147 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The AMD RX 6750 XT graphics card features 12 GB of GDDR6 memory. This onboard memory determines the size of the AI models you can run directly on your hardware. For the best performance and speed, the entire model must fit within this 12 GB limit. If a model exceeds this capacity, your system must use alternative execution methods.

The quantization column indicates the compression level applied to each model. Quantization reduces the size of the model weights to save memory. For example, the 16B DeepSeek-Coder-V2 and Kimi-VL A3B models fit within 11.7 GB using the Q4_K_M quantization. Other models like Gemma 3 12B, Gemma 4 12B, and Mistral NeMo 12B can run at a higher quality Q6_K quantization while using 11.8 GB of memory.

Models like Phi-4, Phi-3 Medium, and Wan 2.2 T2I utilize 11.9 GB of memory at the Q5_K_M quantization level. If you choose smaller models such as the 9B Nemotron Nano or GLM-4, you can use the Q8_0 quantization which requires 11.4 GB of memory. Higher quantization levels preserve more of the original model accuracy but require more memory per parameter.

When a model is too large for the 12 GB onboard memory, you can use CPU offload. This process splits the model between your graphics card and your system RAM. CPU offloading allows you to run larger models like the 22B Codestral or Solar Pro, which need 16.1 GB at Q4_K_M and 18.1 GB of system RAM. You can also run FLUX.1 dev at FP8, which requires 14.4 GB of memory and 16.4 GB of system RAM.

CPU offload comes with a performance cost. Transferring data between your system RAM and the graphics card is much slower than running everything on the GDDR6 memory. Models like CogVLM2 and Qwen-Image require over 15 GB of system RAM when offloaded. This configuration makes execution possible but significantly reduces the generation speed.

Memory calculations must also account for the context window. The listed memory usage figures represent the model at a base 4k context limit. As your conversation or input text grows, the system requires additional memory to store the active context. Running a model very close to the 12 GB limit may cause out of memory errors if your chat history becomes too long.