Best local AI models for AMD RX 6750 GRE 12GB

The largest models that fit fully

The 30 largest of the 147 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The AMD RX 6750 GRE features 12GB of GDDR6 memory. This onboard VRAM determines the size of the artificial intelligence models you can run locally. For the best speed and performance, the entire model must fit inside this 12GB limit. If a model exceeds this capacity, your system must use slower system memory to process the remaining data.

The quantization column indicates the compression level used on each model. Quantization reduces the precision of model weights to save space. A Q4_K_M quantization represents a four bit compression that offers a great balance between size and quality. Higher quantizations like Q5_K_M or Q6_K provide better accuracy but require more VRAM. For example, the 12B models like Gemma 3 12B, Gemma 4 12B, and Mistral NeMo 12B utilize a Q6_K quantization which fits within 11.8 GB of VRAM.

You can run larger models on your hardware by using CPU offload. This technique splits the workload between your GPU and your system RAM. If you have 32 GB of system RAM, you can run models that exceed 12GB. For example, FLUX.1 dev requires 14.4 GB at FP8 and uses 16.4 GB of system RAM. Similarly, Codestral 22B requires 16.1 GB at Q4_K_M and uses 18.1 GB of system RAM. CPU offload allows you to run these larger models, but it significantly reduces your generation speed.

Your VRAM usage estimates assume a standard 4k context window. The context window is the memory used to store your conversation history and prompt details. As your conversation grows longer, the context window consumes more VRAM. If you increase the context window beyond 4k tokens, you will need to choose smaller models to avoid running out of memory.

With 12GB of VRAM, you can run many capable models entirely on your GPU. The 14B models like Phi-4 and Wan 2.1 fit well at Q5_K_M quantization with 11.9 GB used. If you want higher precision, 9B models like GLM-4 9B and Yi-Coder 9B run at Q8_0 quantization using 11.4 GB of VRAM. This hardware provides a versatile platform for local text generation, coding assistance, and image creation.