Best local AI models for AMD RX 7700 XT

The largest models that fit fully

The 30 largest of the 147 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The AMD RX 7700 XT graphics card comes equipped with 12 GB of GDDR6 memory. This onboard memory determines the maximum size of the artificial intelligence models you can run locally. To run a model entirely on your graphics card for fast processing, the model files and the active workspace must fit within this 12 GB limit.

The quantization column shows the compression level applied to each model. Quantization reduces the size of a model so it fits into less memory. For example, the DeepSeek-Coder-V2 16B model fits within 11.7 GB of memory when using the Q4_K_M quantization. For smaller models like Gemma 3 12B or Mistral NeMo 12B, you can use a higher quality Q6_K quantization which takes 11.8 GB of memory.

If a model is too large for the 12 GB memory limit, you can use CPU offload. This method splits the workload between your graphics card and your system RAM. Running FLUX.1 dev at FP8 requires 14.4 GB of memory, which means it needs 16.4 GB of system RAM to assist your graphics card. Other models like Codestral 22B require 16.1 GB of memory and need 18.1 GB of system RAM to run.

CPU offload allows you to run larger models like CogVLM2 or Solar Pro, but it comes with a performance cost. Moving data between your system RAM and the graphics card is much slower than keeping everything in the GDDR6 memory. Your generation speeds will drop significantly when offloading parts of the model to your system memory.

All memory calculations are based on a standard 4k context window. If you increase the context window to process longer documents or chat histories, the model will require more memory. Running a model very close to the 12 GB limit, such as Phi-4 at 11.9 GB with a Q5_K_M quantization, leaves very little room for context expansion.