Best local AI models for AMD RX 5700 XT

The largest models that fit fully

The 30 largest of the 123 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The AMD Radeon RX 5700 XT graphics card comes equipped with 8 GB of GDDR6 memory. This onboard memory determines the maximum size of the artificial intelligence models you can run directly on your hardware. To execute local models efficiently, the entire active weight set should ideally fit within this physical limit. When a model exceeds this capacity, execution speed drops significantly.

To fit larger models into the 8 GB limit, developers use quantization. Quantization reduces the precision of model weights to save space. The quant column indicates the optimal balance of size and accuracy for each model. For example, the Mochi 1 10B model fits within 7.3 GB of memory when using the Q4_K_M quantization level. Similarly, Llama 3.1 8B fits within 7.4 GB of memory at the Q5_K_M quantization level.

For models that require slightly more than the available 8 GB of video memory, you can use CPU offload. This technique splits the workload between your graphics card and your system memory. If you have 32 GB of system RAM, you can run larger models by offloading some layers. For instance, Gemma 3 12B requires 8.8 GB at the Q4_K_M quantization level, which uses 10.8 GB of system RAM during execution.

Other models that utilize CPU offload include FLUX.1 dev, which requires 14.4 GB at FP8 or optimized settings and uses 16.4 GB of system RAM. Mistral NeMo 12B and Pixtral 12B both require 8.8 GB at the Q4_K_M quantization level and use 10.8 GB of system RAM. Vicuna 13B requires 9.5 GB at the Q4_K_M quantization level and uses 11.5 GB of system RAM.

When running models near the memory limit, you must consider the context window. The memory figures listed are calculated with a standard 4k context window. If you increase the context length to process longer documents or conversations, the memory usage will grow. This extra memory demand can push a model past the 8 GB limit and trigger slow performance or system instability.

For maximum speed without offloading, choose models that fit entirely within your video memory. Mistral 7B uses 7.4 GB at the Q6_K quantization level. Qwen2.5 7B, OLMo 2 7B, and Falcon 3 7B all use 6.9 GB at the Q6_K quantization level. These options leave a safe margin of memory for your operating system and display outputs.