Best local AI models for AMD RX 9060 XT 8GB

The largest models that fit fully

The 30 largest of the 123 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The AMD RX 9060 XT graphics card comes equipped with 8 GB of GDDR6 memory. This onboard memory size determines which artificial intelligence models can run entirely on your hardware. For local execution, the model weights and the active context data must fit within this physical limit to ensure fast processing speeds. If a model exceeds this capacity, performance drops significantly because the system must transfer data outside the graphics card.

Quantization is a method used to compress model files so they require less memory. The quant column shows the best compression level that fits within your hardware limits. For example, Gemma 2 9B fits at the Q4_K_M quantization level which uses exactly 8 GB of memory. Other models like Llama 3.1 8B fit at the Q5_K_M quantization level using 7.4 GB of memory. Smaller models like Mistral 7B can run at the higher quality Q6_K quantization level using 7.4 GB of memory.

When a model is slightly too large for the 8 GB onboard memory, you can use CPU offload. This technique splits the model layers between your graphics card and your system RAM. You need at least 32 GB of system RAM to use this option effectively. Running Gemma 3 12B at the Q4_K_M quantization level requires 8.8 GB of video memory and 10.8 GB of system RAM. This offload process allows you to run larger models but it costs performance because system RAM is much slower than GDDR6 memory.

Many popular models can run on this hardware using CPU offload. FLUX.1 dev requires 14.4 GB of video memory at FP8 or optimized settings along with 16.4 GB of system RAM. Other models like Mistral NeMo 12B, Pixtral 12B, FLUX.1 schnell, FLUX.1 Kontext dev, and FLUX.1 Krea dev all require 8.8 GB of video memory at the Q4_K_M quantization level and 10.8 GB of system RAM. Vicuna 13B requires 9.5 GB of video memory at Q4_K_M and 11.5 GB of system RAM.

For fully local execution without offloading, you have many options under the 8 GB limit. Mochi 1 10B fits at Q4_K_M using 7.3 GB of memory. Nemotron Nano 4B / 9B, GLM-4 9B / GLM-4.5-Air, Yi-Coder 1.5B / 9B, GLM-4-9B-Chat / CodeGeeX4, and GLM-4V-9B / GLM-4.1V-Thinking all fit at Q5_K_M using 7.7 GB of memory. Chroma 8.9B fits at Q5_K_M using 7.6 GB of memory.

Several models run at the high quality Q6_K quantization level. Granite 3.3 2B / 8B, Ministral 3B / 8B, InternLM 3 8B, OpenCoder 1.5B / 8B, Seed-Coder 8B, MiniCPM-V 2.6 / MiniCPM-o 2.6, Idefics 3 8B, Fuyu-8B, Emu3, and Stable Diffusion 3.5 Large / Turbo all use 7.9 GB of memory. EXAONE 3.5 2.4B / 7.8B uses 7.7 GB of memory. Qwen2.5 0.5B / 1.5B / 3B / 7B, OLMo 2 1B / 7B, Falcon 3 1B / 3B / 7B, Command R7B, OpenHermes 2.5, Zephyr 7B Beta, OpenChat 3.5, Starling LM 7B, and Codestral Mamba 7B all use 6.9 GB of memory.

You must consider the context window size when loading these models. The memory usage figures are calculated using a standard 4k context window. If you increase the context window to process longer documents or chat histories, the memory requirement will grow. Running a model right at the 8 GB limit might cause out of memory errors or force CPU offloading if your context size exceeds this 4k limit.