Best local AI models for AMD RX Vega 64

The largest models that fit fully

The 30 largest of the 123 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The AMD RX Vega 64 graphics card features 8 GB of HBM2 memory. This onboard memory determines the maximum size of the artificial intelligence models you can run locally. When a model fits entirely within this 8 GB space, it runs at the maximum speed of your hardware. If a model exceeds this limit, you must use system memory to handle the extra data.

The quantization column indicates the compression level applied to each model. Quantization reduces the size of a model so it can fit into your hardware. For example, the Mochi 1 10B model fits into 7.3 GB of memory using the Q4_K_M quantization. The Gemma 2 9B model uses exactly 8 GB of memory at the Q4_K_M quantization. Models like Llama 3.1 8B use 7.4 GB of memory with the Q5_K_M quantization.

Smaller models can use higher quality quantizations. The Granite 3.3 2B / 8B, Ministral 3B / 8B, and InternLM 3 8B models all use 7.9 GB of memory at the Q6_K quantization. Other models like OpenCoder 1.5B / 8B, Seed-Coder 8B, and MiniCPM-V 2.6 / MiniCPM-o 2.6 also run at the Q6_K quantization using 7.9 GB of memory. This quantization level is also used by Idefics 3 8B, Fuyu-8B, Emu3, and Stable Diffusion 3.5 Large / Turbo.

When you run larger models, you must offload some data to your system RAM. This process requires a system with at least 32 GB of system RAM. Offloading allows you to run models like FLUX.1 dev, which needs 14.4 GB at FP8 / optimized and uses 16.4 GB of system RAM. The Gemma 3 12B, Gemma 4 12B, Mistral NeMo 12B, and Pixtral 12B models all need 8.8 GB at Q4_K_M and use 10.8 GB of system RAM. Offloading makes these models run much slower because system RAM is slower than HBM2 memory.

Other offload options include Open-Sora 2.0, which needs 8.1 GB at Q4_K_M and uses 10.1 GB of system RAM. The FLUX.1 schnell, FLUX.1 Kontext dev, and FLUX.1 Krea dev models also need 8.8 GB at Q4_K_M and use 10.8 GB of system RAM. The Vicuna 13B model needs 9.5 GB at Q4_K_M and uses 11.5 GB of system RAM. You must also reserve memory for the context window, which is the active memory used during a conversation. The memory figures listed here assume a standard 4k context window, and increasing this window will require more memory.