Best local AI models for AMD RX Vega 56

The largest models that fit fully

The 30 largest of the 123 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The AMD RX Vega 56 graphics card features 8 GB of HBM2 memory. This dedicated video memory is the most critical factor for running local artificial intelligence models. When a model fits entirely within this 8 GB space, the graphics processor runs at its maximum speed. If a model exceeds this limit, performance drops significantly because the system must transfer data across slower system channels.

To fit larger models into the 8 GB HBM2 memory, you must use quantized versions. Quantization reduces the precision of model weights to save space. The quant column shows the best balance of quality and size for each model. For example, the 10B Mochi 1 fits at the Q4_K_M quantization level using 7.3 GB of memory. Smaller models like Mistral 7B can run at the higher quality Q6_K quantization level using 7.4 GB of memory.

Several 9B and 8B models fit comfortably within the 8 GB limit of the card. Gemma 2 9B runs at Q4_K_M using exactly 8 GB of memory. Nemotron Nano 4B / 9B, GLM-4 9B / GLM-4.5-Air, and Yi-Coder 1.5B / 9B all run at Q5_K_M using 7.7 GB of memory. You can also run Llama 3.1 8B at Q5_K_M using 7.4 GB of memory. Granite 3.3 2B / 8B and Ministral 3B / 8B run at Q6_K using 7.9 GB of memory.

For models that exceed 8 GB of video memory, you can use CPU offload if your computer has 32 GB of system RAM. This process splits the model layers between your graphics card and your system memory. This allows you to run larger models like Gemma 3 12B, Gemma 4 12B, and Mistral NeMo 12B at Q4_K_M. These models require 8.8 GB of video memory and 10.8 GB of system RAM. You can even run Vicuna 13B at Q4_K_M using 9.5 GB of video memory and 11.5 GB of system RAM.

CPU offload comes with a significant performance cost. Processing data in system RAM is much slower than using the fast HBM2 memory on your card. Additionally, all memory calculations assume a standard 4k context window. If you increase the context window to process longer documents or chat histories, the model will require more memory. This extra memory usage can push a model over the 8 GB limit and trigger slow system RAM usage.