Best local AI models for AMD RX Vega M GL

The largest models that fit fully

The 30 largest of the 81 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The AMD RX Vega M GL graphics processor features 4 GB of high bandwidth memory. This dedicated memory determines the size of the artificial intelligence models you can run locally. When a model fits entirely within this 4 GB limit, the graphics processor handles all calculations. This local execution ensures the fastest possible processing speeds for your tasks.

The quant column shows the quantization level used to compress each model. Quantization reduces the precision of model weights to save memory. For example, the 5B Lumina-Next model fits in 3.7 GB of memory when compressed to the Q4_K_M quantization level. Similarly, the 4B Qwen3 model fits in 3.9 GB of memory using the Q6_K quantization level. Smaller models like the 3B SmolLM3 can run at the higher Q8_0 quantization level using 3.8 GB of memory.

Running models at a standard 4k context window requires additional memory. The context window stores the history of your current conversation. As your chat history grows toward 4000 tokens, the memory usage increases. You must select a model that leaves enough free space within the 4 GB limit to accommodate this active context data.

If a model exceeds the 4 GB limit, you must use CPU offload. This technique splits the model between your graphics memory and your system memory. We assume your system has 32 GB of system RAM for these calculations. CPU offload allows you to run larger models, but the transfer of data between the system RAM and the graphics processor reduces processing speeds.

For example, the 7B Mistral model requires 5.7 GB of memory at the Q4_K_M quantization level. To run this model, you must offload 7.7 GB of data to your system RAM. The 7B Qwen2.5 model requires 5.1 GB of memory at the Q4_K_M quantization level and needs 7.1 GB of system RAM. Stable Diffusion XL requires 4.1 GB of memory at the FP8 level and needs 6.1 GB of system RAM.