Best local AI models for AMD RX 7650 GRE

The largest models that fit fully

The 30 largest of the 123 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The AMD RX 7650 GRE graphics card features 8 GB of GDDR6 video memory. This dedicated memory determines the maximum size of the artificial intelligence models you can run entirely on your hardware. When a model fits completely within this limit, your system processes tokens at maximum speed. If a model exceeds this limit, you must use alternative execution strategies.

Quantization is a method used to compress model weights. The quant column shows the optimal balance between model accuracy and memory usage. For example, Gemma 2 9B fits at the Q4_K_M quantization level using exactly 8 GB of video memory. Smaller models like Llama 3.1 8B can run at a higher quality Q5_K_M quantization using 7.4 GB of memory. Models like Mistral 7B run at the even higher quality Q6_K quantization using 7.4 GB of memory.

You can run larger models by offloading parts of the computation to your system RAM. This approach requires a system with at least 32 GB of system RAM. For instance, FLUX.1 dev requires 14.4 GB of memory at FP8 or optimized settings, which uses 16.4 GB of system RAM. Similarly, Gemma 3 12B requires 8.8 GB of memory at Q4_K_M quantization, which uses 10.8 GB of system RAM. Offloading allows you to run these larger models but decreases generation speeds significantly.

Other models that utilize CPU offloading include Vicuna 13B, which needs 9.5 GB at Q4_K_M quantization and uses 11.5 GB of system RAM. Mistral NeMo 12B and Pixtral 12B both need 8.8 GB at Q4_K_M quantization and use 10.8 GB of system RAM. Image generation models like FLUX.1 schnell also need 8.8 GB at Q4_K_M quantization and use 10.8 GB of system RAM. Open-Sora 2.0 needs 8.1 GB at Q4_K_M quantization and uses 10.1 GB of system RAM.

Memory calculations for these models assume a standard 4k context window. The context window is the total amount of text the model can remember during a single conversation. If you increase the context window beyond 4k tokens, the model will require more video memory. This extra memory requirement might force you to use a lower quantization level or trigger CPU offloading.