Best local AI models for AMD RX 6650M XT

The largest models that fit fully

The 30 largest of the 123 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The AMD Radeon RX 6650M XT is a mobile graphics processor equipped with 8 GB of GDDR6 dedicated video memory. This memory pool determines the size of the artificial intelligence models you can run locally. To run a model entirely on your graphics hardware, the model files and its working memory must fit within this 8 GB limit. Running models locally ensures your data remains private and does not rely on an internet connection.

Model sizes are measured in parameters like 7B or 9B, which stand for billions of variables. To fit these models into your 8 GB of video memory, developers use quantization. The quantization column shows the compression level, such as Q4_K_M, Q5_K_M, or Q6_K. Lower quantization levels compress the model more to save space, while higher levels like Q6_K preserve more original model quality but require more memory.

With 8 GB of video memory, you can run several highly capable models entirely on the graphics card. The Mochi 1 10B model fits at a Q4_K_M quantization using 7.3 GB of memory. Gemma 2 9B fits at Q4_K_M using exactly 8 GB of memory. Several 9B models like Nemotron Nano 4B / 9B, GLM-4 9B / GLM-4.5-Air, Yi-Coder 1.5B / 9B, GLM-4-9B-Chat / CodeGeeX4, and GLM-4V-9B / GLM-4.1V-Thinking fit at Q5_K_M using 7.7 GB of memory. Chroma 8.9B also fits at Q5_K_M using 7.6 GB of memory.

You can also run many 8B and 7B models at higher quality settings. Llama 3.1 8B fits at Q5_K_M using 7.4 GB of memory. Models like Granite 3.3 2B / 8B, Ministral 3B / 8B, InternLM 3 8B, OpenCoder 1.5B / 8B, Seed-Coder 8B, MiniCPM-V 2.6 / MiniCPM-o 2.6, Idefics 3 8B, Fuyu-8B, Emu3, and Stable Diffusion 3.5 Large / Turbo fit at Q6_K using 7.9 GB of memory. EXAONE 3.5 2.4B / 7.8B fits at Q6_K using 7.7 GB of memory. Mistral 7B fits at Q6_K using 7.4 GB of memory. Other 7B models like Qwen2.5 0.5B / 1.5B / 3B / 7B, OLMo 2 1B / 7B, Falcon 3 1B / 3B / 7B, Command R7B, OpenHermes 2.5, Zephyr 7B Beta, OpenChat 3.5, Starling LM 7B, and Codestral Mamba 7B fit at Q6_K using 6.9 GB of memory.

When a model is too large for your 8 GB of video memory, you can use CPU offloading. This process splits the model between your graphics card and your system RAM. Offloading allows you to run larger models, but it comes with a speed cost. Because system RAM is much slower than GDDR6 video memory, your generation speeds will drop significantly when offloading.

Assuming you have 32 GB of system RAM, you can offload several larger models. Open-Sora 2.0 11B needs 8.1 GB at Q4_K_M and 10.1 GB of system RAM. FLUX.1 dev 12B needs 14.4 GB at FP8 / optimized and 16.4 GB of system RAM. Models like Gemma 3 12B, Gemma 4 12B, Mistral NeMo 12B, Pixtral 12B, FLUX.1 schnell 12B, FLUX.1 Kontext dev 12B, and FLUX.1 Krea dev 12B need 8.8 GB at Q4_K_M and 10.8 GB of system RAM. Vicuna 13B needs 9.5 GB at Q4_K_M and 11.5 GB of system RAM.

All memory calculations shown here assume a standard 4k context window. The context window is the amount of text the model can remember during a single conversation. If you increase the context window beyond 4000 tokens, the model will require significantly more video memory. This extra memory usage might force you to use a lower quantization level or rely on CPU offloading to avoid running out of memory.