Best local AI models for AMD R9 M275X

The largest models that fit fully

The 30 largest of the 56 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The AMD Radeon R9 M275X is a legacy mobile graphics card equipped with 2 GB of GDDR5 video memory. This hardware configuration limits the size of artificial intelligence models you can run entirely on the GPU. To execute local models successfully, you must select small architectures and use optimized quantization levels to fit within this strict hardware ceiling.

The memory size of 2 GB GDDR5 dictates the maximum footprint of the active model. The quantization column indicates the specific compression format used to shrink the weights of the model. For example, the 2.8B Allegro model and the 2.7B Open-Sora Plan model both utilize a Q4_K_M quantization to fit exactly into 2 GB of video memory. Similarly, the LFM2 2.6B and Playground v2.5 models require a Q4_K_M quantization which consumes 1.9 GB of memory.

As models decrease in parameter size, you can use higher quality quantizations. The Stable Diffusion 3 Medium, Pyramid Flow, and Wav2Vec2 XLS-R models all have a 2B parameter size and run at a Q6_K quantization using 2 GB of memory. Even smaller models like the 1.6B StableLM 2, Sana 1.6B, Zonos 0.1, and Dia 1.6B can run at a high quality Q8_0 quantization while utilizing 2 GB of video memory. TinyLlama 1.1B and SantaCoder 1.1B run at Q8_0 quantization and use only 1.4 GB of memory.

When a model exceeds the 2 GB physical limit of the graphics card, you must use CPU offload. This process splits the model weights between your video memory and your system RAM. For these scenarios, we assume your computer has 32 GB of system RAM. For instance, running the 3B SmolLM3, Replit Code v1.5 3B, or Kandinsky 3.1 requires 2.2 GB of video memory at Q4_K_M quantization and an additional 4.2 GB of system RAM. Larger models like the 3.5B SDXL Turbo and SDXL Lightning need 2.6 GB of video memory at Q4_K_M quantization and 4.6 GB of system RAM.

CPU offload allows you to run larger models but it introduces a severe performance cost. Moving data between the system RAM and the graphics card over the system bus is much slower than processing data directly inside the GDDR5 memory. This transfer delay significantly reduces the generation speed of your text or images.

You must also consider the 4k context window caveat when running text models on this hardware. The memory numbers listed only cover the static model weights. As your conversation grows toward a 4k context window, the active memory requirements will increase. This extra data can easily push a tight model like the 1.7B SmolLM2 or the 1.9B Moondream 2 over the 2 GB limit and trigger slow system RAM fallback.