Best local AI models for AMD R9 M375X

The largest models that fit fully

The 30 largest of the 56 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The AMD Radeon R9 M375X is an older mobile graphics card equipped with 2 GB of GDDR5 video memory. This dedicated memory pool is the primary bottleneck for running artificial intelligence models locally. To run a model entirely on this hardware, the model files must fit within this 2 GB limit. This page lists the largest compatible models and explains how to configure them for the best performance.

The quant column indicates the quantization level used to compress each model. Quantization reduces the precision of model weights to save memory. For example, the Allegro 2.8B model fits in 2 GB of memory when compressed to the Q4_K_M quantization level. Smaller models like SmolLM2 1.7B can run at the higher quality Q6_K quantization level while using 1.7 GB of memory. Models like TinyLlama 1.1B can run at the Q8_0 level using 1.4 GB of memory.

When a model exceeds the 2 GB video memory limit, you must use CPU offload. This technique stores part of the model in your system RAM instead of the video memory. For example, running the SmolLM3 3B model at Q4_K_M requires 2.2 GB of memory, which uses your entire video memory and requires 4.2 GB of system RAM. This offload process allows you to run larger models like Stable Diffusion XL, but it significantly reduces processing speed.

Running models with a 4k context window or larger will increase memory consumption during inference. The listed memory usage figures represent the base model size. As you type longer prompts or generate longer responses, the active memory usage will grow. You may need to select a smaller model size or a lower quantization level to prevent running out of memory during long conversations.

For audio and speech tasks, you can run models like Whisper Large v3 at Q8_0 quantization using 2 GB of memory. For image generation, Stable Diffusion 3.5 Medium fits at Q4_K_M quantization using 1.8 GB of memory. If you have 32 GB of system RAM, you can offload larger tools like SDXL Turbo which requires 2.6 GB of video memory and 4.6 GB of system RAM.