Best local AI models for AMD R7 M370

The largest models that fit fully

The 30 largest of the 56 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The AMD Radeon R7 M370 is an entry level graphics card equipped with 2 GB of GDDR5 video memory. This dedicated memory capacity determines which artificial intelligence models can run directly on your hardware. To execute local AI tasks efficiently, the model files must fit within this 2 GB limit. Running models entirely on your graphics card ensures the fastest possible processing speeds.

The quantization column indicates the compression level applied to each model. Quantization reduces the size of the model weights to save memory. For example, the 2.8B Allegro model fits in 2 GB of video memory when using the Q4_K_M quantization. Smaller models like the 1.7B SmolLM2 can run at a higher quality Q6_K quantization while using 1.7 GB of memory. The 1.1B TinyLlama runs at the premium Q8_0 quantization using only 1.4 GB of video memory.

When a model exceeds your 2 GB video memory, you can use CPU offload if your computer has enough system RAM. This setup assumes you have 32 GB of system RAM. Offloading allows you to run larger models like the 3B SmolLM3 or the 3.5B SDXL Turbo. However, sharing the workload between your graphics card and system RAM introduces a performance cost. Your processing speed will decrease because system RAM is much slower than GDDR5 video memory.

Using CPU offload requires specific memory allocations. The 3.417B Stable Diffusion XL model needs 4.1 GB of memory at FP8 or optimized settings, which requires 6.1 GB of system RAM. The 3.3B MusicGen needs 2.4 GB of memory at Q4_K_M quantization and requires 4.4 GB of system RAM. Smaller offloaded models like the 3B Kandinsky 3.1 or the 3B Orpheus TTS need 2.2 GB of memory at Q4_K_M quantization and require 4.2 GB of system RAM.

You must also consider the 4k context window caveat when running text models. The memory figures listed only cover the base model files. As you type longer prompts and receive longer answers, the active context window consumes additional video memory. If you use the maximum 4k context length, the system might run out of video memory. You may need to use a smaller model or a lower quantization to prevent crashes during long conversations.