Best local AI models for AMD R9 370

The largest models that fit fully

The 30 largest of the 56 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The AMD Radeon R9 370 graphics card features 2 GB of GDDR5 onboard memory. This memory limit dictates the size of the artificial intelligence models you can run locally. To fit within this 2 GB frame buffer, models must undergo quantization. Quantization reduces the precision of the model weights to save space. The quant column indicates the specific level of compression applied to the model. A Q4_K_M quant offers medium compression, while Q8_0 represents a higher quality eight bit quantization that requires more memory.

For fully local execution, the model and its active memory must fit entirely within the 2 GB GDDR5 limit. The Allegro 2.8B model is the largest fitting model at a Q4_K_M quant, using exactly 2 GB of video memory. Other models like Open-Sora Plan 2.7B and LFM2 2.6B also fit using Q4_K_M quants. If you select models with higher precision quants like SmolVLM 2B at Q6_K or Zonos 1.6B at Q8_0, they will also utilize up to 2 GB of your video memory.

When a model exceeds the 2 GB video memory limit, you must use CPU offload. This technique splits the workload between your graphics card and your system RAM. We assume your computer has 32 GB of system RAM for these scenarios. Offloading allows you to run larger models like SmolLM3 3B or Kandinsky 3.1. These 3B models need 2.2 GB of video memory at Q4_K_M and require an additional 4.2 GB of system RAM to function.

CPU offload comes with a performance cost. Transferring data between the AMD R9 370 GDDR5 memory and the system RAM over the PCIe bus is much slower than running entirely on the graphics card. You will experience lower generation speeds for models like MusicGen 3.3B or Stable Diffusion XL. Stable Diffusion XL needs 4.1 GB at FP8 or optimized settings, which requires 6.1 GB of system RAM.

You must also consider the context window when running text models. The memory figures listed are for the base model files. Running a model with a standard 4k context window increases the memory footprint during active generation. This extra memory usage can push a model that fits at idle over the 2 GB limit. For the best performance on the AMD R9 370, choose smaller models like TinyLlama 1.1B at Q8_0 which only uses 1.4 GB of video memory.