Best local AI models for AMD R7 360

The largest models that fit fully

The 30 largest of the 56 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The AMD Radeon R7 360 graphics card features 2 GB of GDDR5 memory. This dedicated video memory determines the maximum size of the artificial intelligence models you can run directly on the hardware. Because 2 GB is a highly constrained memory limit for modern AI tasks, selecting the correct quantization level is critical to prevent out of memory errors.

Quantization is a compression method that reduces the size of model weights. The quant column indicates the optimal format for balancing model accuracy and memory consumption. For this graphics card, models like Allegro 2.8B and Open-Sora Plan 2.7B require a Q4_K_M quantization to fit within 2 GB of video memory. Smaller models such as SmolLM2 1.7B and Moondream 2 can use a higher quality Q6_K quantization while staying within 1.7 GB and 1.9 GB of memory respectively. The smallest models like TinyLlama 1.1B can run at Q8_0 quantization using only 1.4 GB of video memory.

When a model exceeds the 2 GB physical limit of your graphics card, you must use CPU offload. This technique splits the model layers between your graphics card and your system RAM. For example, running the 3B parameter SmolLM3 or Replit Code v1.5 3B requires 2.2 GB of video memory at Q4_K_M quantization plus an additional 4.2 GB of system RAM. Larger models like SDXL Turbo 3.5B require 2.6 GB of video memory at Q4_K_M and 4.6 GB of system RAM.

CPU offload allows you to run larger architectures but it comes with a severe performance cost. Transferring data between your system RAM and the graphics card over the system bus is much slower than using the dedicated GDDR5 memory directly. This transfer bottleneck significantly reduces the generation speed of models like Stable Diffusion XL which requires 4.1 GB of video memory at FP8 and 6.1 GB of system RAM.

You must also consider the memory cost of context length. Running a text model with a standard 4k context window requires additional video memory to store the active conversation history. If your model already consumes the entire 2 GB of video memory, any active context will spill over into your system RAM. This spillover slows down processing speeds during longer chat sessions.