Best local AI models for AMD R7 240

The largest models that fit fully

The 30 largest of the 56 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The AMD R7 240 is an entry level graphics card equipped with 2 GB of GDDR5 memory. This dedicated memory size dictates the maximum size of the artificial intelligence models you can run entirely on the hardware. To fit within this 2 GB limit, models must undergo quantization. Quantization reduces the precision of model weights to save space. The quant column indicates the best format that balances model accuracy with the strict memory constraints of your hardware.

For local execution without system memory assistance, the largest fitting models include Allegro 2.8B and Open-Sora Plan 2.7B. Both models utilize the Q4_K_M quantization level and consume exactly 2 GB of video memory. Other options like LFM2 2.6B and Playground v2.5 2.6B also run efficiently using Q4_K_M quantization while consuming 1.9 GB of video memory. These configurations ensure the model stays within the physical limits of the graphics card.

Slightly smaller models can utilize higher precision quantization levels for better output quality. Stable Diffusion 3 Medium 2B and SmolVLM 2B run at the Q6_K quantization level using 2 GB of video memory. If you require even higher precision, models like StableLM 2 1.6B and Sana 1.6B can run at the Q8_0 quantization level. These Q8_0 models use 2 GB of video memory and offer minimal quality loss compared to their uncompressed versions.

When a model exceeds the 2 GB video memory limit, you must use CPU offloading. This technique splits the model between your graphics card and your system RAM. For example, running SmolLM3 3B or Kandinsky 3.1 requires 2.2 GB of video memory at Q4_K_M quantization and 4.2 GB of system RAM. Larger models like Stable Diffusion XL require 4.1 GB of video memory at FP8 or optimized settings along with 6.1 GB of system RAM.

CPU offloading allows you to run larger models but it comes with a performance cost. Transferring data between the AMD R7 240 and system RAM is much slower than using dedicated video memory. This transfer bottleneck significantly reduces the generation speed. Additionally, running text models with a standard 4k context window increases memory consumption during generation, which may require further offloading or smaller batch sizes.