Best local AI models for AMD HD 8870M

The largest models that fit fully

The 30 largest of the 56 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The AMD HD 8870M is a mobile graphics card equipped with 2 GB of GDDR5 memory. This dedicated video memory determines the maximum size of the artificial intelligence models you can run directly on the hardware. When running models locally, the entire weights of the neural network must fit within this memory limit to achieve acceptable processing speeds.

To fit larger models into the 2 GB limit, quantization is used to compress the model weights. The quantization column shows the best format that balances model size and output quality. For example, the Allegro 2.8B model fits into 2 GB of video memory using a Q4_K_M quantization. Smaller models like Moondream 2 with 1.9B parameters can run with a higher quality Q6_K quantization while using 1.9 GB of video memory.

When a model exceeds the 2 GB video memory limit, you must use CPU offload. This technique stores part of the model in your 32 GB of system RAM instead of the graphics card memory. Running models this way is slower because data must travel between the system RAM and the graphics processor. For instance, Stable Diffusion XL requires 4.1 GB of memory at FP8, which uses all 2 GB of video memory and needs an additional 6.1 GB of system RAM.

Other offload examples include the MusicGen small/medium/large model at 3.3B parameters. This model needs 2.4 GB of memory at Q4_K_M, which requires 4.4 GB of system RAM to function. Similarly, the SDXL Turbo and SDXL Lightning models at 3.5B parameters need 2.6 GB at Q4_K_M, requiring 4.6 GB of system RAM. These configurations allow you to run larger architectures at the cost of generation speed.

You must also consider the memory cost of context length when running text models. The memory figures listed are calculated for a standard 4k context window. If you increase the context window to process longer documents, the memory usage will rise. This extra memory demand can push a model that normally fits within 2 GB into requiring CPU offload.