Best local AI models for AMD RX 7700S

The largest models that fit fully

The 30 largest of the 123 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The AMD Radeon RX 7700S is a mobile graphics processor equipped with 8 GB of GDDR6 dedicated video memory. This hardware limit dictates the maximum size of the artificial intelligence models you can run entirely on the graphics card. To run a model smoothly without performance drops, the model weights and the active working memory must fit within this 8 GB boundary.

The quantization column indicates the compression level applied to each model. Quantization reduces the precision of the model weights to save space. For example, the Q4_K_M and Q5_K_M quants compress weights to approximately four and five bits. Higher quants like Q6_K offer better accuracy but require more memory. On this hardware, Gemma 2 9B fits at Q4_K_M using exactly 8 GB, while Llama 3.1 8B fits at Q5_K_M using 7.4 GB.

When a model exceeds the 8 GB video memory limit, you must use CPU offloading. This process splits the model layers between your graphics card and your system RAM. We assume your system has 32 GB of system RAM for these calculations. For instance, running FLUX.1 dev at FP8 requires 14.4 GB of memory, which uses all your video memory and needs an additional 16.4 GB of system RAM.

CPU offloading allows you to run larger models like Vicuna 13B or Mistral NeMo 12B, but it comes with a speed penalty. Transferring data between the system RAM and the graphics card over the system bus is much slower than using dedicated GDDR6 memory. Models like Gemma 3 12B and Gemma 4 12B require 8.8 GB of memory at Q4_K_M, which forces 10.8 GB of data into your system RAM and slows down generation speeds.

You must also consider the memory required for the context window. The memory figures listed here are calculated using a standard 4k context window. If you increase the context length to process longer documents or conversations, the system will require more memory. This extra demand can push a model that normally fits, such as Mochi 1 10B at 7.3 GB, over the 8 GB limit.