Best local AI models for AMD RX 7800M

The largest models that fit fully

The 30 largest of the 147 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The AMD Radeon RX 7800M is a mobile graphics processor equipped with 12 GB of GDDR6 memory. This dedicated video memory determines the maximum size of the artificial intelligence models you can run locally. To run a model entirely on the graphics hardware, the model files and its working memory must fit within this 12 GB limit.

The quantization column shows the optimal precision format for each model. Quantization reduces the size of model weights to save memory. For example, a Q4_K_M quantization allows the DeepSeek-Coder-V2 16B model to fit into 11.7 GB of memory. Models like Gemma 3 12B can run at a higher Q6_K quantization using 11.8 GB of memory. Smaller models like GLM-4 9B can run at Q8_0 quantization using 11.4 GB of memory.

When a model exceeds the 12 GB limit, you must use CPU offload. This process splits the model between your graphics card and your system memory. For example, running FLUX.1 dev at FP8 requires 14.4 GB of memory, which uses 16.4 GB of system RAM. Running Codestral 22B at Q4_K_M requires 16.1 GB of memory, which uses 18.1 GB of system RAM. CPU offload allows you to run larger models like CogVLM2 or Solar Pro, but it reduces generation speed.

Memory calculations in this guide assume a standard 4k context window. The context window is the amount of text the model can process at one time. If you increase the context window beyond 4k tokens, the model will require more memory. This extra memory usage might force you to use a lower quantization or enable CPU offload to prevent out of memory errors.

Your hardware can run diverse model types. For text and reasoning, you can run Phi-4 or Qwen2.5 14B within 11.9 GB and 11.6 GB of memory. For video generation, HunyuanVideo and LTX-Video fit within 11.1 GB of memory using Q5_K_M quantization. Image generation models like FLUX.1 schnell fit within 11.8 GB of memory using Q6_K quantization.