Best local AI models for AMD FirePro W7100

The largest models that fit fully

The 30 largest of the 123 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The AMD FirePro W7100 is equipped with 8 GB GDDR5 memory. This onboard memory size dictates the maximum size of the artificial intelligence models you can run directly on the graphics card. When a model fits entirely within this 8 GB limit, it runs at the maximum speed supported by your hardware. If a model exceeds this limit, you must use system memory to run it.

The quantization column indicates the compression level applied to each model. Quantization reduces the size of a model so it can fit into smaller memory spaces. For example, the Mochi 1 10B model fits into 7.3 GB of memory using the Q4_K_M quantization. Models like Gemma 2 9B require exactly 8 GB of memory at the Q4_K_M quantization, which fully utilizes the onboard memory of your card.

Many popular models can run comfortably within the 8 GB limit of your card. The Nemotron Nano 9B, GLM-4 9B, Yi-Coder 9B, GLM-4-9B-Chat, and GLM-4V-9B models all use 7.7 GB of memory at the Q5_K_M quantization. The Chroma 8.9B model uses 7.6 GB at the same quantization level. Additionally, Llama 3.1 8B fits into 7.4 GB using the Q5_K_M quantization.

Highly optimized 8B models can run at the Q6_K quantization level. Granite 3.3 8B, Ministral 8B, InternLM 3 8B, OpenCoder 8B, Seed-Coder 8B, MiniCPM-V 2.6, Idefics 3 8B, Fuyu-8B, Emu3, and Stable Diffusion 3.5 Large all use 7.9 GB of memory. The EXAONE 3.5 7.8B model uses 7.7 GB at Q6_K. Standard 7B models like Mistral 7B use 7.4 GB, while Qwen2.5 7B, OLMo 2 7B, Falcon 3 7B, Command R7B, OpenHermes 2.5, Zephyr 7B Beta, OpenChat 3.5, Starling LM 7B, and Codestral Mamba 7B use 6.9 GB at Q6_K.

When a model is too large for the 8 GB graphics memory, you can offload parts of it to your system RAM. This offloading process allows you to run larger models but reduces processing speed. For example, running FLUX.1 dev at FP8 requires 14.4 GB of memory, which uses your graphics card and 16.4 GB of system RAM. Gemma 3 12B, Gemma 4 12B, Mistral NeMo 12B, Pixtral 12B, FLUX.1 schnell, FLUX.1 Kontext dev, and FLUX.1 Krea dev require 8.8 GB of memory at Q4_K_M, which uses 10.8 GB of system RAM. Open-Sora 2.0 requires 10.1 GB of system RAM, and Vicuna 13B requires 11.5 GB of system RAM.

You must consider the memory cost of the context window when running these models. The memory figures listed are calculated using a standard 4k context window. If you increase the context window to process longer documents or larger chat histories, the model will require more memory. This extra memory usage might force a model that normally fits on your card to offload to system RAM.