Best local AI models for AMD FirePro W4170M

The largest models that fit fully

The 30 largest of the 56 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The AMD FirePro W4170M is a mobile workstation graphics card equipped with 2 GB of GDDR5 video memory. This memory capacity determines the size of the artificial intelligence models you can run locally. To fit within this limit, models must be compressed using quantization. The best quantization column shows the optimal format to balance output quality and memory usage without exceeding your hardware limits.

For fully local execution on the GPU, you can run models up to 2.8B parameters. The Allegro 2.8B model fits using the Q4_K_M quantization, which consumes exactly 2 GB of video memory. Similarly, Open-Sora Plan 2.7B fits at Q4_K_M with 2 GB used. You can also run LFM2 2.6B or Playground v2.5 2.6B at Q4_K_M, both requiring 1.9 GB of video memory.

Slightly smaller models allow for higher precision quantizations. Stable Diffusion 3 Medium 2B, SmolVLM 2B, Pyramid Flow 2B, and Wav2Vec2 / XLS-R 2B all run at the Q6_K quantization while using 2 GB of video memory. If you choose models like StableLM 2 1.6B, Sana 1.6B, Zonos 0.1 1.6B, or Dia 1.6B, you can use the high precision Q8_0 quantization, which utilizes 2 GB of video memory.

When a model is too large for the 2 GB video memory, you must use CPU offload. This process splits the model between your graphics card and your system RAM. For example, running SmolLM3 3B, Replit Code v1.5 3B, Kandinsky 3.1 3B, Voxtral Mini / Small 3B, Orpheus TTS 3B, or Higgs Audio v2 3B at Q4_K_M requires 2.2 GB of video memory and 4.2 GB of system RAM. MusicGen small/medium/large 3.3B requires 2.4 GB of video memory and 4.4 GB of system RAM.

Larger image generation models also rely on CPU offload. Stable Diffusion XL 3.417B requires 4.1 GB of video memory and 6.1 GB of system RAM at FP8 / optimized. SDXL Turbo 3.5B and SDXL Lightning 3.5B both require 2.6 GB of video memory and 4.6 GB of system RAM at Q4_K_M. Offloading allows these models to run, but it reduces processing speed because system RAM is slower than GDDR5 video memory.

You must also consider the context window size when running text models. The memory figures listed here are calculated for a standard 4k context window. If you increase the context window to process longer documents, the system will require more memory. This extra memory demand may force you to use a smaller model or rely more heavily on slower CPU offload.