Best local AI models for AMD FirePro W5170M

The largest models that fit fully

The 30 largest of the 56 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The AMD FirePro W5170M is a mobile workstation graphics card equipped with 2 GB GDDR5 memory. This dedicated memory size is the primary constraint when running artificial intelligence models locally. To run a model entirely on this hardware, the model files and active memory must fit within this 2 GB limit. If a model exceeds this capacity, it cannot run solely on the graphics processor.

The quantization column indicates the compression level applied to each model. Quantization reduces the precision of model weights to save space. For example, the Allegro 2.8B model fits in the 2 GB limit using the Q4_K_M quantization. Smaller models like SmolLM2 1.7B can use the higher quality Q6_K quantization while using 1.7 GB of memory. The TinyLlama 1.1B model fits easily using the high precision Q8_0 quantization which uses 1.4 GB of memory.

When a model is slightly too large for the 2 GB graphics memory, you can use CPU offloading. This process splits the workload between your graphics card and your system RAM. Assuming your system has 32 GB of system RAM, you can run larger models like the SmolLM3 3B or Kandinsky 3.1. These 3B models need 2.2 GB of graphics memory at Q4_K_M quantization and require an additional 4.2 GB of system RAM to function.

Offloading allows you to run advanced models but it comes with a performance cost. Moving data between the graphics card and system RAM is much slower than keeping everything in the GDDR5 memory. Models like Stable Diffusion XL need 4.1 GB at FP8 and require 6.1 GB of system RAM. While offloading makes these models run on your system, the generation speed will be significantly slower than running fully on the graphics card.

You must also consider the memory cost of context length. The memory figures listed are calculated at a base 4k context window. Running longer conversations or processing larger documents increases memory usage. If you push the context window beyond 4k, the model may exceed the 2 GB limit of your AMD FirePro W5170M and force system offloading even for smaller models.