Best local AI models for AMD Pro WX 4100

The largest models that fit fully

The 30 largest of the 81 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The AMD Radeon Pro WX 4100 is a low profile workstation graphics card. It features 4 GB of GDDR5 frame buffer memory. This memory size limits the size of the artificial intelligence models you can run entirely on the hardware. To run local models successfully on this card, you must select model files that fit within this 4 GB limit while leaving a small amount of space for your display and operating system.

The quantization column indicates the compression level of the model weights. Quantization reduces the precision of the model parameters to save memory. For example, a Q4_K_M quantization uses four bit precision, while a Q8_0 quantization uses eight bit precision. Higher quantization levels like Q8_0 preserve more model accuracy but require more memory. Lower quantization levels like Q4_K_M or Q5_K_M allow larger models to fit into the 4 GB memory space.

Several models can run entirely inside the graphics memory of this card. The largest fitting models include Lumina-Next or Lumina-Image 2.0 at 5B parameters using the Q4_K_M quantization which takes 3.7 GB. CogVideoX 2B or 5B also fits at 5B parameters using Q4_K_M quantization for 3.7 GB. DeepSeek-VL2 at 4.5B parameters fits using Q5_K_M quantization at 3.8 GB. DeepFloyd IF at 4.3B parameters fits using Q5_K_M quantization at 3.7 GB. Phi-3.5-vision at 4.2B parameters fits using Q5_K_M quantization at 3.6 GB.

You can also run 4B parameter models like Qwen3 4B, Gemma 3 4B, Gemma 4 E4B, MiniCPM 3 4B, Danube 3 4B, and Fish Speech 1.5 or OpenAudio S1. These models use Q6_K quantization and require 3.9 GB. Phi-4-mini-instruct, Phi-3.5 Mini, and OmniGen or OmniGen2 at 3.8B parameters use Q6_K quantization and require 3.7 GB. SD Cascade (Würstchen v3) at 3.6B parameters uses Q6_K quantization and requires 3.5 GB. SDXL Turbo, SDXL Lightning, and ACE-Step at 3.5B parameters use Q6_K quantization and require 3.4 GB. MusicGen small/medium/large at 3.3B parameters uses Q6_K quantization and requires 3.2 GB.

Smaller models can run at higher precision levels. SmolLM3 3B, Replit Code v1.5 3B, Kandinsky 3.1, Voxtral Mini or Small, Orpheus TTS, and Higgs Audio v2 at 3B parameters use Q8_0 quantization and require 3.8 GB. Allegro at 2.8B parameters uses Q8_0 quantization and requires 3.6 GB. Open-Sora Plan at 2.7B parameters uses Q8_0 quantization and requires 3.4 GB. LFM2 1.2B or 2.6B and Playground v2.5 at 2.6B parameters use Q8_0 quantization and require 3.3 GB. Stable Diffusion 3.5 Medium at 2.5B parameters uses Q8_0 quantization and requires 3.2 GB.

When a model is too large for the 4 GB graphics memory, you can offload parts of it to your system RAM. This offload process slows down the generation speed because system RAM is much slower than GDDR5 graphics memory. For offloading, we assume a system with 32 GB of system RAM. Stable Diffusion XL at 3.417B parameters needs 4.1 GB at FP8 or optimized settings and requires 6.1 GB of system RAM. Phi-3 Mini at 3.8B parameters needs 4.4 GB at Q4_K_M and requires 6.4 GB of system RAM. Phi-4-multimodal at 5.6B parameters needs 4.1 GB at Q4_K_M and requires 6.1 GB of system RAM. Magicoder-S-DS 6.7B needs 4.9 GB at Q4_K_M and requires 6.9 GB of system RAM.

Larger 7B models also require offloading. Mistral 7B needs 5.7 GB at Q4_K_M and requires 7.7 GB of system RAM. Qwen2.5 0.5B or 1.5B or 3B or 7B, OLMo 2 1B or 7B, Falcon 3 1B or 3B or 7B, Command R7B, and OpenHermes 2.5 all need 5.1 GB at Q4_K_M and require 7.1 GB of system RAM. Please note that these memory requirements are calculated for a standard 4k context window. Running longer context windows will increase memory consumption and may cause the model to exceed your available memory.