Best local AI models for NVIDIA Quadro K2200M

The largest models that fit fully

The 30 largest of the 56 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The NVIDIA Quadro K2200M is a mobile workstation graphics card equipped with 2 GB of GDDR5 memory. This dedicated video memory determines the maximum size of the artificial intelligence models you can run entirely on the hardware. When running models locally, the weights must fit inside this physical memory space to ensure acceptable processing speeds.

The quantization column indicates the compression level used to shrink these models. Standard models are often too large for a 2 GB frame buffer. Using quantized formats like Q4_K_M, Q5_K_M, Q6_K, or Q8_0 reduces the precision of the model weights. This compression allows larger architectures to fit into the limited GDDR5 memory without a massive loss in output quality.

For models that fit entirely within the 2 GB limit, you can run options like the Allegro 2.8B at Q4_K_M or the Open-Sora Plan 2.7B at Q4_K_M. Other compatible options include the LFM2 2.6B and Playground v2.5, which both use 1.9 GB of memory at Q4_K_M. You can also run Stable Diffusion 3.5 Medium or Canary Qwen-2.5B at Q4_K_M using 1.8 GB of video memory.

Smaller models can run at higher precision levels on this hardware. The SmolVLM 2B, Stable Diffusion 3 Medium, Pyramid Flow, and Wav2Vec2 XLS-R all utilize 2 GB of memory at Q6_K quantization. If you select Whisper Large v3, it runs at Q8_0 quantization and uses the full 2 GB of video memory. TinyLlama 1.1B and SantaCoder 1.1B run at Q8_0 quantization while using only 1.4 GB of video memory.

When a model exceeds the 2 GB video memory limit, you must use CPU offloading. This process splits the model layers between your graphics card and your system RAM. For example, running SmolLM3 3B, Replit Code v1.5 3B, Kandinsky 3.1, Voxtral Mini, Orpheus TTS, or Higgs Audio v2 requires 2.2 GB of video memory at Q4_K_M and 4.2 GB of system RAM. MusicGen requires 2.4 GB of video memory and 4.4 GB of system RAM. SDXL Turbo requires 2.6 GB of video memory and 4.6 GB of system RAM.

CPU offloading allows you to run larger models like Stable Diffusion XL, which needs 4.1 GB at FP8 and 6.1 GB of system RAM. However, offloading introduces a speed penalty. Moving data between the system RAM and the graphics card over the system bus is much slower than keeping the data entirely inside the GDDR5 memory. You must also monitor your context window, as expanding the context to 4k tokens increases memory consumption and can cause out of memory errors.