Best local AI models for NVIDIA Quadro P2000
5 GB GDDR5. At a 4k context, 85 of the 233 models in our catalog with verified parameter counts fit fully, up to Magicoder-S-DS 6.7B at 6.7B parameters.
Check your own machine against every model →The largest models that fit fully
The 30 largest of the 85 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.
| Model | Parameters | Best quant that fits | Memory used at 4k |
|---|---|---|---|
| Magicoder-S-DS 6.7B | 6.7B | Q4_K_M | 4.9 GB |
| Phi-4-multimodal | 5.6B | Q5_K_M | 4.8 GB |
| Lumina-Next / Lumina-Image 2.0 | 5B | Q6_K | 4.9 GB |
| CogVideoX 2B / 5B | 5B | Q6_K | 4.9 GB |
| DeepSeek-VL2 | 4.5B | Q6_K | 4.4 GB |
| DeepFloyd IF | 4.3B | Q6_K | 4.2 GB |
| Phi-3.5-vision | 4.2B | Q6_K | 4.1 GB |
| Qwen3 4B | 4B | Q6_K | 3.9 GB |
| Gemma 3 4B | 4B | Q6_K | 3.9 GB |
| Gemma 4 E4B | 4B | Q6_K | 3.9 GB |
| MiniCPM 3 4B | 4B | Q6_K | 3.9 GB |
| Danube 3 4B | 4B | Q6_K | 3.9 GB |
| Fish Speech 1.5 / OpenAudio S1 | 4B | Q6_K | 3.9 GB |
| Phi-3 Mini | 3.8B | Q5_K_M | 4.8 GB |
| Phi-4-mini-instruct | 3.8B | Q8_0 | 4.8 GB |
| Phi-3.5 Mini | 3.8B | Q8_0 | 4.8 GB |
| OmniGen / OmniGen2 | 3.8B | Q8_0 | 4.8 GB |
| SD Cascade (Würstchen v3) | 3.6B | Q8_0 | 4.6 GB |
| SDXL Turbo | 3.5B | Q8_0 | 4.5 GB |
| SDXL Lightning | 3.5B | Q8_0 | 4.5 GB |
| ACE-Step | 3.5B | Q8_0 | 4.5 GB |
| Stable Diffusion XL | 3.417B | FP8 / optimized | 4.1 GB |
| MusicGen small/medium/large | 3.3B | Q8_0 | 4.2 GB |
| SmolLM3 3B | 3B | Q8_0 | 3.8 GB |
| Replit Code v1.5 3B | 3B | Q8_0 | 3.8 GB |
| Kandinsky 3.1 | 3B | Q8_0 | 3.8 GB |
| Voxtral Mini / Small | 3B | Q8_0 | 3.8 GB |
| Orpheus TTS | 3B | Q8_0 | 3.8 GB |
| Higgs Audio v2 | 3B | Q8_0 | 3.8 GB |
| Allegro | 2.8B | Q8_0 | 3.6 GB |
Close, but only with CPU offload
These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.
| Model | Parameters | Memory at Q4_K_M | System RAM at 4k |
|---|---|---|---|
| Mistral 7B | 7B | 5.7 GB needed | 7.7 GB |
| Qwen2.5 0.5B / 1.5B / 3B / 7B | 7B | 5.1 GB needed | 7.1 GB |
| OLMo 2 1B / 7B | 7B | 5.1 GB needed | 7.1 GB |
| Falcon 3 1B / 3B / 7B | 7B | 5.1 GB needed | 7.1 GB |
| Command R7B | 7B | 5.1 GB needed | 7.1 GB |
| OpenHermes 2.5 | 7B | 5.1 GB needed | 7.1 GB |
| Zephyr 7B Beta | 7B | 5.1 GB needed | 7.1 GB |
| OpenChat 3.5 | 7B | 5.1 GB needed | 7.1 GB |
| Starling LM 7B | 7B | 5.1 GB needed | 7.1 GB |
| Codestral Mamba 7B | 7B | 5.1 GB needed | 7.1 GB |
How to read this
The NVIDIA Quadro P2000 is equipped with 5 GB of GDDR5 graphics memory. This memory size determines which artificial intelligence models can run directly on your hardware. To fit inside this limit, models must use quantization. Quantization is a compression method that reduces the size of a model while keeping most of its accuracy. The quant column shows the best quality level that fits within your graphics memory.
For local execution without sharing memory, several models fit entirely on the card. Magicoder-S-DS 6.7B fits at the Q4_K_M quantization level using 4.9 GB of memory. Phi-4-multimodal fits at Q5_K_M using 4.8 GB of memory. Lumina-Next or Lumina-Image 2.0 and CogVideoX 2B or 5B both fit at Q6_K using 4.9 GB of memory. DeepSeek-VL2 fits at Q6_K using 4.4 GB of memory. DeepFloyd IF fits at Q6_K using 4.2 GB of memory. Phi-3.5-vision fits at Q6_K using 4.1 GB of memory.
Several 4B models fit at the Q6_K quantization level using 3.9 GB of memory. These models include Qwen3 4B, Gemma 3 4B, Gemma 4 E4B, MiniCPM 3 4B, Danube 3 4B, and Fish Speech 1.5 or OpenAudio S1. For 3.8B models, Phi-3 Mini fits at Q5_K_M using 4.8 GB of memory. Phi-4-mini-instruct, Phi-3.5 Mini, and OmniGen or OmniGen2 fit at Q8_0 using 4.8 GB of memory.
Other media and text models fit within the graphics memory limit. SD Cascade (Würstchen v3) fits at Q8_0 using 4.6 GB of memory. SDXL Turbo and SDXL Lightning fit at Q8_0 using 4.5 GB of memory. ACE-Step fits at Q8_0 using 4.5 GB of memory. Stable Diffusion XL fits at FP8 or optimized using 4.1 GB of memory. MusicGen small, medium, or large fits at Q8_0 using 4.2 GB of memory. SmolLM3 3B, Replit Code v1.5 3B, Kandinsky 3.1, Voxtral Mini or Small, Orpheus TTS, and Higgs Audio v2 fit at Q8_0 using 3.8 GB of memory. Allegro fits at Q8_0 using 3.6 GB of memory.
You can run larger models by offloading parts of them to your system memory. This requires a system with 32 GB of system RAM. Offloading allows you to run 7B models, but it reduces processing speed because system RAM is slower than graphics memory. Mistral 7B requires 5.7 GB at Q4_K_M and needs 7.7 GB of system RAM. Other 7B models require 5.1 GB at Q4_K_M and need 7.1 GB of system RAM. These models include Qwen2.5 0.5B or 1.5B or 3B or 7B, OLMo 2 1B or 7B, Falcon 3 1B or 3B or 7B, Command R7B, OpenHermes 2.5, Zephyr 7B Beta, OpenChat 3.5, Starling LM 7B, and Codestral Mamba 7B.
Memory calculations are based on a standard context window of 4k tokens. Running larger context windows or longer conversations will increase memory usage. If you exceed the 5 GB limit of your graphics card, the model will fail to load or experience severe slowdowns.