Best local AI models for NVIDIA Quadro M2000M

The largest models that fit fully

The 30 largest of the 81 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The NVIDIA Quadro M2000M is a mobile workstation graphics card equipped with 4 GB of GDDR5 memory. This dedicated memory size determines which artificial intelligence models can run directly on your graphics hardware. When a model fits entirely within this 4 GB limit, it benefits from the hardware speed of your graphics processor. If a model exceeds this limit, you must use alternative execution strategies.

The quantization column indicates the compression level applied to each model. Quantization reduces the precision of model weights to save memory. For example, a Q4_K_M quantization uses fewer bits per weight than a Q6_K or Q8_0 quantization. This compression allows larger models to fit into the 4 GB memory space of your hardware. A lower quantization number saves more space but can slightly reduce the output quality of the model.

For models that fit completely inside your graphics memory, you can run options like the Lumina-Next or Lumina-Image 2.0 5B model at Q4_K_M quantization using 3.7 GB of memory. You can also run the CogVideoX 2B or 5B model at Q4_K_M quantization using 3.7 GB of memory. Other compatible options include DeepSeek-VL2 at Q5_K_M quantization using 3.8 GB of memory and DeepFloyd IF at Q5_K_M quantization using 3.7 GB of memory. The Phi-3.5-vision model fits at Q5_K_M quantization using 3.6 GB of memory.

Several 4B models fit well using Q6_K quantization. These include Qwen3 4B, Gemma 3 4B, Gemma 4 E4B, MiniCPM 3 4B, Danube 3 4B, and Fish Speech 1.5 or OpenAudio S1. Each of these options uses 3.9 GB of memory. The Phi-4-mini-instruct and Phi-3.5 Mini models use 3.7 GB of memory at Q6_K quantization. OmniGen or OmniGen2 also uses 3.7 GB of memory at Q6_K quantization. You can also run SD Cascade (Würstchen v3) using 3.5 GB of memory or SDXL Turbo and SDXL Lightning using 3.4 GB of memory.

When a model is too large for the 4 GB graphics memory, you can offload parts of it to your system RAM. This offload process requires a system with 32 GB of system RAM. Offloading allows you to run larger models but reduces processing speed because system RAM is slower than graphics memory. For example, Mistral 7B at Q4_K_M quantization needs 5.7 GB of graphics memory and 7.7 GB of system RAM. The Qwen2.5 0.5B / 1.5B / 3B / 7B model at Q4_K_M quantization needs 5.1 GB of graphics memory and 7.1 GB of system RAM.

Other offload options include OLMo 2 1B / 7B, Falcon 3 1B / 3B / 7B, Command R7B, and OpenHermes 2.5. Each of these 7B models needs 5.1 GB of graphics memory and 7.1 GB of system RAM at Q4_K_M quantization. You must also consider the context limit of these models. Running these models with a standard 4k context window increases memory consumption. If you experience out of memory errors, you must lower the active context length to keep the memory usage within your hardware limits.