Best local AI models for NVIDIA Quadro M600M

The largest models that fit fully

The 30 largest of the 56 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The NVIDIA Quadro M600M is an entry level mobile workstation graphics card. It features 2 GB of GDDR5 dedicated video memory. This memory size is the strict hardware limit for running local AI models entirely on the GPU. To run a model successfully on this hardware, the model files must fit within this 2 GB boundary. If a model exceeds this limit, it will require system memory offloading which reduces processing speed.

The quantization column indicates the compression level applied to each model. Quantization reduces the size of the model weights to save memory. For example, the Allegro 2.8B model fits in 2 GB of video memory when using the Q4_K_M quantization. Other models like SmolVLM 2B or Stable Diffusion 3 Medium use the Q6_K quantization to fit exactly in 2 GB. Smaller models like TinyLlama 1.1B or SantaCoder 1.1B can run at the higher quality Q8_0 quantization while using only 1.4 GB of video memory.

When a model size exceeds the 2 GB physical limit of the GPU, you must use CPU offload. This process shares the workload with your system RAM. For instance, running the SmolLM3 3B or Kandinsky 3.1 models at Q4_K_M requires 2.2 GB of video memory and 4.2 GB of system RAM. Running Stable Diffusion XL requires 4.1 GB at FP8 or optimized settings along with 6.1 GB of system RAM. Offloading allows you to run larger models but it lowers generation speed because system RAM is slower than GDDR5 video memory.

You must also consider the context window when running text models. The memory figures listed are calculated for a standard 4k context window. If you increase the context window to process longer documents, the memory usage will grow. This extra memory demand can push a fitting model over the 2 GB limit of your graphics card. You may need to use a smaller model or a lower quantization level to maintain a larger context window.

This hardware can run a variety of specialized models within its limits. For audio and speech tasks, you can run Whisper Large v3 at Q8_0 using 2 GB of video memory or Parler-TTS at Q5_K_M using 1.9 GB. For image generation, Stable Diffusion 3.5 Medium fits at Q4_K_M using 1.8 GB of video memory. These options allow you to deploy diverse local AI capabilities on your laptop workstation.