Best local AI models for NVIDIA Quadro M1000M

The largest models that fit fully

The 30 largest of the 56 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The NVIDIA Quadro M1000M is an entry level professional graphics card with 2 GB of GDDR5 memory. This dedicated memory size determines which artificial intelligence models can run entirely on your graphics hardware. When a model fits completely within this 2 GB limit, it processes data at the maximum speed supported by the hardware. If a model exceeds this limit, you must use alternative execution strategies.

To fit larger models into the limited memory, developers use quantization. The quant column shows the specific compression level required to run each model. For example, the 2.8B Allegro model fits inside 2 GB of memory when using the Q4_K_M quantization level. Smaller models like the 1.6B Dia can run at the higher quality Q8_0 quantization level while still staying within the 2 GB hardware limit.

When a model size exceeds the onboard graphics memory, you can offload parts of the workload to your system RAM. This approach assumes your computer has 32 GB of system RAM. For instance, running the 3B SmolLM3 requires 2.2 GB of graphics memory at Q4_K_M quantization and needs an additional 4.2 GB of system RAM. Offloading allows you to run larger models like the 3.5B SDXL Turbo, but the transfer of data between system RAM and graphics memory reduces processing speeds.

Your graphics card must also reserve memory space for processing user inputs. The memory figures listed for these models do not account for long conversation histories. If you run a text model with a 4k context window, the active memory usage will increase beyond the base model size. You should choose smaller models like the 1.1B TinyLlama at Q8_0 to ensure you have enough remaining memory for active processing.

For audio and speech tasks, you can run specialized models within the hardware limits. The 1.55B Whisper Large v3 fits into 2 GB of memory using the Q8_0 quantization level. You can also run the 1.5B AudioGen or the 1.5B AudioLDM 2 with Q8_0 quantization, which both use 1.9 GB of graphics memory. These configurations allow you to perform local speech recognition and audio generation without relying on cloud services.