Best local AI models for NVIDIA Quadro K620M

The largest models that fit fully

The 30 largest of the 56 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The NVIDIA Quadro K620M is an entry level mobile workstation graphics card. It features 2 GB of DDR3 video memory. This memory size is the main limiting factor when running artificial intelligence models locally. Because the onboard memory is small, you must select highly optimized models and specific quantization levels to avoid running out of video memory.

Quantization is a compression method that reduces the size of a model. The quant column shows the best format for each model to fit your hardware. For example, the Allegro 2.8B model fits using the Q4_K_M quantization which uses exactly 2 GB of video memory. Other models like the SmolVLM 2B or Stable Diffusion 3 Medium use the Q6_K quantization to fit within the same 2 GB limit. Smaller models like TinyLlama 1.1B can run at the higher quality Q8_0 quantization while using only 1.4 GB of video memory.

When a model exceeds the 2 GB video memory limit, you must use CPU offload. This technique splits the model between your graphics card and your system memory. We assume your computer has 32 GB of system RAM for these calculations. Offloading allows you to run larger models like the SmolLM3 3B or Kandinsky 3.1. These models need 2.2 GB of video memory at Q4_K_M quantization and require 4.2 GB of system RAM to handle the overflow.

Using CPU offload comes with a performance cost. System RAM is much slower than video memory, especially when paired with the DDR3 memory on this graphics card. Running models like Stable Diffusion XL or SDXL Turbo with offloading will result in much slower generation speeds. Stable Diffusion XL requires 4.1 GB of video memory at FP8 or optimized settings and needs 6.1 GB of system RAM to function.

You must also consider the context window when running text models. The memory usage figures are calculated at a base 4k context window. If you increase the context length to process longer documents, the memory usage will rise quickly. This extra memory demand can cause a model that normally fits to exceed the 2 GB limit and fail or slow down.