Best local AI models for NVIDIA GT 1010

The largest models that fit fully

The 30 largest of the 56 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The NVIDIA GT 1010 is an entry level graphics card equipped with 2 GB of GDDR5 video memory. This hardware memory limit determines which local AI models you can run directly on the graphics processing unit. To run a model entirely on this card the model files and runtime memory must fit within this 2 GB boundary.

The quantization column shows the best compression format for each model. Quantization reduces the precision of model weights to save space. For example a Q4_K_M quantization allows larger models like Allegro 2.8B or Open-Sora Plan 2.7B to fit into 2 GB of video memory. Smaller models like TinyLlama 1.1B can use higher precision Q8_0 quantization because they require only 1.4 GB of space.

When a model exceeds the 2 GB video memory limit you must use CPU offload. This process splits the model between your graphics card and your system RAM. If you have a system with 32 GB of system RAM you can run larger models by offloading the extra weight files. This method allows you to run models that would otherwise fail to load on your hardware.

Offloading comes with a performance cost. System RAM is much slower than GDDR5 video memory. While offloading allows you to run a 3.5B model like SDXL Turbo or SDXL Lightning it will generate outputs much slower than models that fit entirely on the graphics card. You will need 2.6 GB of video memory and 4.6 GB of system RAM for these specific models.

Memory calculations for text models assume a standard 4k context window. Generating longer responses or inputting large documents increases memory usage. If you exceed the context limit the system will run out of memory and crash. You must monitor your context size when running models close to the 2 GB limit.