Best local AI models for NVIDIA GT 720

The largest models that fit fully

The 30 largest of the 56 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The NVIDIA GT 720 is an entry level graphics card equipped with 2 GB of DDR3 memory. This hardware configuration places strict limits on the size of local AI models you can run directly on the graphics processing unit. To run any model successfully without system crashes, the total memory used by the model must remain under this 2 GB threshold. This page helps you identify which models fit within these hardware boundaries.

The model size determines how much memory is required for execution. The quantization column shows the best compression format that allows the model to fit into your hardware. For example, the Allegro 2.8B model fits exactly at the Q4_K_M quantization level which uses 2 GB of memory. Smaller models like the TinyLlama 1.1B can run at the higher quality Q8_0 quantization level while using only 1.4 GB of memory.

When a model exceeds the 2 GB limit of your graphics card, you must use CPU offload. This process splits the model layers between your graphics card memory and your system RAM. We assume your computer has 32 GB of system RAM for these calculations. Offloading allows you to run larger models but it comes with a significant performance cost. Your processing speed will drop because transferring data between system RAM and the graphics card is slow.

Several models require this offload strategy to function. The SmolLM3 3B and Replit Code v1.5 3B models both need 2.2 GB at Q4_K_M quantization which requires 4.2 GB of system RAM. Image generators like Stable Diffusion XL need 4.1 GB at FP8 or optimized settings which requires 6.1 GB of system RAM. SDXL Turbo and SDXL Lightning both require 2.6 GB at Q4_K_M quantization and 4.6 GB of system RAM.

You must also consider the memory cost of context length. Running models with a standard 4k context window requires additional memory beyond the base model size. This extra memory overhead can easily push a model that is right at the 2 GB limit over the threshold. For the best stability on your hardware, you should reduce the active context window or choose smaller models like the SmolLM2 1.7B which uses 1.7 GB of memory at Q6_K quantization.