Best local AI models for NVIDIA GT 720M

The largest models that fit fully

The 30 largest of the 56 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The NVIDIA GT 720M is an entry level mobile graphics card equipped with 2 GB DDR3 video memory. This hardware specification determines the maximum size of the artificial intelligence models you can run locally. To load a model entirely on this GPU, the total memory footprint of the model must remain under the 2 GB physical limit of your card.

The quantization column indicates the compression level applied to each model. Quantization reduces the precision of model weights to save space. For example, the Allegro 2.8B model fits in 2 GB of video memory when using the Q4_K_M quantization. Smaller models like SmolLM2 1.7B can run at a higher quality Q6_K quantization while using 1.7 GB of video memory. TinyLlama 1.1B can run at the Q8_0 quantization level using 1.4 GB of video memory.

When a model size exceeds the 2 GB video memory limit, you must use CPU offload. This process splits the workload between your graphics card and your system RAM. We assume your computer has 32 GB of system RAM for these scenarios. Running a model like SmolLM3 3B requires 2.2 GB of video memory at Q4_K_M quantization and an additional 4.2 GB of system RAM.

Other models also rely on CPU offload to function on this hardware. Stable Diffusion XL requires 4.1 GB of video memory at FP8 or optimized settings along with 6.1 GB of system RAM. SDXL Turbo and SDXL Lightning both require 2.6 GB of video memory at Q4_K_M quantization and 4.6 GB of system RAM. This offloading process allows you to run larger models but it reduces processing speed.

You must also consider the 4k context window caveat when running local models. The memory numbers listed here represent the base model size. Generating text or processing data increases memory consumption. Running a model with a full 4k context window requires extra video memory beyond the static file size. You may need to use smaller models or lower quantization levels to avoid running out of memory during long conversations.