Best local AI models for NVIDIA GT 735M

The largest models that fit fully

The 30 largest of the 56 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The NVIDIA GT 735M is an entry level mobile graphics card equipped with 2 GB of DDR3 video memory. This hardware configuration limits the size of artificial intelligence models you can run entirely on the graphics processor. To execute models locally on this hardware you must select highly optimized versions that fit within the strict 2 GB VRAM ceiling. Running out of video memory causes execution to fail or slow down significantly.

The model size and quantization level determine how much memory is used. Quantization is a compression technique that reduces the precision of model weights to save space. A lower quantization level like Q4_K_M allows larger models to fit into the 2 GB limit but reduces output quality. Higher quantization levels like Q6_K or Q8_0 preserve more original model quality but require smaller model parameter counts to stay within the available hardware limits.

For complete on device execution you can run models up to 2.8B parameters. The Allegro 2.8B model fits using the Q4_K_M quantization which uses exactly 2 GB of video memory. The Open-Sora Plan 2.7B model also fits at Q4_K_M using 2 GB. Smaller models like the SmolLM2 1.7B can run at the higher quality Q6_K quantization using 1.7 GB of memory. TinyLlama 1.1B runs at Q8_0 quantization using 1.4 GB of video memory.

When a model exceeds the 2 GB video memory limit you must use CPU offload. This technique splits the model layers between your graphics card and your system RAM. Assuming your computer has 32 GB of system RAM you can run larger models like the SmolLM3 3B or Kandinsky 3.1. These 3B models require 2.2 GB of video memory at Q4_K_M quantization along with 4.2 GB of system RAM. Offloading allows these models to run but it reduces processing speed because system RAM is much slower than video memory.

Larger generation models can also run using CPU offload. Stable Diffusion XL with 3.417B parameters needs 4.1 GB of memory at FP8 or optimized settings which requires 6.1 GB of system RAM. The SDXL Turbo 3.5B model needs 2.6 GB of video memory at Q4_K_M quantization and 4.6 GB of system RAM. While offloading enables these advanced tasks you must expect longer generation times due to the slow DDR3 memory interface on the graphics card.

You must also consider the context window size when running text models. The memory figures listed are calculated using a standard 4k context window. If you increase the context window to process longer documents or chat histories the memory usage will rise. This extra memory demand can quickly push a model past the 2 GB limit of your NVIDIA GT 735M and force the system to use slow CPU offloading.