Best local AI models for NVIDIA GT 730

The largest models that fit fully

The 30 largest of the 56 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The NVIDIA GT 730 graphics card features 2 GB of DDR3 memory. This hardware configuration limits the size of the artificial intelligence models you can run entirely on the graphics processor. To load a model successfully into the onboard memory, the total size of the model must remain under this 2 GB limit. This page lists the largest compatible models and explains how to configure them for this specific hardware.

The quantization column shows the best compression level for each model. Quantization reduces the size of a model by using fewer bits to represent its weights. For example, the Allegro 2.8B model fits into 2 GB of memory when compressed to the Q4_K_M quantization level. Other models like SmolVLM 2B can run at the higher Q6_K quantization level while still using exactly 2 GB of memory. Smaller models like TinyLlama 1.1B can use the Q8_0 quantization level which requires only 1.4 GB of memory.

When a model exceeds the 2 GB onboard memory limit, you must use CPU offload. This technique splits the workload between your graphics card and your system memory. To use CPU offload, your computer should have 32 GB of system RAM. For example, running the SmolLM3 3B model at Q4_K_M requires 2.2 GB of graphics memory and 4.2 GB of system RAM. Similarly, the Stable Diffusion XL model requires 4.1 GB of graphics memory and 6.1 GB of system RAM when running at the FP8 optimized level.

Using CPU offload comes with a performance cost. Moving data between the DDR3 graphics memory and the system RAM slows down the processing speed. Models like MusicGen 3.3B require 2.4 GB of graphics memory and 4.4 GB of system RAM, which will run slower than models that fit entirely within the 2 GB onboard memory. For the fastest generation speeds, choose models that fit completely inside the local graphics memory.

You must also consider the 4k context window caveat when running language models. As your conversation history grows, the model requires more memory to track the context. A model that fits within the 2 GB limit at the start of a chat might exceed the memory limit as the context approaches 4000 tokens. To avoid running out of memory during long sessions, you should select smaller models like the SmolLM2 1.7B or the TinyLlama 1.1B.