Best local AI models for NVIDIA GT 745A

The largest models that fit fully

The 30 largest of the 56 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The NVIDIA GT 745A is an entry level graphics card equipped with 2 GB of DDR3 memory. This dedicated video memory is the primary constraint when running local artificial intelligence models. To run a model entirely on this hardware the model files and active memory must fit within this 2 GB limit. If a model exceeds this capacity it cannot run on the graphics card alone.

Quantization is a method that compresses model files to save space. The quant column indicates the optimal compression level for each model on this hardware. For example the Allegro 2.8B model uses the Q4_K_M quant to fit into 2 GB of video memory. Smaller models like Moondream 2 can use the higher quality Q6_K quant and still fit within 1.9 GB of memory. The Whisper Large v3 model uses the Q8_0 quant which requires 2 GB of memory.

When a model is too large for the 2 GB video memory you must use CPU offload. This process splits the workload between your graphics card and your system RAM. We assume your computer has 32 GB of system RAM for these scenarios. Offloading allows you to run larger models like the 3B SmolLM3 or the 3.5B SDXL Turbo. However offloading comes with a performance cost because transferring data between system RAM and the graphics card is much slower than using dedicated video memory.

The 4k context caveat is an important factor for text models. The memory numbers listed here represent the model at its base state. As you type longer prompts and the model generates longer answers the memory usage increases. Running a model close to the 2 GB limit leaves very little room for context. You may need to use smaller models like the TinyLlama 1.1B which only uses 1.4 GB of memory to ensure you have enough room for longer conversations.

This hardware can run a variety of specialized tools within its limits. For image generation you can run Stable Diffusion 3 Medium at the Q6_K quant using 2 GB of memory. For audio tasks the Parler-TTS model fits using the Q5_K_M quant with 1.9 GB of memory. If you require code generation the SantaCoder 1.1B model fits easily using the Q8_0 quant and requires only 1.4 GB of video memory.