Best local AI models for NVIDIA GT 745M

The largest models that fit fully

The 30 largest of the 56 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The NVIDIA GT 745M is an older mobile graphics card equipped with 2 GB of DDR3 video memory. This physical memory limit dictates the size of the artificial intelligence models you can run locally. To load a model entirely on this GPU, the total memory footprint of the model must remain under the 2 GB threshold. This page lists the largest compatible models and their optimal configurations for this hardware.

The quantization column indicates the compression level applied to each model. Quantization reduces the precision of model weights to save memory. For example, the Allegro 2.8B model fits within 2 GB of video memory when using the Q4_K_M quantization. Other models like the Stable Diffusion 3 Medium 2B model require a Q6_K quantization to fit exactly 2 GB of video memory. Smaller models like the TinyLlama 1.1B can run at the higher precision Q8_0 quantization while using only 1.4 GB of video memory.

When a model exceeds the 2 GB video memory limit, you must use CPU offload. This process splits the model layers between your GPU video memory and your system RAM. For example, running the SmolLM3 3B model requires 2.2 GB of video memory at Q4_K_M quantization and an additional 4.2 GB of system RAM. This configuration assumes your computer has 32 GB of system RAM available. CPU offload allows you to run larger models like the Stable Diffusion XL 3.417B model, but it will significantly reduce processing speeds.

Running text models on this hardware comes with a strict context window caveat. The memory usage figures listed here are calculated for a standard 4k context window. If you increase the context length to process longer documents or chat histories, the memory usage will rise quickly. This extra memory demand can easily exceed the 2 GB limit of your GPU and force the system to slow down.

You can choose from several model types depending on your needs. For image generation, you can run the Stable Diffusion 3.5 Medium 2.5B model at Q4_K_M quantization using 1.8 GB of video memory. For audio tasks, the Whisper Large v3 1.55B model fits into 2 GB of video memory using the Q8_0 quantization. These configurations ensure you get the best possible performance out of your hardware.