Best local AI models for NVIDIA GTX 760

The largest models that fit fully

The 30 largest of the 56 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The NVIDIA GTX 760 is an older graphics card equipped with 2 GB GDDR5 memory. This limited video memory dictates which local AI models you can run directly on the hardware. To fit models within this 2 GB limit, you must use quantized versions. Quantization reduces the precision of model weights to make the files smaller and less demanding on your hardware.

The best quant column shows the optimal quantization level for each model. For example, the Allegro 2.8B model fits by using the Q4_K_M quant which uses exactly 2 GB of video memory. Other models like the SmolVLM 2B use a Q6_K quant to fit 2 GB of video memory. Smaller models like TinyLlama 1.1B can run at Q8_0 quant while using only 1.4 GB of video memory.

When a model exceeds the 2 GB video memory of the NVIDIA GTX 760, you must use CPU offload. This process splits the model weights between your graphics card and your system RAM. You will need a system with 32 GB system RAM to handle these split workloads. Offloading allows you to run larger models but it reduces processing speed because system RAM is much slower than GDDR5 video memory.

For instance, running the SmolLM3 3B model requires 2.2 GB of video memory at Q4_K_M quant and 4.2 GB of system RAM. The Stable Diffusion XL model at 3.417B parameters needs 4.1 GB at FP8 optimized settings and 6.1 GB of system RAM. Using CPU offload is the only way to run these larger architectures on this specific hardware.

You must also consider the 4k context caveat when running models on this card. Running a model at its maximum context window increases memory usage significantly. The memory figures listed here represent the base model requirements. If you generate long text or process large prompts, the system may run out of memory.