Best local AI models for NVIDIA GTX 770

The largest models that fit fully

The 30 largest of the 56 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The NVIDIA GTX 770 is equipped with 2 GB GDDR5 video memory. This hardware limit dictates the size of the artificial intelligence models you can run locally. To fit inside this memory space, models must undergo quantization. Quantization reduces the precision of model weights to make the file size smaller. The quant column shows the optimal format for each model to run efficiently on this specific card.

For models that fit entirely within your video memory, you can expect the fastest processing speeds. The largest fully fitting models include Allegro 2.8B and Open-Sora Plan 2.7B using the Q4_K_M quant, which use exactly 2 GB of memory. Other options like LFM2 2.6B and Playground v2.5 use 1.9 GB of memory at the same Q4_K_M quantization level. Stable Diffusion 3.5 Medium and Canary 2.5B require 1.8 GB of video memory.

Highly quantized models under 2B parameters can also run directly on the card. SeamlessM4T v2 2.3B, Parler-TTS 2.2B, and Kimi K3 DSpark 2.2B utilize Q5_K_M quantization. For better precision, you can run SmolVLM 2B, Stable Diffusion 3 Medium 2B, Pyramid Flow 2B, or Wav2Vec2 2B at the Q6_K quantization level. Smaller models like SmolLM2 1.7B use 1.7 GB of memory, while TinyLlama 1.1B uses 1.4 GB at the Q8_0 level.

If a model exceeds the 2 GB video memory limit, you must use CPU offload. This technique stores part of the model in your system RAM. For example, running SmolLM3 3B, Replit Code v1.5 3B, Kandinsky 3.1, Voxtral Mini, Orpheus TTS, or Higgs Audio v2 requires 2.2 GB of video memory at Q4_K_M and 4.2 GB of system RAM. MusicGen requires 2.4 GB of video memory and 4.4 GB of system RAM.

Larger image generation models also rely heavily on CPU offload. Stable Diffusion XL requires 4.1 GB of video memory at FP8 and 6.1 GB of system RAM. SDXL Turbo 3.5B and SDXL Lightning 3.5B both require 2.6 GB of video memory at Q4_K_M and 4.6 GB of system RAM. Offloading models to system RAM prevents out of memory errors but significantly reduces generation speeds.

When running text models, remember that context length consumes video memory. The memory figures listed here assume a standard 4k context window. If you increase the context window to process longer documents, the memory usage will rise. This extra memory consumption might force you to use a smaller model or rely on CPU offload.