Best local AI models for NVIDIA GTX 690

The largest models that fit fully

The 30 largest of the 56 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The NVIDIA GTX 690 is a dual GPU graphics card where each GPU has access to 2 GB of GDDR5 memory. When running local AI models, you must fit the model within this 2 GB limit to avoid severe performance drops. The memory size of a model determines if it can run entirely on your graphics hardware. If a model exceeds this limit, your system must use alternative execution methods.

The quantization column shows the compression level used to shrink these models. Quantization reduces the precision of model weights to save space. For example, the 2.8B Allegro model fits in 2 GB of memory using a Q4_K_M quantization. Smaller models like the 2B Stable Diffusion 3 Medium can run at a higher quality Q6_K quantization. Tiny models like the 1.1B TinyLlama run at Q8_0 quantization while using only 1.4 GB of memory.

You can run larger models by offloading layers to your system RAM. This approach requires a system with 32 GB of system RAM. For example, the 3B SmolLM3 needs 2.2 GB of video memory at Q4_K_M quantization and 4.2 GB of system RAM. The 3.417B Stable Diffusion XL needs 4.1 GB of memory at FP8 or optimized settings and 6.1 GB of system RAM. Offloading allows you to run these larger models but it reduces your generation speed.

Many other models can run using this offload method. The 3.5B SDXL Turbo and 3.5B SDXL Lightning both need 2.6 GB of video memory at Q4_K_M quantization and 4.6 GB of system RAM. Audio models like the 3.3B MusicGen small or medium or large need 2.4 GB of video memory at Q4_K_M quantization and 4.4 GB of system RAM. This technique expands your options at the cost of processing time.

You must consider the context window when running text models. Running a model with a standard 4k context window increases memory usage during generation. The memory figures listed here represent the base model size. If you write long prompts or generate long answers, the active memory usage will rise. You may need to select a smaller model or a lower quantization to keep the active context within your 2 GB limit.