Best local AI models for NVIDIA GTX 780M

The largest models that fit fully

The 30 largest of the 81 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The NVIDIA GTX 780M is a mobile graphics card equipped with 4 GB GDDR5 memory. This physical memory size determines the maximum size of the AI models you can run entirely on your GPU. To run a model without slowdowns, the model weights and the active working memory must fit within this 4 GB limit.

The quant column indicates the quantization level used to compress the model. Quantization reduces the precision of model weights to save memory. For example, Q4_K_M uses approximately four bits per weight, while Q8_0 uses eight bits. Lower quantization levels allow larger models to fit into your VRAM, but they may slightly reduce output quality.

With 4 GB of VRAM, you can run several capable models locally. Lumina-Next or Lumina-Image 2.0 at 5B size fits using the Q4_K_M quant, which consumes 3.7 GB of memory. CogVideoX 2B or 5B at 5B size also uses 3.7 GB with the Q4_K_M quant. For vision tasks, DeepSeek-VL2 at 4.5B size fits with a Q5_K_M quant, using 3.8 GB of VRAM.

Other options include Phi-3.5-vision at 4.2B size using 3.6 GB under Q5_K_M. Text models like Qwen3 4B, Gemma 3 4B, Gemma 4 E4B, MiniCPM 3 4B, and Danube 3 4B all fit using the Q6_K quant, consuming 3.9 GB. Audio models like Fish Speech 1.5 or OpenAudio S1 at 4B size also run at Q6_K using 3.9 GB. If you prefer higher precision, SmolLM3 3B and Replit Code v1.5 3B fit using the Q8_0 quant, which uses 3.8 GB.

When a model exceeds 4 GB, you must use CPU offload. This technique splits the model between your GPU and your system RAM. CPU offload allows you to run Mistral 7B at Q4_K_M, which needs 5.7 GB of VRAM and 7.7 GB of system RAM. Similarly, Qwen2.5 7B, OLMo 2 7B, Falcon 3 7B, Command R7B, and OpenHermes 2.5 at 7B size need 5.1 GB at Q4_K_M and 7.1 GB of system RAM. Offloading prevents out of memory errors but significantly reduces processing speed.

Be aware of the context window limit when running models close to your VRAM capacity. The listed memory usage figures assume a standard 4k context window. Generating longer responses or processing large input texts requires extra memory. If you push past the 4k context limit, the model may exceed the 4 GB VRAM of your GTX 780M and trigger slow CPU offloading.