Best local AI models for NVIDIA GTX 1650 MAX-Q

The largest models that fit fully

The 30 largest of the 81 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The NVIDIA GTX 1650 MAX-Q graphics card features 4 GB of GDDR6 memory. This dedicated video memory is the primary constraint when running local AI models. To run a model entirely on your graphics hardware, the model files and active memory must fit within this 4 GB limit. If a model exceeds this capacity, your system must use CPU offloading or the model will fail to load.

The quant column indicates the quantization level used to compress the model. Quantization reduces the size of the model weights to save memory. For example, a Q4_K_M quantization uses approximately four bits per weight, while a Q8_0 quantization uses eight bits. Higher quantization levels like Q8_0 preserve more original model quality but require more memory. Lower levels like Q4_K_M allow larger models to fit into your 4 GB limit.

Several capable models fit completely within the 4 GB memory limit of the GTX 1650 MAX-Q. The largest fitting models include Lumina-Next or Lumina-Image 2.0 at 5B using a Q4_K_M quant with 3.7 GB used. CogVideoX 2B or 5B also runs at 5B using a Q4_K_M quant with 3.7 GB used. DeepSeek-VL2 at 4.5B fits with a Q5_K_M quant using 3.8 GB. Other options include DeepFloyd IF at 4.3B using 3.7 GB and Phi-3.5-vision at 4.2B using 3.6 GB, both at Q5_K_M quantization.

For 4B models, you can run Qwen3 4B, Gemma 3 4B, Gemma 4 E4B, MiniCPM 3 4B, Danube 3 4B, and Fish Speech 1.5 or OpenAudio S1. These 4B models use a Q6_K quant and require 3.9 GB of memory. Phi-4-mini-instruct, Phi-3.5 Mini, and OmniGen or OmniGen2 at 3.8B use a Q6_K quant and require 3.7 GB. Image generators like SD Cascade (Würstchen v3) at 3.6B use 3.5 GB, while SDXL Turbo and SDXL Lightning at 3.5B use 3.4 GB, all under Q6_K quantization.

When using a Q8_0 quant, the maximum model size decreases to 3B. SmolLM3 3B, Replit Code v1.5 3B, Kandinsky 3.1, Voxtral Mini or Small, Orpheus TTS, and Higgs Audio v2 all use 3.8 GB of memory. You can also run Allegro at 2.8B using 3.6 GB, Open-Sora Plan at 2.7B using 3.4 GB, LFM2 1.2B or 2.6B at 2.6B using 3.3 GB, Playground v2.5 at 2.6B using 3.3 GB, and Stable Diffusion 3.5 Medium at 2.5B using 3.2 GB.

If you have 32 GB of system RAM, you can use CPU offloading to run larger models. Offloading splits the workload between your GPU and CPU, but it significantly slows down processing speeds. Under this setup, Mistral 7B needs 5.7 GB at Q4_K_M and 7.7 GB of system RAM. Qwen2.5 0.5B or 1.5B or 3B or 7B, OLMo 2 1B or 7B, Falcon 3 1B or 3B or 7B, Command R7B, and OpenHermes 2.5 all need 5.1 GB at Q4_K_M and 7.1 GB of system RAM.

Other offload options include Magicoder-S-DS 6.7B which needs 4.9 GB at Q4_K_M and 6.9 GB of system RAM. Phi-4-multimodal at 5.6B needs 4.1 GB at Q4_K_M and 6.1 GB of system RAM. Stable Diffusion XL at 3.417B needs 4.1 GB at FP8 or optimized settings and 6.1 GB of system RAM. Be aware that context window size affects memory. The listed requirements assume a standard 4k context window, and increasing this context window will require more memory.