Best local AI models for NVIDIA MX350

The largest models that fit fully

The 30 largest of the 56 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The NVIDIA MX350 is an entry level laptop graphics card equipped with 2 GB of GDDR5 memory. This hardware memory limit dictates which artificial intelligence models can run directly on your graphics processor. When a model fits entirely within this video memory, it executes quickly. If a model exceeds this capacity, you must utilize system memory offloading which slows down performance.

To fit models onto this hardware, developers use quantization. The quant column indicates the compression level applied to the model weights. For example, the Allegro 2.8B model runs at its best quant of Q4_K_M, which uses exactly 2 GB of video memory. Similarly, the Canary 1B / Qwen-2.5B model fits at Q4_K_M while using 1.8 GB of video memory. Lower quantization levels like Q4_K_M compress the model more, while higher levels like Q8_0 offer better accuracy but require more space.

Smaller models can run at higher precision levels on this hardware. The SmolLM2 135M / 360M / 1.7B model runs at a Q6_K quant while using 1.7 GB of video memory. Very compact models like TinyLlama 1.1B can run at Q8_0 precision, consuming only 1.4 GB of video memory. This leaves a small safety margin of your graphics memory free for your operating system display outputs.

When you want to run larger models, you must offload part of the workload to your system RAM. Assuming you have 32 GB of system RAM, you can run the SmolLM3 3B model. This model needs 2.2 GB of video memory at Q4_K_M and requires an additional 4.2 GB of system RAM. The SDXL Turbo 3.5B model also runs via offloading, requiring 2.6 GB of video memory at Q4_K_M and 4.6 GB of system RAM.

Offloading comes with a performance cost. System RAM is much slower than the GDDR5 memory on your graphics card. Transferring data between your system memory and your graphics card creates a processing bottleneck. While offloading allows you to run larger tools like Stable Diffusion XL, your generation speeds will be significantly lower than when running models that fit entirely within your video memory.

You must also consider the memory cost of context length. Running text models with a long context window of 4k tokens or more requires extra memory to store the active conversation history. This active memory overhead is not included in the static model file sizes. If you generate long passages or upload large documents, you may run out of memory even with models that theoretically fit within your 2 GB limit.