Best local AI models for NVIDIA NVS 5400M

The largest models that fit fully

The 30 largest of the 56 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The NVIDIA NVS 5400M is an entry level mobile graphics card equipped with 2 GB of GDDR3 memory. This memory size is the absolute limit for loading artificial intelligence models directly onto the hardware. Because the card has 2 GB of video memory, any model you run entirely on the GPU must fit within this footprint. This limit includes the model weights and the active working memory required during generation.

To fit models onto this hardware, you must use quantized versions. The quantization column shows the specific compression level needed to run each model. For example, the Allegro 2.8B model and the Open-Sora Plan 2.7B model both require a Q4_K_M quantization to fit into 2 GB of used memory. Smaller models like the SmolLM2 1.7B can run at a higher quality Q6_K quantization while using 1.7 GB. TinyLlama 1.1B and SantaCoder 1.1B can run at the highest Q8_0 quantization using 1.4 GB of video memory.

When a model exceeds the 2 GB limit of your video card, you must use CPU offloading. This process splits the model between your graphics card and your system RAM. We assume your computer has 32 GB of system RAM for these scenarios. Offloading allows you to run larger models, but it comes with a massive speed penalty. Your system must constantly transfer data between the slow system memory and the graphics card.

Several models are viable with CPU offloading. The SmolLM3 3B, Replit Code v1.5 3B, Kandinsky 3.1, Voxtral Mini, Orpheus TTS, and Higgs Audio v2 all need 2.2 GB of video memory at Q4_K_M quantization. They also require 4.2 GB of system RAM to function. MusicGen small needs 2.4 GB of video memory at Q4_K_M and 4.4 GB of system RAM. The Stable Diffusion XL model requires 4.1 GB of video memory at FP8 and 6.1 GB of system RAM.

You must also consider the context window size when running these models. The memory figures listed are calculated using a basic 4k context window. If you increase the context length to process longer documents or conversations, the memory usage will rise quickly. This extra memory demand can easily push a model over the 2 GB limit of your NVS 5400M and force slow CPU offloading.