Best local AI models for NVIDIA MX570

The largest models that fit fully

The 30 largest of the 56 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The NVIDIA MX570 graphics card features 2 GB of GDDR6 video memory. This hardware limit dictates which local AI models you can run entirely on your GPU. To load a model successfully, its active weights must fit within this 2 GB boundary. Running models locally on this card requires careful selection of model sizes and quantization levels to prevent system slowdowns.

Quantization is a compression method that reduces the memory footprint of AI models. The quant column shows the best balance of size and quality for your hardware. For example, the Allegro 2.8B model fits in 2 GB of video memory using the Q4_K_M quantization. Smaller models like SmolLM2 1.7B can use the higher quality Q6_K quantization while consuming 1.7 GB of video memory. TinyLlama 1.1B runs at the highest Q8_0 quantization level using only 1.4 GB of video memory.

When a model exceeds your 2 GB video memory, you must use CPU offload. This technique splits the workload between your GPU and system RAM. Assuming you have 32 GB of system RAM, you can run larger models with a performance penalty. Under this setup, SmolLM3 3B needs 2.2 GB of video memory at Q4_K_M quantization and 4.2 GB of system RAM. Stable Diffusion XL requires 4.1 GB of video memory at FP8 or optimized settings along with 6.1 GB of system RAM.

Using CPU offload allows access to advanced tools but slows down processing speeds. MusicGen needs 2.4 GB of video memory at Q4_K_M and 4.4 GB of system RAM. SDXL Turbo needs 2.6 GB of video memory at Q4_K_M and 4.6 GB of system RAM. While offloading makes these larger models run, the data transfer between your system RAM and the GPU bottleneck limits your generation speed.

You must also consider the memory cost of context length. Running a text model with a standard 4k context window consumes extra video memory beyond the base model weights. If your selected model uses close to your maximum 2 GB limit, a long conversation history can cause out of memory errors. Keeping your context windows short helps maintain stable performance on this hardware.