Best local AI models for NVIDIA MX230

The largest models that fit fully

The 30 largest of the 56 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The NVIDIA MX230 features 2 GB of GDDR5 memory. This memory limit determines which AI models run directly on your graphics card. Models that fit within 2 GB provide the fastest performance. You must select the correct quantization to stay under this limit. The quant column refers to the precision of the model weights. Lower precision reduces memory usage but may impact output quality.

Models like Allegro at 2.8B and Open Sora Plan at 2.7B use 2 GB at Q4_K_M. LFM2 1.2B and Playground v2.5 use 1.9 GB at Q4_K_M. Stable Diffusion 3.5 Medium and Canary 1B or Qwen 2.5B use 1.8 GB at Q4_K_M. SeamlessM4T v2 and Parler TTS use 2 GB at Q5_K_M. Kimi K3 DSpark uses 2 GB at Q5_K_M. SmolVLM and Stable Diffusion 3 Medium use 2 GB at Q6_K. Pyramid Flow and Wav2Vec2 or XLS R use 2 GB at Q6_K.

Smaller models offer more room for context. Moondream 2 uses 1.9 GB at Q6_K. Qwen3 1.7B and SmolLM2 use 1.7 GB at Q6_K. StableLM 2 1.6B and Sana 0.6B or 1.6B use 2 GB at Q8_0. Zonos 0.1 and Dia 1.6B use 2 GB at Q8_0. Whisper Large v3 uses 2 GB at Q8_0. ControlNet and Hunyuan DiT use 1.9 GB at Q8_0. Stable Video Diffusion and Whisper Large v2 or turbo use 1.9 GB at Q8_0. AudioGen and AudioLDM 2 use 1.9 GB at Q8_0. Tango 2 uses 1.8 GB at Q8_0. TinyLlama 1.1B and SantaCoder 1.1B use 1.4 GB at Q8_0.

Some models exceed the 2 GB limit of the MX230. These models require CPU offloading. This process uses your 32 GB of system RAM to hold parts of the model. Offloading increases latency because data must travel between the system RAM and the graphics card. SmolLM3 3B and Replit Code v1.5 3B need 2.2 GB at Q4_K_M and 4.2 GB of system RAM. Kandinsky 3.1 and Voxtral Mini or Small need 2.2 GB at Q4_K_M and 4.2 GB of system RAM. Orpheus TTS and Higgs Audio v2 also follow these requirements.

Larger models demand more system resources. MusicGen small or medium or large needs 2.4 GB at Q4_K_M and 4.4 GB of system RAM. Stable Diffusion XL needs 4.1 GB at FP8 and 6.1 GB of system RAM. SDXL Turbo and SDXL Lightning need 2.6 GB at Q4_K_M and 4.6 GB of system RAM. These configurations assume you have enough free system memory to handle the offload overhead.

Keep the 4k context caveat in mind for all models. Context window size consumes additional memory beyond the base model size. Large context windows may cause a model to exceed your available memory even if the base model fits. Monitor your memory usage closely when increasing context length. This ensures the system remains stable during inference.