Best local AI models for NVIDIA RTX 5070

The largest models that fit fully

The 30 largest of the 147 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The NVIDIA RTX 5070 features 12 GB of GDDR7 memory. This dedicated video memory determines the maximum size of the artificial intelligence models you can run locally. To run a model entirely on your graphics hardware, the model files and the active processing memory must fit within this 12 GB limit. If a model exceeds this capacity, your system must use alternative execution methods.

The quantization column indicates the compression level applied to each model. Quantization reduces the size of model weights to save memory. For example, a Q4_K_M quantization represents a medium four bit compression. A Q6_K quantization represents a six bit compression, while Q8_0 represents an eight bit compression. Higher quantization numbers preserve more original model accuracy but require more memory space.

You can run several large models entirely within the video memory of your card. The DeepSeek-Coder-V2 16B and Kimi-VL A3B models fit at Q4_K_M quantization using 11.7 GB of memory. The Apriel-1.5-15B-Thinker and StarCoder2 15B models fit at Q4_K_M using 11 GB of memory. The Qwen2.5 14B model fits at Q4_K_M using 11.6 GB of memory. Models like Phi-4, Phi-4-reasoning, Wan 2.2 T2I, Wan 2.1 14B, and SkyReels V2 fit at Q5_K_M using 11.9 GB of memory.

Other models fit comfortably at higher precision levels. The Gemma 3 12B, Gemma 4 12B, Mistral NeMo 12B, Pixtral 12B, FLUX.1 schnell, FLUX.1 Kontext dev, and FLUX.1 Krea dev models fit at Q6_K quantization using 11.8 GB of memory. You can run the Nemotron Nano 9B, GLM-4 9B, Yi-Coder 9B, and GLM-4-9B-Chat models at Q8_0 quantization using 11.4 GB of memory.

When a model is too large for the video memory, you can offload parts of it to your system RAM. This offload process requires a system with 32 GB of system RAM. For example, FLUX.1 dev requires 14.4 GB at FP8 and uses 16.4 GB of system RAM. Codestral 22B requires 16.1 GB at Q4_K_M and uses 18.1 GB of system RAM. Offloading allows you to run larger models like CogVLM2 or Solar Pro, but it reduces processing speed because system RAM is slower than GDDR7 memory.

All memory calculations assume a standard four kilobyte context window. If you increase the context window to process longer documents or longer conversations, the model will require more memory. This extra memory usage might force you to choose a smaller model or a lower quantization level to prevent memory exhaustion.