Best local AI models for NVIDIA GTX 1070

The largest models that fit fully

The 30 largest of the 123 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The NVIDIA GTX 1070 graphics card features 8 GB of GDDR5 video memory. This memory size determines which artificial intelligence models can run directly on your hardware. To run a model entirely on the graphics card, the model files and the active workspace must fit within this 8 GB limit. Keeping models inside the video memory ensures the fastest possible processing speeds.

The quantization column shows the compression level used to make these models fit. Quantization reduces the precision of model weights to save space. For example, a Q4_K_M quantization represents a four bit medium format. A Q6_K quantization represents a six bit format which offers higher quality but requires more memory. Choosing the correct quantization allows you to balance model accuracy against the available video memory.

Several large models can run entirely within your video memory. Mochi 1 is a 10B model that fits using a Q4_K_M quantization which uses 7.3 GB of memory. Gemma 2 9B fits at Q4_K_M quantization using exactly 8 GB of memory. Nemotron Nano 9B, GLM-4 9B, GLM-4.5-Air, Yi-Coder 9B, GLM-4-9B-Chat, CodeGeeX4, GLM-4V-9B, and GLM-4.1V-Thinking all fit at Q5_K_M quantization using 7.7 GB of memory. Chroma is an 8.9B model that fits at Q5_K_M quantization using 7.6 GB of memory.

Llama 3.1 8B fits at Q5_K_M quantization using 7.4 GB of memory. Other 8B models like Granite 3.3 8B, Ministral 8B, InternLM 3 8B, OpenCoder 8B, Seed-Coder 8B, MiniCPM-V 2.6, MiniCPM-o 2.6, Idefics 3 8B, Fuyu-8B, Emu3, and Stable Diffusion 3.5 Large fit at Q6_K quantization using 7.9 GB of memory. EXAONE 3.5 7.8B fits at Q6_K quantization using 7.7 GB of memory. Mistral 7B fits at Q6_K quantization using 7.4 GB of memory. Qwen2.5 7B, OLMo 2 7B, Falcon 3 7B, Command R7B, OpenHermes 2.5, Zephyr 7B Beta, OpenChat 3.5, Starling LM 7B, and Codestral Mamba 7B fit at Q6_K quantization using 6.9 GB of memory.

When a model is too large for the video memory, you must use CPU offload. This process splits the model between your graphics card and your system RAM. CPU offload allows you to run larger models but it reduces processing speed significantly. For example, FLUX.1 dev is a 12B model that needs 14.4 GB at FP8 or optimized settings and requires 16.4 GB of system RAM. Gemma 3 12B, Gemma 4 12B, Mistral NeMo 12B, Pixtral 12B, FLUX.1 schnell, FLUX.1 Kontext dev, and FLUX.1 Krea dev all need 8.8 GB at Q4_K_M quantization and require 10.8 GB of system RAM.

Other offload options include Open-Sora 2.0 which is an 11B model that needs 8.1 GB at Q4_K_M quantization and requires 10.1 GB of system RAM. Vicuna 13B needs 9.5 GB at Q4_K_M quantization and requires 11.5 GB of system RAM. These memory requirements are calculated using a standard context window of 4000 tokens. If you increase the context window to process longer texts, the model will require more video memory and system memory.