Best local AI models for NVIDIA GTX 1070 Ti

The largest models that fit fully

The 30 largest of the 123 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The NVIDIA GTX 1070 Ti features 8 GB of GDDR5 memory. This onboard memory determines the size of the AI models you can run locally. To fit inside this hardware limit, models must use quantization. Quantization is a compression method that reduces model size while keeping most of the intelligence. The quant column shows the best balance of quality and size for each model.

For models that fit entirely inside your 8 GB of video memory, you get the fastest generation speeds. The largest fully fitting model is Mochi 1 at 10B using the Q4_K_M quant which takes 7.3 GB. Gemma 2 9B fits at Q4_K_M using 8 GB. Several 9B models like Nemotron Nano 4B / 9B, GLM-4 9B / GLM-4.5-Air, Yi-Coder 1.5B / 9B, GLM-4-9B-Chat / CodeGeeX4, and GLM-4V-9B / GLM-4.1V-Thinking fit at the Q5_K_M quant using 7.7 GB. Chroma 8.9B fits at Q5_K_M using 7.6 GB. Llama 3.1 8B fits at Q5_K_M using 7.4 GB.

Other 8B models fit at the higher quality Q6_K quant using 7.9 GB of memory. These include Granite 3.3 2B / 8B, Ministral 3B / 8B, InternLM 3 8B, OpenCoder 1.5B / 8B, Seed-Coder 8B, MiniCPM-V 2.6 / MiniCPM-o 2.6, Idefics 3 8B, Fuyu-8B, Emu3, and Stable Diffusion 3.5 Large / Turbo. EXAONE 3.5 2.4B / 7.8B fits at Q6_K using 7.7 GB. Mistral 7B fits at Q6_K using 7.4 GB. Several 7B models like Qwen2.5 0.5B / 1.5B / 3B / 7B, OLMo 2 1B / 7B, Falcon 3 1B / 3B / 7B, Command R7B, OpenHermes 2.5, Zephyr 7B Beta, OpenChat 3.5, Starling LM 7B, and Codestral Mamba 7B fit at Q6_K using 6.9 GB.

When a model is too large for the 8 GB video memory, you can use CPU offload if you have 32 GB of system RAM. This process splits the model layers between your graphics card and your system memory. Offloading allows you to run larger models but it reduces your generation speed because system RAM is much slower than GDDR5 video memory.

For CPU offload, Open-Sora 2.0 at 11B needs 8.1 GB at Q4_K_M and 10.1 GB of system RAM. FLUX.1 dev at 12B needs 14.4 GB at FP8 / optimized and 16.4 GB of system RAM. Several 12B models like Gemma 3 12B, Gemma 4 12B, Mistral NeMo 12B, Pixtral 12B, FLUX.1 schnell, FLUX.1 Kontext dev, and FLUX.1 Krea dev need 8.8 GB at Q4_K_M and 10.8 GB of system RAM. Vicuna 13B needs 9.5 GB at Q4_K_M and 11.5 GB of system RAM.

All memory calculations assume a standard 4k context window. If you increase the context window to process longer texts, the memory usage will grow. This extra memory demand might force you to use a lower quant or use CPU offload to avoid running out of video memory.