Best local AI models for NVIDIA RTX 3070 Ti

The largest models that fit fully

The 30 largest of the 123 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The NVIDIA RTX 3070 Ti graphics card features 8 GB of GDDR6X video memory. This dedicated memory determines the size of the artificial intelligence models you can run locally. To fit a model entirely on the graphics card, the model files and the active memory must stay within this limit. Running models fully on the video memory ensures the fastest processing speeds.

The quantization column shows the compression level used to shrink these models. Quantization reduces the precision of model weights to save space. For example, the Mochi 1 10B model fits in 7.3 GB of video memory using the Q4_K_M quantization. Other models like Llama 3.1 8B fit in 7.4 GB of video memory using the Q5_K_M quantization. Many models like Mistral 7B use the Q6_K quantization to fit in 7.4 GB of video memory.

When a model size exceeds the 8 GB video memory limit, you must use CPU offloading. This process splits the model between your graphics card and your system memory. We assume a standard system configuration with 32 GB of system RAM for these scenarios. Offloading allows you to run larger models, but it significantly reduces generation speed because system RAM is much slower than GDDR6X video memory.

Several models require CPU offloading on this hardware. The FLUX.1 dev 12B model needs 14.4 GB of memory at FP8 or optimized settings, which requires 16.4 GB of system RAM. The Gemma 3 12B, Gemma 4 12B, Mistral NeMo 12B, Pixtral 12B, FLUX.1 schnell, FLUX.1 Kontext dev, and FLUX.1 Krea dev models all need 8.8 GB of memory at Q4_K_M quantization, requiring 10.8 GB of system RAM. Vicuna 13B needs 9.5 GB of memory at Q4_K_M, requiring 11.5 GB of system RAM.

Memory calculations in our catalog are based on a standard 4k context window. As your conversation history grows, the model requires more video memory to track the context. If you write very long prompts or generate long responses, the memory usage will increase. This extra usage can push a model over the 8 GB limit and trigger automatic CPU offloading.