Best local AI models for NVIDIA RTX 3070 Ti Laptop

The largest models that fit fully

The 30 largest of the 123 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The NVIDIA RTX 3070 Ti Laptop graphics card features 8 GB of GDDR6 video memory. This dedicated memory determines the maximum size of the artificial intelligence models you can run entirely on your graphics hardware. Running models completely within this video memory ensures the fastest possible processing speeds for text generation and image creation.

The quantization column indicates the compression level applied to each model. Quantization reduces the size of a model so it fits into your hardware. For example, Gemma 2 9B fits at the Q4_K_M quantization level using 8 GB of video memory. Models like Llama 3.1 8B fit at the Q5_K_M quantization level using 7.4 GB of video memory. Smaller models like Mistral 7B can run at the higher quality Q6_K quantization level using 7.4 GB of video memory.

You can run larger models by offloading parts of the workload to your system memory. This approach assumes your laptop has 32 GB of system RAM. For instance, FLUX.1 dev requires 14.4 GB of video memory at FP8 or optimized settings, which uses 16.4 GB of system RAM. Similarly, Gemma 3 12B and Mistral NeMo 12B require 8.8 GB of video memory at Q4_K_M quantization, which utilizes 10.8 GB of system RAM.

Offloading models to system RAM comes with a performance cost. System memory is much slower than the GDDR6 video memory on your graphics card. While offloading allows you to run larger models like Vicuna 13B or Open-Sora 2.0, your generation speeds will decrease significantly compared to models that fit entirely within your 8 GB video memory.

Memory calculations for these models are based on a standard 4k context window. If you increase the context window to process longer documents or extended conversations, the memory usage will rise. You must monitor your memory usage closely when expanding the context window to avoid running out of video memory.