Best local AI models for NVIDIA RTX 2070 Laptop

The largest models that fit fully

The 30 largest of the 123 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The NVIDIA RTX 2070 Laptop graphics card features 8 GB GDDR6 of dedicated video memory. This memory size is the absolute limit for running local AI models entirely on your GPU. To run a model smoothly, the model weights and the active context must fit within this 8 GB boundary. If a model exceeds this limit, your system will slow down significantly.

The quant column shows the recommended quantization level for each model. Quantization is a compression method that reduces model size with minimal loss in quality. For example, Gemma 2 9B fits at the Q4_K_M quant using 8 GB of video memory. Models like Llama 3.1 8B run at the higher quality Q5_K_M quant using 7.4 GB. Smaller models like Mistral 7B can run at the Q6_K quant using 7.4 GB.

When a model is too large for the 8 GB video memory, you must use CPU offload. This process splits the model layers between your GPU and your system RAM. We assume your laptop has 32 GB of system RAM for these scenarios. Offloading allows you to run larger models, but it costs performance because system RAM is much slower than GDDR6 video memory.

For example, FLUX.1 dev is a 12B model that needs 14.4 GB of memory at FP8 or optimized settings. Running it requires offloading 16.4 GB to your system RAM. Other 12B models like Gemma 3 12B, Gemma 4 12B, Mistral NeMo 12B, and Pixtral 12B need 8.8 GB at the Q4_K_M quant, which requires offloading 10.8 GB to system RAM.

You can also run image generation models like FLUX.1 schnell, FLUX.1 Kontext dev, and FLUX.1 Krea dev. These 12B models need 8.8 GB at the Q4_K_M quant and require 10.8 GB of system RAM. Larger text models like Vicuna 13B need 9.5 GB at the Q4_K_M quant, which requires offloading 11.5 GB to system RAM.

All listed memory requirements are calculated using a basic 4k context window. As your conversation grows longer, the context window consumes more video memory. If you write very long prompts or have long chat histories, the model might exceed the 8 GB limit. You should monitor your memory usage during long sessions to avoid slowdowns.