Best local AI models for NVIDIA Quadro 7000
6 GB GDDR5. At a 4k context, 114 of the 233 models in our catalog with verified parameter counts fit fully, up to Granite 3.3 2B / 8B at 8B parameters.
Check your own machine against every model →The largest models that fit fully
The 30 largest of the 114 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.
| Model | Parameters | Best quant that fits | Memory used at 4k |
|---|---|---|---|
| Granite 3.3 2B / 8B | 8B | Q4_K_M | 5.9 GB |
| Ministral 3B / 8B | 8B | Q4_K_M | 5.9 GB |
| InternLM 3 8B | 8B | Q4_K_M | 5.9 GB |
| OpenCoder 1.5B / 8B | 8B | Q4_K_M | 5.9 GB |
| Seed-Coder 8B | 8B | Q4_K_M | 5.9 GB |
| MiniCPM-V 2.6 / MiniCPM-o 2.6 | 8B | Q4_K_M | 5.9 GB |
| Idefics 3 8B | 8B | Q4_K_M | 5.9 GB |
| Fuyu-8B | 8B | Q4_K_M | 5.9 GB |
| Emu3 | 8B | Q4_K_M | 5.9 GB |
| Stable Diffusion 3.5 Large / Turbo | 8B | Q4_K_M | 5.9 GB |
| EXAONE 3.5 2.4B / 7.8B | 7.8B | Q4_K_M | 5.7 GB |
| Mistral 7B | 7B | Q4_K_M | 5.7 GB |
| Qwen2.5 0.5B / 1.5B / 3B / 7B | 7B | Q5_K_M | 6 GB |
| OLMo 2 1B / 7B | 7B | Q5_K_M | 6 GB |
| Falcon 3 1B / 3B / 7B | 7B | Q5_K_M | 6 GB |
| Command R7B | 7B | Q5_K_M | 6 GB |
| OpenHermes 2.5 | 7B | Q5_K_M | 6 GB |
| Zephyr 7B Beta | 7B | Q5_K_M | 6 GB |
| OpenChat 3.5 | 7B | Q5_K_M | 6 GB |
| Starling LM 7B | 7B | Q5_K_M | 6 GB |
| Codestral Mamba 7B | 7B | Q5_K_M | 6 GB |
| CodeGemma 2B / 7B | 7B | Q5_K_M | 6 GB |
| aiXcoder-7B | 7B | Q5_K_M | 6 GB |
| Nxcode / CodeQwen 1.5 7B | 7B | Q5_K_M | 6 GB |
| Janus-Pro 1B / 7B | 7B | Q5_K_M | 6 GB |
| Ruyi-Mini-7B | 7B | Q5_K_M | 6 GB |
| Qwen2-Audio 7B | 7B | Q5_K_M | 6 GB |
| Qwen2.5-Omni 3B / 7B | 7B | Q5_K_M | 6 GB |
| YuE | 7B | Q5_K_M | 6 GB |
| Magicoder-S-DS 6.7B | 6.7B | Q5_K_M | 5.7 GB |
Close, but only with CPU offload
These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.
| Model | Parameters | Memory at Q4_K_M | System RAM at 4k |
|---|---|---|---|
| Llama 3.1 8B | 8B | 6.4 GB needed | 8.4 GB |
| Chroma | 8.9B | 6.5 GB needed | 8.5 GB |
| Gemma 2 9B | 9B | 8 GB needed | 10 GB |
| Nemotron Nano 4B / 9B | 9B | 6.6 GB needed | 8.6 GB |
| GLM-4 9B / GLM-4.5-Air | 9B | 6.6 GB needed | 8.6 GB |
| Yi-Coder 1.5B / 9B | 9B | 6.6 GB needed | 8.6 GB |
| GLM-4-9B-Chat / CodeGeeX4 | 9B | 6.6 GB needed | 8.6 GB |
| GLM-4V-9B / GLM-4.1V-Thinking | 9B | 6.6 GB needed | 8.6 GB |
| Mochi 1 | 10B | 7.3 GB needed | 9.3 GB |
| Open-Sora 2.0 | 11B | 8.1 GB needed | 10.1 GB |
How to read this
The NVIDIA Quadro 7000 graphics card features 6 GB of GDDR5 onboard memory. This hardware memory size determines which artificial intelligence models can run directly on your GPU. When a model fits entirely within this video memory, it processes tokens at maximum speed. If a model exceeds this limit, you must use alternative execution strategies.
The quantization column indicates the compression level applied to each model. Quantization reduces the size of model weights to save memory. For example, the Q4_K_M quant represents a medium four bit quantization that balances speed and accuracy. The Q5_K_M quant uses five bits for slightly higher precision. Using these specific quants allows the card to host larger models than would otherwise be possible.
Several 8B models can run entirely inside your video memory using the Q4_K_M quantization level. Granite 3.3 8B, Ministral 8B, InternLM 3 8B, OpenCoder 8B, Seed-Coder 8B, MiniCPM-V 2.6, Idefics 3 8B, Fuyu-8B, Emu3, and Stable Diffusion 3.5 Large use 5.9 GB of video memory. EXAONE 3.5 7.8B and Mistral 7B fit comfortably at 5.7 GB with the same Q4_K_M quantization.
Many 7B models run on this card using the Q5_K_M quantization level. Qwen2.5 7B, OLMo 2 7B, Falcon 3 7B, Command R7B, OpenHermes 2.5, Zephyr 7B Beta, OpenChat 3.5, Starling LM 7B, Codestral Mamba 7B, CodeGemma 7B, aiXcoder-7B, Nxcode, Janus-Pro 7B, Ruyi-Mini-7B, Qwen2-Audio 7B, Qwen2.5-Omni 7B, and YuE all require exactly 6 GB of video memory. Magicoder-S-DS 6.7B requires 5.7 GB of video memory at this same quantization level.
When a model is too large for the onboard memory, you can offload parts of it to your system RAM. This offloading process allows you to run larger models but reduces generation speeds. For instance, Llama 3.1 8B needs 6.4 GB of video memory at Q4_K_M and requires 8.4 GB of system RAM. Gemma 2 9B needs 8 GB of video memory and 10 GB of system RAM. Mochi 1 needs 7.3 GB of video memory and 9.3 GB of system RAM. Open-Sora 2.0 needs 8.1 GB of video memory and 10.1 GB of system RAM.
Other offload options include Chroma, Nemotron Nano 9B, GLM-4 9B, Yi-Coder 9B, GLM-4-9B-Chat, and GLM-4V-9B. These models all need 6.6 GB of video memory at Q4_K_M and require 8.6 GB of system RAM. Be aware that all memory calculations assume a standard 4k context window. Increasing the context length will require more video memory and may force additional offloading to your system RAM.