Best local AI models for NVIDIA 840A

The largest models that fit fully

The 30 largest of the 56 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The NVIDIA 840A graphics card features 2 GB of DDR3 memory. This hardware configuration requires careful management of local AI models because of the limited memory capacity. The memory size determines how much model data can reside directly on the graphics card for fast processing. Running out of graphics memory will cause execution to fail or slow down significantly.

To fit models into this 2 GB limit you must use quantized versions. The quantization column shows the optimal format for each model. For example Allegro 2.8B and Open-Sora Plan 2.7B require a Q4_K_M quantization to use exactly 2 GB of memory. Models like SmolVLM 2B and Stable Diffusion 3 Medium can run at a higher Q6_K quantization while still fitting within the 2 GB limit. Smaller models such as TinyLlama 1.1B can run at Q8_0 quantization using only 1.4 GB of memory.

When a model exceeds the 2 GB graphics memory limit you must use CPU offload. This technique splits the model between your graphics card and your system RAM. We assume your computer has 32 GB of system RAM for these scenarios. For instance running SmolLM3 3B or Kandinsky 3.1 at Q4_K_M requires 2.2 GB of graphics memory and 4.2 GB of system RAM. Larger models like Stable Diffusion XL require 4.1 GB of memory at FP8 or optimized settings and 6.1 GB of system RAM.

CPU offload allows you to run larger systems but it introduces a performance cost. Transferring data between the DDR3 graphics memory and the system RAM over the system bus is slow. You will experience much lower generation speeds compared to models that fit entirely within the 2 GB graphics memory. For the best performance you should select models that run completely on the graphics card.

You must also consider the memory cost of context length. Running a model with a standard 4k context window requires additional graphics memory during execution. The memory figures listed for the models represent the base weights only. If you increase the context length or process long documents the memory usage will exceed the listed limits and you may need to use CPU offload even for smaller models.