Running a large language model on your own machine is, more than anything, a memory problem. Speed matters for how quickly it responds, but whether it runs at all comes down to a simpler question: do the model's weights fit in the memory you have? Once you understand how that math works, the requirements stop being mysterious.
A model is a large pile of numbers called weights, and its size is quoted in parameters, usually billions of them. Each weight takes up space, and how much space depends on the precision you store it at. This is the part most people miss. A weight stored at sixteen-bit precision takes two bytes. The same weight at eight-bit takes one byte. At four-bit, it takes half a byte. The number of parameters is fixed by the model, but the bytes per parameter are a choice, and that choice changes the memory footprint dramatically.
You can do the arithmetic yourself. Multiply the number of parameters by the bytes per weight and you get the memory just to hold the model. A model with a given parameter count needs roughly twice as much memory at sixteen-bit as it does at eight-bit, and roughly four times as much as at four-bit. That is the entire reason quantisation exists.
Quantisation is the process of storing those weights at lower precision. Instead of keeping every weight at full sixteen-bit precision, a quantised model represents them with fewer bits. The payoff is memory: a model that will not fit on your graphics card at full precision may fit comfortably once quantised to eight or four bits. This is what lets models that sound impossibly large run on ordinary consumer hardware.
There is a trade-off, and it is honest to state it plainly. Lowering precision loses some information. Done carelessly, aggressive quantisation can make a model's output noticeably worse. Done well, with modern methods, the quality cost of moderate quantisation is often small relative to the memory it saves. The right level depends on the model and on what you are asking it to do, and it is genuinely a judgement call rather than a fixed rule.
The weights are not the only thing that needs memory. Running a model also requires room for the context, the tokens it is currently working with, and for the intermediate values it computes as it generates. The longer the context you use, the more memory that takes, on top of the weights. So the total memory you need is the model's weights plus overhead, and the overhead grows with how much text you are asking it to consider at once. When you estimate whether a model fits, leave headroom above the raw weight size rather than budgeting to the last byte.
Where the model runs matters too. A graphics card's video memory is fast and ideal for this, but limited. System memory is larger but slower, and running from it means the model responds more sluggishly. Some setups split a model across both, keeping as much as possible on the card and spilling the rest to system memory. It works, but you feel the difference. The cleanest experience is a model that fits entirely in video memory with room to spare.
So when you ask whether your machine can run a particular model, reframe it as a memory question. Start from the parameter count. Decide a precision, which usually means decide a quantisation level. Multiply to estimate the weights. Add overhead for context and generation. Then compare that total against your video memory first, and your system memory as a fallback. If the honest answer is that it only fits by spilling into slow memory, that is worth knowing before you download tens of gigabytes.
None of this requires benchmarks or promises about speed. It is arithmetic about memory, and it is the most reliable way to know in advance whether a model belongs on your hardware.
