Best local AI models for Intel Arc A380

The largest models that fit fully

The 30 largest of the 114 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The Intel Arc A380 graphics card features 6 GB of GDDR6 memory. This dedicated memory size determines which artificial intelligence models can run entirely on your hardware. To execute local models efficiently, the model files must fit within this onboard space. If a model exceeds this limit, your system must use alternative execution methods.

The quantization column indicates the compression level applied to each model. Quantization reduces the precision of model weights to save space. For the Intel Arc A380, the best quantization level for 8B models is Q4_K_M, which typically uses 5.9 GB of memory. Models like Granite 3.3 8B, Ministral 8B, InternLM 3 8B, OpenCoder 8B, Seed-Coder 8B, MiniCPM-V 2.6, MiniCPM-o 2.6, Idefics 3 8B, Fuyu-8B, Emu3, and Stable Diffusion 3.5 Large use this configuration. EXAONE 3.5 7.8B also runs at Q4_K_M using 5.7 GB of memory.

Several 7B models can run at a higher Q5_K_M quantization level using 6 GB of memory. These include Qwen2.5 7B, OLMo 2 7B, Falcon 3 7B, Command R7B, OpenHermes 2.5, Zephyr 7B Beta, OpenChat 3.5, Starling LM 7B, Codestral Mamba 7B, CodeGemma 7B, aiXcoder-7B, Nxcode, CodeQwen 1.5 7B, Janus-Pro 7B, Ruyi-Mini-7B, Qwen2-Audio 7B, Qwen2.5-Omni 7B, and YuE. Mistral 7B uses 5.7 GB at Q4_K_M, while Magicoder-S-DS 6.7B uses 5.7 GB at Q5_K_M.

When a model is too large for the 6 GB graphics memory, you can offload parts of it to your system RAM. This process requires a system with 32 GB of system RAM. Offloading allows you to run larger models, but it costs performance because system RAM is slower than GDDR6 memory. For example, Llama 3.1 8B needs 6.4 GB at Q4_K_M and requires 8.4 GB of system RAM. Chroma needs 6.5 GB at Q4_K_M and 8.5 GB of system RAM.

Other offload options include Gemma 2 9B, which needs 8 GB at Q4_K_M and 10 GB of system RAM. Nemotron Nano 9B, GLM-4 9B, GLM-4.5-Air, Yi-Coder 9B, GLM-4-9B-Chat, CodeGeeX4, GLM-4V-9B, and GLM-4.1V-Thinking all need 6.6 GB at Q4_K_M and 8.6 GB of system RAM. Mochi 1 needs 7.3 GB at Q4_K_M and 9.3 GB of system RAM. Open-Sora 2.0 needs 8.1 GB at Q4_K_M and 10.1 GB of system RAM.

You must consider the context window size when loading these models. The memory calculations on this page assume a standard 4k context limit. Running longer context windows increases memory consumption. If you increase the context length beyond 4k, the model may exceed the 6 GB limit and fail to run entirely on the graphics card.