Best local AI models for AMD R7 260X

The largest models that fit fully

The 30 largest of the 56 models that fit; every smaller model in the catalog fits too. Best quant means the highest quality compression whose weights and 4k context both sit inside the memory.

Close, but only with CPU offload

These need more than the card holds at their smallest practical quant, so part of the model runs from system memory (figures assume 32 GB of it). They work, several times slower.

How to read this

The AMD Radeon R7 260X graphics card features 2 GB of GDDR5 onboard memory. This memory size is the primary limiting factor for running local artificial intelligence models. Because the graphics hardware must store the active model weights to perform fast calculations, only models that fit entirely within this 2 GB limit can run at maximum speed. If a model exceeds this capacity, it cannot load directly onto the card without alternative execution strategies.

To fit models onto this hardware, developers use quantization. The quantization column shows the optimal format for each model, such as Q4_K_M, Q5_K_M, Q6_K, or Q8_0. These terms represent different levels of numerical compression. A lower quantization level like Q4_K_M compresses the model weights to four bits, which reduces the memory footprint so larger models can fit. A higher quantization level like Q8_0 uses eight bits, which preserves more original model accuracy but requires more memory.

For models that fit entirely within the 2 GB limit, you can run options like the 2.8B Allegro or the 2.7B Open-Sora Plan at Q4_K_M quantization, which use exactly 2 GB of memory. You can also run the 2.6B LFM2 or the 2.6B Playground v2.5 at Q4_K_M, which require 1.9 GB. Other compatible options include the 2.5B Stable Diffusion 3.5 Medium and the 2.5B Canary 1B or Qwen-2.5B, which use 1.8 GB. Highly compressed options like the 1.1B TinyLlama or the 1.1B SantaCoder at Q8_0 use only 1.4 GB.

When a model is too large for the onboard memory, you must use CPU offload. This technique splits the model between your graphics card and your system RAM. For example, running the 3B SmolLM3, 3B Replit Code v1.5, 3B Kandinsky 3.1, 3B Voxtral Mini or Small, 3B Orpheus TTS, or 3B Higgs Audio v2 at Q4_K_M requires 2.2 GB of video memory and 4.2 GB of system RAM. Similarly, the 3.3B MusicGen small or medium or large needs 2.4 GB of video memory and 4.4 GB of system RAM.

Larger image generation models also require CPU offload on this hardware. The 3.417B Stable Diffusion XL requires 4.1 GB of video memory at FP8 or optimized settings, along with 6.1 GB of system RAM. The 3.5B SDXL Turbo and the 3.5B SDXL Lightning both require 2.6 GB of video memory at Q4_K_M and 4.6 GB of system RAM. While offloading allows you to run these larger models, it costs significant processing speed because data must constantly travel over the system bus.

You must also consider the 4k context window caveat when running text models. The memory figures listed only cover the base model weights at startup. As you input longer prompts and generate more text, the active context memory grows. Running a model near the 2 GB limit with a full 4k context window will likely exceed your hardware capacity, which will cause the system to slow down or crash.