The Secret in the Basement

While everyone else is fighting over the latest consumer cards with 16GB of VRAM and software-locked features, there’s a graveyard of enterprise-grade power sitting in data centres. The AMD Radeon Instinct MI50.

It’s a gfx906 beast. It’s old enough to be “cheap” on the secondary market, but powerful enough to run models that would make a 3090 look like a calculator — at least for token crunching, where memory bandwidth is king. On raw compute and modern tricks, the 3090 still wins. This build isn’t fast; it’s capable.

The Goal: 64GB Capacity Across Dual Cards

By pairing two MI50s, we’re creating a 64GB VRAM footprint. We aren’t talking unified pools or Infinity Fabric — we’re talking high-bandwidth tensor splitting. In the world of local LLMs, VRAM is the only metric that actually matters. 64GB is the “sweet spot” where you can run Llama 3 70B at Q4/Q5 quantisations with significant headroom for context.

The Software Reality (The “Gotchas”)

This is where the “enthusiast” builds often fail:

  • ROCm Status: Official ROCm 7.x has dropped gfx906 support. To make this work, you’ll be relying on community-maintained forks and specific Docker images (like the mixa3607/ML-gfx906 builds).
  • The BF16 Gap: Vega 20 has no hardware support for bfloat16. While llama.cpp handles this gracefully, some vLLM/Triton workflows will be rough.
  • The Path: llama.cpp is your primary friend here. It’s the smoothest way to bridge the gap between legacy hardware and modern weights.

The Build Specs (The “Don’t Brick Your House” List)

  • The Cards: Dual AMD Instinct MI50 (target: ~£150 each, eBay UK, used).
  • The Board: A dedicated x8/x8 PCIe split board. Critical: you must ensure “Above-4G-Decoding” is enabled in your BIOS, or the cards won’t even enumerate.
  • The Power: A minimum 1200W Tier-A PSU. These things pull ~300W each under load. You aren’t just building a PC; you’re building a space heater that happens to do math.
  • The Cooling: Active ducting is non-negotiable. These are passively cooled enterprise cards. If you don’t move the air, they’ll throttle or cook within 120 seconds of a heavy inference pass.

The Verdict

It’s not “easy.” You have to deal with community-forked drivers and serious thermal management. But for a few hundred quid, you get a workstation that puts most “high-end” consumer setups to shame on capacity alone.

This is the start of the build log. Next up: the PCIe board selection deep dive and the ROCm fork setup — with the actual numbers once the box is running.