Building a Multi-GPU AI Workstation on a Budget

When I started this AI Workstation project, I loaded the build with a 2TB NVMe SSD and in a matter of a couple weeks I had filled the entire thing up with models I wanted to test, software packages to use and data, lots of data.  You wouldn’t believe how much data you can generate when you turn an LLM loose on 20 years of written work just to build a LLM Fine Tune Training dataset.  Combine that with copies of models used for quantization and Python virtual environment overhead and you will quickly watch your storage space and SSD wear levels switch places.

It’s actually quite mesmerizing.

I recently rebuilt the machine to upgrade from Debian 12 to Debian 13, and during the switch I upgraded to a 4TB primary drive and moved the 2TB unit for backups and long(er) term storage.  So far this has worked out in my favor and something I would strongly recommend if you decide to build an AI Workstation, your internet connection will thank you.

The case you choose is personal preference.  When I first assembled this AI Workstation, I used a Fractal Meshify 2 Compact and actually rather enjoyed the result.  The case comes with 140mm intake fans and, as the name suggests, is rather compact allowing me to put the build just about anywhere. 

When I migrated the build, I needed to swap out the CPU cooler.  In the fractal I was able to use the Noctua NH-U14S TR4-SP3 which had a single 140mm fan.  This is the cooler I would recommend if you have space.

The Silverstone WS380-E Workstation is basically a 4U server in the pedestal configuration.  Due to the limited amount of space I needed to move to the Noctua NH-U9 TR4-SP3 which comes equipped with dual 92mm fans.  It is a little noisier but, when compared to the 120mm fans in the WS380-E you would never notice.

With the recent rebuild I migrated everything into the SilverStone W380-E Workstation case and quickly discovered several things.

• That case is massive in what it supports including AIO Watercooling, dual 4-drive SAS bays and up to 4-GPUs
• My ASUS ROG Strix TRX40-E Gaming cannot take advantage of what the chassis provides
• The 8th expansion slot is extremely helpful as it allowed me to move one of my RTX A4500 cards down a slot and thus exposing the middle 16x slot instead of the secondary 4x slot coming off the chipset. This gives me options to install a 3rd GPU or, a bifurcated NVMe card for more storage options including an offload accelerator.
  Note: The Mesify case allowed this too but due to the bottom mounted power supply would obscure the cooling fan to the GPU.

Much like the motherboard I have a number of power supplies in the lab and one of the best is the SilverStone Strider 1200w.  This is an 80-Plus platinum unit designed during the era of 4-Way SLI with the individual 8-Pin PCI Express power cables to support 4x dual socket GPUs and still have power left over.

For an AI workstation build I wouldn’t go with anything less than 1200w unless you are rocking the 50-Series generation.  For those cards you will want something supporting dual 12VHPWR connections at 1500w or greater.