Balancing Model Quality and Hardware Demands in AI Workstations

There is no doubt that AI is one of the hottest topics over the past four years. Of course, AI has been around before that in various forms.  Simple challenge and response apps were rather common on BBS systems while machine learning decision trees have been used in video games and vector-based data analysis can be found on Intel 386 retro machines. While these things existed they never went mainstream until the launch of ChatGPT.

Early versions of AI were extremely hardware limited making the development and deployment rather challenging.  Much like with video games there is a minimum system spec which establishes a starting point for developers.  Better hardware will increase performance but limit sales and increase production costs.  Targeting lower class hardware can maximize your ROI but may also create something that was completely unusable.  This is a major reason why early AI/ML had limited market penetration and why the first Macintosh was a failure.  The hardware wasn’t there to make it worthwhile and/or useful.

Thankfully, with modern hardware, we don’t have so many limitations and can make private and local AI useful and allow us to do just about anything from running a local LLM chatbot or voice assistant to actually developing AI applications and even fine tune train your own model.

Building an AI LLM workstation is quite similar to building a gaming PC however, the class of hardware does change a bit and makes an old computer builders’ phrase relevant once again.

“Well, it depends on what you plan to do”

Gaming PC hardware is interesting in that for the longest time it didn’t matter what you had because games and office applications were optimized to run on a 99 percent of existing computer hardware and gaming consoles.  However, with AI hardware there is a very distinct line between the minimal system specs and what would be considered a recommended system spec and to know how to determine the difference we need to first understand the different components.