Buying a ready-made AI workstation will cost serious dough. For example, a single RTX 4090, as of this writing, is going for north of $1,600 on Amazon. And that’s not counting the rest of the computer…
Of course, before you make such an investment you need to have a solid handle on your why. In my case, I’m not going to be training foundation models from the ground up. Mostly I need to do local inference, including text-to-audio. Running some of the chatbots locally will also be important. This is all doable on 8Gb of VRAM, so that’s my entry level. Everything beyond that is targeting cost efficiency.
I found a GTX 1070Ti 8Gb on Ebay, with a guarantee, for $80. Perfect.
For a motherboard, I took an eBay gamble and got an Asus Prime Z370-P II board for $40 (which included a CPU air cooler). The listing specified a “bent pin”–and it turned out to literally be one bent pin on the LGA 1151 socket. One other pin was slightly twisted, and might have just worked, but I fixed it too. Now, the “1151” in the socket designation is the actual number of pins. Well over a thousand tiny pins, tightly packed. Straightening a pin requires a steady hand and great attention to detail. If the pin breaks off, it may or may not be a board-ending catastrophe…it all depends if the pin in question is one of the redundant ones–for example there’s over a dozen power rail pins, and losing one or two is probably sustainable. I wouldn’t recommend this approach unless you’re familiar with electronics repair.
To that I added 64Gb of factory new RAM for $98, and $180 for another eBay special Intel Core i7-8700K CPU from a reputable vendor. I wanted plenty of headroom for future upgrades so I grabbed a 1050 watt power supply from Amazon for $71. And in perhaps my biggest mistake, a SSD of only 1Tb for $42.
A few more details like some mounting brackets and a case fan for an old ATX case I had lying around came to $17.
Grand total: $529 and a weekend to build it and fiddle.
(I later added a name brand monitor, a used optical drive, and new feet for the ATX case for another $90, but these are basically luxuries.)
I haven’t run benchmarks, but at first glance it appears about 4x faster than my Macbook M1 Pro at generating 512×512 images in Stable Diffusion.
This will do nicely.
Special thanks to Uche Ogbuji who advised me on some of what’s changed in computer-building in the long time since I’ve last attempted it.
This post 100% human-written.