The problem is simple: consumer motherboards don’t have that many PCIe slots, and consumer CPUs don’t have enough lanes to run 3+ GPUs at full PCIe gen 3 or gen 4 speeds.
My idea was to buy 3-4 computers for cheap, slot a GPU into each of them and use 4 of them in tandem. I imagine this will require some sort of agent running on each node which will be connected through a 10Gbe network. I can get a 10Gbe network running for this project.
Does Ollama or any other local AI project support this? Getting a server motherboard with CPU is going to get expensive very quickly, but this would be a great alternative.
Thanks
I see. I must be doing something wrong because the only ones I found were over $1000 on eBay. Do you have any tips/favoured listings?
All I did for that one was search “Threadripper” and look at the pictures for ones with 4x x16 slots that were not hella expensive. There are technically filters for that, but, I don’t trust people to list their things correctly.
For which chipsets, ect to look for, check out this page. If you click on Learn More next to AM5 for example, it tells you how many PCIe lanes are on each chipset type which can give you some initial search criteria to look for. (That is what made me point out x670E as it has the most lanes, but is not newest gen, so you can find used versions.)