Go to https://www.piavpn.com/HardwareHaven to get 83% off from our sponsor Private Internet Access with 4 months free! ► Want to support the channel and unlock some perks in the process? Become a RAID member on Patreon or YouTube! 🔓 Patreon: https://www.patreon.com/hardwarehaven 🔓 YouTube: https://www.youtube.com/channel/UCgdTVe88YVSrOZ9qKumhULQ/join ► Checkout items I used (includes affiliate links from which I may receive compensation): 🛍️ V100 SXM2 16G: https://ebay.us/Czmgxx 🛍️ SXM2 PCIe Adapter (Amazon): https://amzn.to/3PgPOWu 🛍️ Cheaper SXM2 Adapters on Ebay: https://ebay.us/3GoalM 🛍️ 80mm Noctua Fan: https://amzn.to/3PaOcxz 🛍️ Motherboard used for testing: https://amzn.to/4ndEBTd ►Here's the fan shroud model, but seriously, it's not very good: https://www.printables.com/model/1707084-80mm-fan-shroud-for-nvidia-v100-sxm2 🎥 Curious About the equipment I use to make my videos? Click Here ► https://hardwarehaven.media/gear --------------------------------------------------- Music (in order): "Hardware Haven Theme" -Me (https://youtu.be/FwD2mOYDPNA) "CRENSHAW VIBES" - GARRISON (https://soundcloud.com/garrison-brown) "Sunshower" - LATASHÁ(https://soundcloud.com/best-music-pro...) "If You Want To" - Me --------------------------------------------------- Timestamps: 0:00 This is the strangest GPU you've ever seen 0:56 What is SXM? 2:35 PCIe Adapter 3:15 Nvidia V100 4:17 Building my GPU 4:48 Private Internet Access (sponsor) 5:49 Test system and booting it up 6:23 Cooling solution 8:11 Total cost breakdown 9:08 Trying out Ollama 10:11 RTX 3060 12G Comparison 10:57 My rudimentary benchmark results 13:12 Trying out Frigate NVR 15:06 Future plans with this GPU 15:40 Some other SXM adapters 16:28 Trying to stay positive
ADVERTISEMENT
You violated the first rule of the SXM club, we're not talking about the SXM group, now the value will skyrocket and destroy the option.
Now the price of the v100 will go up, mf
great now craftcomputing will get hold on this and make a gaming server out of this
Datacenters are designed for maximum performance per watt at peak utilization. "idle" is not really a thing that's being considered. They also don't usually control fan speeds, as the hardware is generally running between 75% and 100% anyway, and you could get strange oscillations in airflow and vibrations if a whole rack of fans is ramping up and down. TL;DR ya boi is born to run, so it never bothered learning to crawl.
Are those real images from the front of your house? Be careful, you may be revealing more about your location than you realize.
Keep using the zip tie. Long live the jank.
Those SXM2/3 modules are super finicky. I tried this route a while back and bought 3 V100 - 32GB at 350 each and I couldn't get any of them to work with that adapter card. There are also a few variants of those adapters as well, the more pricier one comes with a low profile cooler and blower fan. Also for those wondering, the price of these are probably not going to go up they're around (450-500 for the 32GB and as low as $150 for 16gb). It's been a known in thing in the LLM scene for a long time. Also instead of going this route with SXM2/3 and dapter, if you're going to get the 16GB variant, just get the standard PCIe-form factor version. V100 are workhorses but when you need things like flash-attn, they're not supported. Lastly they are no longer supported as of CUDA 13 -- which at the moment isn't a big deal since a lot of models are still supporting CUDA 12. However, inference engines like llamacpp are starting to give more focus on newer GPUs and CUDA 13, optimization for these cards with newer LLMs will vary. Just be ware of these things when you build your V100 setup and potentially how long you can get out of them.
Let's rob my local datacenter for their ram
Next Step: Make a video with our dad Jeff Geerling about running AI models in Raspberry Pi 5 with a pcie adapter and this card
before the AI bobble you could get a server with 4x of these v100 16GB for cheap.
No Flash Attention kills the value of the V100. FA is important.
"Using an Nvidia V100 SXM2 Card in My Homelab" - thanks to whoever dearrow'd the title. dearrow is an awesome browser plugin.
Ridiculous content _should_ be this good!
These are quite good at cracking passwords, as well. Perf/watt with hashcat was unbeatable until only a few years ago.
honestly SXM socket should become the standard for retail too. Your GPU would connect parallel to the motherboard, not perpendicular. No sag, no annoying 12V wiring, greater room for a tower heatsinks (possibly standardized too so GPUs aftermarket heatsinks would also be an opion like CPUs)
Have you consider solar for powering your servers? Like Harbor Freight solar panels, a 200-watt panel is about $50 on sale. I know even 50% power is often optimistic, but I would love to see AI home lab models doing sun tracking and energy to run basically off the sun, storing extra power from day by either selling extra power back to the grid during the day and buying at night or the more expensive route of actually storing it to batteries.
Wow Colten, that was a really cool and slightly janky use of nearly e-waste hardware thanks for sharing this.
I’ve been wondering when someone would cover these, I’m more so interested in blender rendering performance on these lil guys.
In school one day, i was browsing an online marketplace for a bigger rack to put all of my servers (basicly e-waste😅). A friend of my looked and asked "Why do you spend so much money on computer parts and you dont even game?". Every cell in my body cried.
i remember two years ago a bunch of these modules dropped on ebay for even cheaper but the problem was the cost of the adapter (or maybe p100s?). for 200 total it makes sense to get. i hear new tiny models are actually usable for tool calls.