This one rack is 120kW of Nvidia AI compute. Google, Meta, Apple, OpenAI AND others are buying these like candy. In fact, there is a waiting list to get your hands on it. Each compute is super expensive too. All liquid cooled. Crazy tech. Crazy energy requirements too 🔥🤬all such massive energy requirements so that AI companies can sell LLM from stolen content from many humans and put everyone else out of the job while heating our planet.
@nixCraft to add some context for people on what 120kW means…
An average US home uses 10,500kWh per year, or an average of 29kWh per day. This averages out at 1.2kW.
In other words, if that server rack runs its PSU at 100%, it’s using as much power as 100 homes. In about 6sqft of floor space. Not including power used to cool it.
@nixCraft Holy crap. Thats a lot of energy. I just charged my 2018 Tesla Model S 75D the other day at home (220v/40amp) and according to TeslaFi I went from 6%-90% in 6.5 hours and pulled 60kW from the wall. The amount this needs is jaw dropping.
@nixCraft 120 kW is roughly the power output of a small automobile, which is to say, a fuсking lot of power given the still-relatively primitive state of artificial intelligence. And that’s just for a single rack. Whatever purposes they are being used for likely uses hundreds or thousands of these.
We badly need alternative computing substrates that consume a fraction of the energy of digital computers if we want this to scale up without also destroying the planet.
@nixCraft Hopefully they're at least using the waste heat to do something useful. That way having a nice, hot shower can be a happy side effect of plagiarism.
@nixCraft and even with that power it will still be a little more than an advanced auto complete with hallucinations. I really think that some companies are promoting high power usage ML as a way to expand their clouds with new features, now that many companies are getting out of more traditional offerings. The kind of money spent on energy would be better spent in a variety of better places including actual research.
@nixCraft why is cooling water lost? This puzzles me. You could cool your machine, thereby warming the water without adding anything to it, and then dump the warm water back, right?
@nixCraft Agree. Who needs the Internet when we have fax machines!!!1
I'm sorry, but this is just a horrible take. No content is stolen, we're putting people out of a job as much as refridgerators changed the ice-selling business and these datacenters have a small carbon footprint compared to the big stuff we can (and are) going after.
New technology is usually good for humanity. Look at the smog filled skies of LA from the 70's. LLMs will either find a good usecase or they will not.
Add comment