Comments Locked

40 Comments

Back to Article

  • ballsystemlord - Monday, June 27, 2022 - link

    So, how about liquid nitrogen cooling? You could create micro-through-chiplet and inter-chiplet vias to allow the liquid nitrogen to flow through the chips and chiplet interconnects to cool them down.
    Potentially, the spaces between chiplets (created by the micro-solder-bumps), could be utilized as pathways without the need for making any vias at all.
  • ingwe - Monday, June 27, 2022 - link

    I suspect the phase change would create significant problems given the small size we are talking about. I don't know though and it is an interesting idea.
  • Wereweeb - Monday, June 27, 2022 - link

    That's an entirely different can of worms, and an expensive one at that.
  • ChefJeff789 - Monday, June 27, 2022 - link

    An interesting idea, but capillary action prevents fluids from travelling through extremely small openings unless there is a force acting to pull the fluid through each cavity. To keep it practical, you'd need one inlet and one outlet (or at least one inlet region and one outlet region, if you wanted to feed multiple channels), so you need to create an explicit path through the chip for the fluid to follow. Perhaps doable, if you make the channel part of the chip stack during manufacturing, but difficult. Also, if even one region ends up with an air pocket, you could get some really harsh effects. I suspect cavitation would completely destroy a chip in seconds, if the thermal gradient from air-fluid doesn't do it first.
  • xol - Monday, June 27, 2022 - link

    active cooling aka refrigeration is going to add an additional cost (energy to refrigerate), whilst liquid cooling can just use passive radiators to lose the heat to the environment.

    that being said refrigeration for chips *could* be interesting since it low temps will drastically reduce wire resistance (even though most of chip power is lost in switching not resistance) ..

    I think another challenger is thermal coefficients of expansion/contraction .. room temp to 99C is around 75 degrees difference. room temp to liquid N2 is ~225degrees - and this is exacerbated by materials becoming more brittle at lower temps.(eg solder joints)
  • xol - Monday, June 27, 2022 - link

    [edit] above "challenge" not "challenger" (typo)
  • meacupla - Monday, June 27, 2022 - link

    AFAIK, the preferred method for cryogenic cooling is pumping liquid helium through a piping system of some sort.
    The downside with both liquid nitrogen and liquid helium, is that they are both cryogenic forms of cooling, and will create a lot of condensation/ice around whatever metal is exposed to the room.
  • Foo Barred - Monday, June 27, 2022 - link

    Water / fluid cooling is about transferring the heat to someplace where it can be removed/cooled, possibly in a more effective manner than with a local heat-sink and airflow.

    Liquid nitrogen is a very effective cooling solution but it consumes the nitrogen in the process and removes(cools) rather than transferring the heat. I.e. very effective for setting over clocking records, but not practical at scale.
  • ballsystemlord - Monday, June 27, 2022 - link

    I mean that you could use a closed loop system. Thus you wouldn't "consume" the nitrogen.
  • Kevin G - Tuesday, June 28, 2022 - link

    The problem is that the liquid nitrogen becomes a gas when it comes into contact with the chip heating it up. To form a closed loop, you'd have to have a condenser/compressor in the loop to convert the nitrogen gas back into its liquid form. Due to inefficiencies, this also takes energy to do. Not impossible to pull off but very very impractical.
  • Foo Barred - Tuesday, November 22, 2022 - link

    And in that case you are talking about an active (refrigerator) and you would then do much better with the well-known and engineered refrigerants rather than nitrogen or helium.
  • meacupla - Tuesday, June 28, 2022 - link

    We already have R-134A for such applications.
    It's the stuff used in ACs and refrigerators.
  • ballsystemlord - Wednesday, June 29, 2022 - link

    You could use that instead. I just thought liquid nitrogen would give you the biggest cooling per cubic mm/in.
  • byte99 - Tuesday, June 28, 2022 - link

    Liquid nitrogen is useful for getting things to really cold temperatures. It's less useful for maintaining objects with high thermal output at moderate temperatures. Liquid nitrogen has decent heat capacity--about half that of water. But you'd need liquid nitrogen to be significantly below its 77K boiling point to have the capacity to carry away thermal energy. If it's close to its boiling point, then when it comes into contact with a heat source it can easily turn to gas, because it won't take much thermal energy to get it to its boiling point, and furthermore won't take much thermal energy to get it to boil (its heat of vaporization is only 1/10 that of water).
  • ballsystemlord - Wednesday, June 29, 2022 - link

    Good point!
  • SarahKerrigan - Monday, June 27, 2022 - link

    Why did they use a SPARC T5 die shot, seemingly from the Hot Chips presentation, for their ">50B transistors" graphic? What a strange choice.
  • shabby - Monday, June 27, 2022 - link

    Greta: how dare you!
  • mode_13h - Monday, June 27, 2022 - link

    Well yeah, but maybe no. If you calculate a complete carbon footprint, it would account for production, maintenance, and time in service. There are also computational overheads when you try to scale to more nodes, with some algorithms experiencing a bigger hit on efficiency than others.

    So, it's not inconceivable that running fewer processors at higher wattage could actually have a lesser carbon footprint than if you'd run more of them at lower wattage, or do upgrades every 2 years instead of every 3 years, etc.

    Politicians like to ban things like incandescent light bulbs or 2-stroke engines, but that's too narrow and could prevent the best solution from being available in niche cases where nothing else would work as well. It could even prevent advancement of said technologies which could alleviate their deficiencies. Instead, they should really focus on carbon pricing, green infrastructure, and green R&D.

    If carbon is priced appropriately or the energy is 100% renewable, then there's nothing inherently wrong with a 1 kW CPU. It might even power more climate simulations or the very R&D leading towards more efficient solar panels.
  • mode_13h - Monday, June 27, 2022 - link

    That said, it's undeniable there'd be less demand for such high-power CPUs, if carbon were priced appropriately. I'm just saying to focus on the energy market & infrastructure, but let the market decide how to make best use of energy available under those constraints.
  • FunBunny2 - Tuesday, June 28, 2022 - link

    "let the market decide how to make best use of energy available under those constraints."

    that's a nice idea (straight out of 1776 and Adam Smith), but it assumes what we all know is false: that the market is free and balanced. what we have, esp. in IT, is highly concentrated, and thus less than a handful of players in any one segment with vast market controlling power. dat won't get you efficient use of resources. just look at fab citing: yet more in the arid SW, losing yet more water as the days go by.
  • xol - Monday, June 27, 2022 - link

    [carbon footprint] I'm just going to leave this here :

    https://www.dell.com/en-us/dt/corporate/social-imp... .. pick any

    Generally the estimate is 85% impact during use, most of the remainder from manufacturing - this skews favor towards using the most efficient chips, and the lowest clocks..

    There's also some alternate "life cycle assesments" (see LCA PowerEdge R6515 R7515 R6525 R7525) on the same page which give a slightly lower figure. It's clear from those alt links that 'silicon' is the vast majority of the impact from manufacture, and than includes both CPU, RAM and SSD -in fact SSD is typically >50%, RAM >25%, with Motherboard+CPU only 15%.

    These figures kind of suprised me, but make sense having thought about it.

    My conclusion would be that these figures make low clocks (ie low wattage CPU/thread) a no-brainer for carbon efficiency.

    imo it's easier to de-carbonize server electrical supply than the whole supply/energy chain of chip manufacture - if that is an aim.
  • mode_13h - Monday, June 27, 2022 - link

    > Generally the estimate is 85% impact during use

    They don't account for maintenance of the hardware - just making, powering, and disposing of it.

    > these figures make low clocks (ie low wattage CPU/thread) a no-brainer for carbon efficiency.

    Again, you assume the workload scales to more cores, efficiently. Some algorithms don't.

    The subject is HPC, not hyperscalers. If hyperscalers started using 1 kW processors, I'd be worried.
  • meacupla - Monday, June 27, 2022 - link

    It's not just carbon footprint. When it comes to data centers, it's also water consumption.

    Google, for instance, doesn't want you, the public, to know how much fresh water they are consuming from the Colorado River, just to keep their data centers from overheating.
    And the Colorado River is not exactly doing well when it comes to plentiful water.
  • mode_13h - Tuesday, June 28, 2022 - link

    Hopefully, they just use that water in a closed loop, in which case there shouldn't really be any ongoing consumption.

    If they're dumping their waste heat as hot water downstream, someone should shut them down. Even worse would be simply evaporating the river water to cool their data center. Let's hope it's neither of those.
  • meacupla - Tuesday, June 28, 2022 - link

    No, it's worse than that.
    Google is using evaporative cooling for their data centers sitting along the Colorado River. None of the water that gets used up goes back into the river.

    All the water is wasted, and gets turned into very local high humidity.
  • mode_13h - Monday, July 4, 2022 - link

    That's hard to believe.
  • xol - Monday, June 27, 2022 - link

    [more] there's limited info on carbon impact related to node size .. though this IMEC blog has some data

    https://www.imec-int.com/en/articles/environmental...

    Though numbers get bigger as nodes get smaller up to 5nm they are not increasing faster than energy efficiency - which is good. At 3/2nm that may not be true but that info must be an estimate (?)
  • mode_13h - Monday, June 27, 2022 - link

    I have a fantasy that CPUs will one day make the package into a vapor chamber, instead of using solder or thermal paste to conduct heat between the die and heat spreader. The benefit should be much more even heat distribution across the heat spreader, as well as better thermal conductivity between the dies and the heat spreader.
  • Kevin G - Tuesday, June 28, 2022 - link

    The challenge I see would be maintaining pressurization over time. Bonding the heat spreader to the substrate currently is not done to maintain a seal, much less at the pressures involved for a pure vapor chamber. Not a bad idea but it may require going back to ceramic packaging to pull it off due to how flexible substrates are currently.

    There is one added bonus to this though: you can have silicon stacks of different heights be cooled efficiently. Creating optimal heat spreaders requires some very precise machining which is rare. The more common solution being silicon shims to level off the stacks with material having the same thermal properties. Moving to a vapor chamber heat spreader removes those issues. I would be worried that the material in the vapor chamber is conductive and could get in between the silicon stacks.

    This sounds like one of those wonderful engineering problems.
  • Doug_S - Monday, June 27, 2022 - link

    There should be a decent energy efficiency gain using liquid cooling, as the power required for fans for servers, evaporators and high velocity movement of air inside the datacenter can be greatly reduced (depending on what remains to be cooled via forced air) with almost all power consumed for pumps and compressors only.
  • HappyCracker - Tuesday, June 28, 2022 - link

    I work in the enterprise data center space. This is a true statement. We've tested rack-scale liquid cooling for servers (basically the same as your home PC with a centralized manifold. It works well and does drop the overall energy consumption of the system (those fans draw some power). The other advantage is that you can move the heat to a more convenient location compared to the traditional hot/cold aisle approach.
  • FunBunny2 - Tuesday, June 28, 2022 - link

    "There should be a decent energy efficiency gain using liquid cooling"

    would be nice if it were true, but the Laws of Thermodynamics make it impossible.
  • davedriggers - Tuesday, June 28, 2022 - link

    Direct to Chip cooling leveraging water can easily handle these power levels. Direct to Chip has been leveraged for over a decade and at these power levels is actually fairly inexpensive, relative to the cost of the processors being created. The 700 watt Nvidia H100 will cost over $30,000 each, even if the cooling solution costs $1/watt (should be lower on scale), that would be just over a 2% premium. Direct to Chip cooling is also extremely efficient on scale, which would likely make up in operational costs as well as improving reliability and performance. The majority of the OEMs are planning on supporting Direct to Chip for the H100 GPUs, so we should start to see more off-the-shelf solutions entering the market vs. custom one off cooling systems fairly soon.
  • mode_13h - Monday, July 4, 2022 - link

    By "direct to chip" water cooling, I take it you mean having a waterblock on each processor?
  • davedriggers - Tuesday, July 5, 2022 - link

    Yes vs. liquid to the rack i.e. rear door heat exchanger.
  • Jp7188 - Wednesday, June 29, 2022 - link

    I can see a day when it becomes the norm to install a geo loop under the datacenter foundation at time of construction. The ground can absorb a lot of heat and direct cooling of water pipes in this fashion is very efficient.
  • mode_13h - Monday, July 4, 2022 - link

    > The ground can absorb a lot of heat

    Well, the thermal gradient should stabilize over time, eventually resulting in less efficient heat disposal. If the datacenter is located somewhere that gets cold, then you could get rid of some built-up heat during the winter. That's the only way it makes much sense to me.
  • alpha754293 - Thursday, June 30, 2022 - link

    Direct die evaporative cooling is nothing new.

    My college housemates and I have been talking about that since AT LEAST as early as 2007.

    Back then, I was already trying to design my own custom waterblock to water cool/liquid cool a Socket940 AMD Opteron, as a 3rd year mechanical engineering student.

    This is nothing new.
  • ravikrieg - Wednesday, July 20, 2022 - link

    Transistor power consumption isn't scaling down nearly as quickly as transistor sizes.
    https://wikiguide.tips/yahoo-mail-sign-up

Log in

Don't have an account? Sign up now