Unleashing the Power: Exploring the Exciting Features of the AMD Instinct MI325X 8-GPU Baseboard!

Over the years, I have likely dismantled more than two dozen GPU servers. Just this week in Taipei, I have disassembled at least two AMD Instinct MI300X / MI325X configurations, along with several other intriguing servers. I eventually posed a question that I felt embarrassed to simply ask. What is this component on these large GPU baseboards?

This is an Engaging AMD Instinct MI325X 8-GPU Baseboard Feature

For a brief recap, the AMD Instinct MI300X features 192GB of HBM3 memory and is designed in an OAM form factor. Eight of these units are present on a Universal Baseboard (UBB), and this setup is somewhat comparable to a NVIDIA HGX 8 GPU platform, though NVIDIA incorporates the NVLink Switch architecture on its HGX 8 GPU setups.

The AMD Instinct MI325X is the HBM3e revision featuring 256GB of HBM3e memory, offering increased capacity and higher memory bandwidth. It is presented on a very analogous platform. At the forefront of the platform’s extensive heatsink array lies a large black metal component.

On the opposite side, there is yet another one of these metal towers encasing the PCIe retimer heatsink.

Today, we had to replace an AMD Instinct MI325X 8-GPU baseboard, and these components are quite hefty. The shipping container with padding weighs approximately 39.5kg or 87lbs. Those dark metal loops, as I was informed, serve as handles to facilitate the movement of the substantial assembly.

Following my astonishment at a man who swiftly utilized these grips to replace the entire UBB in less than 90 seconds, I felt compelled to confirm that I had indeed witnessed these in setups before without recognizing them. Here is a reasonably clear image from the 2021 Inspur NF5488A5 8x NVIDIA A100 HGX Platform Review, where the metallic handle is visible as well.

What is amusing is that I have spotted these handles for several years across numerous systems and simply never recognized their purpose.

Concluding Remarks

Contemporary 8-GPU configurations from AMD and NVIDIA are substantial since they necessitate a significant amount of metal in their heat sinks. Even the liquid-cooled coldplate variants may appear smaller, but the copper blocks are frequently quite weighty. Utilizing these handles simplifies the management of these pricey motherboards considerably.

Perhaps as we enter the new year, there’s a meaningful lesson here: always feel free to pose a question, even if it seems trivial. The aim of this post is to illustrate that if I had this query, chances are that many others might have a similar one.