by 
Nvidia, a leading player in the AI hardware space, has unveiled its next-generation AI chip platform. The platform is powered by the new Grace Hopper Superchip, which is the world’s first to incorporate the HBM3e processor. This cutting-edge technology is designed to handle heavy generative AI workloads, such as Large Language Models, similar to ChatGPT. It’s also capable of managing other AI applications like recommender systems and vector databases.
The dual configuration of the platform offers 3.5 times more memory and 3 times more bandwidth than its predecessor. It’s equipped with a single server that boasts 144 Arm Neoverse cores, eight petaflops of AI performance, and 282GB of HBM3e memory technology.
Nvidia CEO, Jensen Huang, highlighted the enhancements that the new platform brings. These include improved throughput, the ability to connect GPUs to aggregate performance without compromise, and a server design that can be easily deployed across an entire data center. The Grace Hopper Superchip can be linked with other Superchips via Nvidia’s NVLink to boost the computing power necessary to deploy large-scale generative AI models.
This connection allows the GPU to fully access the CPU memory, providing a combined 1.2TB of memory in dual configuration mode. The new HBM3e memory is 50% faster than HBM3, offering a total of 10TB/sec of combined bandwidth. This allows it to run models that are 3.5 times larger, with performance further enhanced by the 3 times greater bandwidth. The Grace Hopper Superchip platform with HBM3e is fully compatible with the Nvidia MGX server specification.
This compatibility allows any manufacturer to integrate Grace Hopper into over a hundred server variations quickly and cost-effectively, according to Nvidia. Nvidia’s A100 GPUs were previously used to power the machines behind ChatGPT, the chatbot that paved the way for advanced automated computing. The company followed this up with its successor, the H100. The announcement of the new platform underscores Nvidia’s continued dominance in the AI hardware space. Nvidia expects system makers to produce the first models based on the Grace Hopper platform in the second quarter of 2024.
