Another fantastic GTC draws to a close, and this year was bigger, bolder and better than ever
The curtains have closed on another year of the GPU Technology Conference (GTC24) in San Jose, and Boston played its part in doing the dynamic world of high-performance computing proud. Our journey through GTC24 was one to be remembered, as we explored groundbreaking products, experienced the latest innovations from NVIDIA and formed connections with industry peers and partners.
At the heart of our presence at GTC24 was our booth, a bustling hub of activity where attendees gathered to witness firsthand the latest advancements in technology. Partnering with industry giants such as Supermicro and ZutaCore, we had an impressive array of products on display that offered a glimpse at the future of computing. From Supermicro's next-gen servers to ZutaCore's revolutionary cooling solutions, our booth was a testament to our commitment to pushing the boundaries of innovation.
We were also delighted to host SECQAI, adding a touch of quantum computing expertise to our exhibit. Their presence sparked engaging discussions and offered attendees an opportunity to peer into the transformative potential of quantum technology. There was some incredible tech on show throughout this year’s GTC, look at Boston’s stand outs, below!
The Blackwell is NVIDIA’s newest platform for building and deploying Trillion-Parameter Scale Generative AI, named in honour of David Harold Blackwell - a mathematician who specialised in game theory and statistics, and the first Black scholar inducted into the National Academy of Sciences.
Relative to its predecessor – Hopper, Blackwell achieves: 30X more AI Inference Performance, 4X faster AI Training, 25X lower energy use, and 25X lower TCO. It introduces groundbreaking advancements for accelerated computing through six technological innovations:
- AI Superchip: 208B Transistors built from two of the largest possible die.
- 2nd Gen Transformer Engine: Enabling FP4/FP6 Tensor Core for double the performance and model size.
- 5th Generation NVLink: Scales up to 576 GPUs in a single NVLink domain.
- RAS Engine: 100% In-System Self-Test for reliability for GPU at scale AI.
- Secure AI: Full Performance Encryption & TEE, Confidential compute without compromise.
- Decompression Engine: 800GB/sec of performance to compute on compressed data without CPU decompression time.
Blackwell GPUs are available in three system configurations: GB200 NVL72, HGX B200, and HGX B100 and will be available later this year.
For AI inference, the GB200 NVL72 achieves 30X more performance, 25X less energy, and 25X less cost than Hopper for trillion-parameter models. The HGX B200 achieves 15X more performance,12X less energy, and 12X less cost.
For AI training, the GB200 NVL72 achieves 4X more performance, 3.5X less energy, and 3.5X less cost than Hopper for trillion-parameter models. The HGX B200 achieves 3X more performance, 2.5X less energy, and 2.5X less cost.
For data processing, the GB200 NVL72 achieves 18X more performance, 7X less energy, and 5X less cost than CPUs for simulated database workloads. The HGX B200 achieves 6X more performance and 2X less energy.
NVIDIA’s next generation NVLink rack scale architecture for Trillion Parameter Scale AI
It consists of 36 GB200 Grace Blackwell Superchips each with two Blackwell GPUs connected to one Grace CPU for a total of 72 Blackwell GPUs in a single NVLink domain. GB200 NVL72 is a multi-node, liquid-cooled, rack-scale system for the most compute-intensive workloads. The platform acts as a single GPU with 1.4 exaflops of AI performance and up to 30TB of fast memory and is a building block for the newly announced DGX SuperPOD. To achieve maximum compute density, the design uses liquid-cooling.
- GB200 Superchip Compute Nodes: Contains single or dual GB200 Superchips, PCIe gen 6 slots, and includes NVIDIA BlueField®-3 DPUs for cloud network acceleration, composable storage, zero-trust security, and GPU compute elasticity in hyperscale AI clouds. 18 compute nodes fit in a GB200 NVL72 rack.
- NVLink Switch Tray: Contains two NVLink Switch Chips per switch tray in GB200 NVL72. Nine switch trays in GB200 NVL72 provide 130TB/s total bandwidth for GPU communications. 72 switch trays create a 576 GPU NVLink domain with 1PB/s total bandwidth.
- NVLink Switch Chip: Creates the highest performance compute fabric for large AI and HPC models. Provides 72 NVLink ports at 100GB/s per chip, encryption for 128 GPUs in a Confidential Computing domain, and is managed by Unified Fabric Manager.
NVIDIA Blackwell Systems: HGX B200 & HGX B100
The premier accelerated x86 scale-up platforms designed for the most demanding generative AI, data analytics, and high-performance computing (HPC) workloads.
- HGX B200 delivers the best performance (15X more AI inference perf than HGX H100) and TCO (12X less than HGX H100) for x86 scale-up platforms and infrastructure.
- HGX B100 is designed for the fastest time to deployment with drop-in replacement compatibility for existing HGX H100 infrastructure.
DGX SuperPOD with DGX GB200 systems
A liquid-cooled, rack-scale AI infrastructure based on the NVIDIA GB200 NVL72 platform that scales to thousands of NVIDIA GB200 Grace Blackwell Superchips for training and inferencing trillion parameter generative AI models.
- High compute density, liquid-cooled, rack-scale design.
- 36 GB200 Grace Hopper Superchips with 36 Grace CPUs and 72 Blackwell GPUs per rack, connected via fifth generation NVLink.
- Scalable to thousands of GB200 Superchips with InfiniBand.
- Full-stack resiliency features to deliver constant uptime.
This entirely new type of DGX SuperPOD features massive GPU memory and NVIDIA Grace CPUs, built on the ARM CPU architecture. It is a high compute density, liquid-cooled, rack-scale design. Racks are connected via NVIDIA Quantum-2 InfiniBand for DGX SuperPODs that can scale up to tens of thousands of GB200 Superchips. An optional configuration that connects 576 Blackwell GPUs with NVLink will also be available to power massive models.
NVIDIA DGX B200
The latest iteration of NVIDIA’s legendary DGX systems powered by NVIDIA Blackwell GPUs. DGX B200 is the proven choice for enterprises, a unified AI platform that accelerates any AI workload, from training, to fine-tuning, to inference.
- Features 8 NVIDA Blackwell GPUs with a total of 1,440GB of GPU memory and two 5th Gen Intel Xeon 8570 CPUs.
- 3X more training performance, 15X more inference performance compared to DGX H100.
- 4x OSFP ports serving 8x single-port NVIDIA ConnectX-7 400Gbp/s InfiniBand and Ethernet, and 2x dual-port NVIDIA BlueField-3 DPUs.
- DGX B200 systems are the building blocks of the next-generation NVIDIA DGX POD and NVIDIA DGX SuperPOD.
- Is air-cooled and occupies 10 rack units in a standard server rack.
NVIDIA Quantum-X800 and Spectrum-X800
The Quantum-X800 and Spectrum-X800 platforms are the next stage of InfiniBand and Ethernet evolution to support, respectively, the highest performance for AI, and optimised performance for AI cloud and enterprise.
The Quantum-X800 platform includes the NVIDIA Quantum Q3400 switch and NVIDIA ConnectX-8 SuperNIC to achieve end-to-end throughput of 800 Gb/s and provides SHARPv4 for 5x higher bandwidth capacity and 9x greater in-network computing compared to the previous generation.
The Spectrum-X800 platform includes the NVIDIA Spectrum SN5600 800 Gb/s switch and NVIDIA BlueField-3 SuperNIC optimised for multi-tenant AI cloud and enterprise infrastructure to facilitate tenant-isolation of AI workloads for optimal and consistent performance.
The experienced networking team at Boston are extremely excited about the launch of the X800 series and are keen to begin building out even more performant networks that are truly optimised for AI and HPC workloads.
NVIDIA AI Enterprise 5.0
NVIDIA AI Enterprise 5.0 is a major release update which introduces several new features and enhancements. Chief amongst these is the introduction NIM Microservices providing optimised runtime and building blocks for AI development – more on that later.
Alongside the introduction of NIM, support has been introduced for the following new GPUs:
- NVIDIA H800 SXM5 80GB
- NVIDIA RTX 5880 Ada
- NVIDIA GH200 96GB (CG1) Grace Hopper™ Superchip
- NVIDIA GH200 144GB (CG1) Grace Hopper Superchip
As well as adding vGPU support for the following models which were previously bare metal only:
- NVIDIA H800 SXM5 80GB
- NVIDIA RTX 5880 Ada
- NVIDIA GH200 96GB (CG1) Grace Hopper™ Superchip
- NVIDIA GH200 144GB (CG1) Grace Hopper Superchip
NVIDIA NIM
NIMS or NVIDIA Inference Micro Services to give it the full name is possibly one of the most interesting announcements made by Jenson at GTC ’24. It’s been revealed that a catalogue of NIMS will enable as many as two dozen popular pre-trained and optimised AI models will be provided either directly by NVIDIA or by its partner network via prebuilt containers.
Partners include,?A121, Adept,?Cohere, Getty Images, and Shutterstock as well as open models from Google,?Hugging Face, Meta, Microsoft, Mistral AI and Stability AI.
Customers with NVIDIA AI Enterprise 5 licenses will be able to access these as part of their existing license subscription, helping to minimise their time to take AI into production. NIMS provide industry-standard APIs for domains such as language, speech and drug discovery to enable developers to quickly build AI applications using their proprietary data hosted securely in their own infrastructure.
Naturally, Boston will be enabling and supporting our customers deploying NIMS on our wide range of NVIDIA-Certified systems, stay tuned or enquire to hear more about this exciting way to deploy AI models with ease.
NVIDIA Quantum Cloud
Nvidia launched a new cloud-based platform - Nvidia Quantum Cloud, during Jenson’s GTC keynote. It’s designed to assist researchers, developers and organisations to access quantum computing clusters and quantum hardware through cloud-based platform which are fully managed and supplied by Nvidia, simplifying the whole process.
Nvidia also offers Quantum Development Kits which includes tutorials, documentation and example code to assist developers on their quantum computing journey. Using Nvidia Quantum Cloud, various use cases around quantum machine learning (such as quantum support vector machine and principal component analysis) and quantum neural networks (such as Image Classification) can be run.
Overall, Nvidia Quantum Cloud is a powerful platform for quantum computing and machine learning that can be used in a variety of applications, including cryptography, optimisation, and simulation.
Boston Labs Quantum Centre of Excellence
Complimenting NVIDIA Quantum Cloud, here at Boston we have a similar approach for developing quantum computing projects in house. Our quantum centre of excellence, an extension of Boston Labs is hosted in state-of-the-art datacentre - Cyxtera LHR3. Similalrly, this lab is well equipped with computing resources and tools are designed for quantum computing in particular. It includes a wide range of quantum software tools and libraries such as Cirq, Qiskit and Pennylane which are super useful to execute quantum algorithms, optimise quantum circuits and quantum simulations.
This cloud-based platform is equipped with QPUs (Quantum Processing Units), GPU and CPU resources, meeting the needs of heterogenous computing. Boston Labs also includes dedicated platforms for quantum computing which are equipped with GPUs including RTX6000, L40S & H100, the latest generation CPUs from NVIDIA, AMD and Intel, pre-configured and optimised with all the required software dependencies.
Just like Nvidia Quantum Cloud, Boston Labs is designed to be accessible in user friendly fashion and open to all researchers, developers, and organisations of all sizes. Practitioners can experiment with quantum computing resources and hardware in very cost-effective ways and can avoid the need to invest in often expensive hardware and other infrastructure.
Paired with the Centre of Excellence, there is also Boston Training Academy, enabling education on Quantum and other computing paradigms for a wider audience. The academy covers tutorials around evolving quantum technology as well as traditional GPU and CPU computing methodologies.