NVIDIA H100 PCIe vs. SXM5 Form Factors

Which GPU is right for your company?

Erik Kimmerer
February 27, 2023
< Blog Home

As artificial intelligence and machine learning continue to gain popularity, companies across many industries are investing in the latest and greatest GPU servers to enhance their data science and AI capabilities. With NVIDIA being the leading player in the GPU market, it’s challenging to determine which NVIDIA GPU server is suitable for your company. In this blog post, I will compare the PCIe and SXM5 form factors for NVIDIA H100 GPUs, the highest-performing GPUs currently available, and contrast performance and costs to help you make an informed decision.

PCIe vs. SXM5

NVIDIA H100 GPUs are available in two form factors: PCIe and SXM5. PCIe GPUs can be installed into standard PCIe slots on a motherboard, while SXM5 GPUs require a specialized form factor incompatible with standard PCIe slots. PCIe GPUs are easier to install since they don't require specialized hardware or connectors. They are also easier to replace or upgrade since you can simply remove the GPU, unlike the SXM5, which is attached to the chassis via thermal paste. Thanks to NVSwitch, SXM5 GPUs offer higher performance over PCIe GPUs, making them ideal for multi-node data-intensive workloads.

NVLink vs. NVSwitch + InfiniBand

NVLink is a direct GPU-to-GPU interconnect that scales multi-GPU input/output (IO) within the server and is available in both form factors. Exclusive to SXM5, NVSwitch connects multiple NVLinks to provide all-to-all GPU communication at full NVLink speed within a single node. InfiniBand enables NVSwitch connections to be extended across nodes to create a seamless, high-bandwidth, multi-node GPU cluster - effectively forming a data center-sized GPU, making it possible to solve even the most extensive AI jobs rapidly.

NVIDIA H100 PCIe with NVLink GPU-to-GPU connection + NVIDIA H100 SXM5 with NVSwitch NVLink-to-NVLink connection
NVIDIA H100 PCIe with NVLink GPU-to-GPU connection + NVIDIA H100 SXM5 with NVSwitch NVLink-to-NVLink connection

Performance and Costs

The NVIDIA H100 GPU outperforms its predecessor, the A100, by up to 10x for AI workloads. The SXM5 GPU raises the bar considerably by supporting 80 GB of fast HBM3 memory, delivering over 3 TB/sec of memory bandwidth, effectively a 2x increase over the memory bandwidth of the A100, which launched just two years prior. The PCIe H100 provides 80 GB of fast HBM2e with over 2 TB/sec of memory bandwidth.

H100 enables next-generation AI and HPC breakthroughs
H100 enables next-generation AI and HPC breakthroughs

Conclusion

Choosing the correct NVIDIA H100 GPU server for your company requires careful consideration between performance and cost. The PCIe form factor is easier to install and upgrade, while the SXM5 form factor offers higher performance and NVSwitch/InfiniBand for building optimized multi-node clusters. The NVIDIA H100 GPU server offers unparalleled performance compared to its predecessors, making it the ideal choice for data-intensive workloads. The specific requirements of your company’s workloads will determine the final decision of the form factor that’s suitable for you.

Looking to learn more about which H100 is right for your workloads?

Book a free consultation with one of our GPU experts.

Upgrade Your Infrastructure with NVIDIA H100 Servers

4 GPUs
Borealis H100 SXM5 5U
Borealis H100 Server - 5U
Does Have Feature
4 x NVIDIA H100 (80GB) SXM5 GPUs
Does Have Feature
2 x Intel® Xeon® Platinum 8458P Processor
Does Have Feature
3-year Hardware Coverage
Does Have Feature
NVLink + NVSwitch GPU connectivity
Does Have Feature
2,048 GB of system memory
Does Have Feature
2 x 4 TB SSD
Starting at
$225,000
Learn More
8 GPUs
Borealis H100 SXM5 8U
Borealis H100 Server - 8U
Does Have Feature
8 x NVIDIA H100 (80GB) SXM5 GPUs
Does Have Feature
2 x Intel® Xeon® Platinum 8458P Processor
Does Have Feature
3-year Hardware Coverage
Does Have Feature
NVLink + NVSwitch GPU connectivity
Does Have Feature
2,048 GB of system memory
Does Have Feature
2 x 4 TB SSD
Starting at
$314,900
Learn More