NVIDIA H100 PCIe vs. SXM5 Form Factors

Which GPU is Right for Your Company?

Erik Kimmerer
Sales & Marketing Specialist
February 27, 2023

NVIDIA H100 PCIe vs. SXM5 Form Factors

As artificial intelligence and machine learning continue to gain popularity, companies across many industries are investing in the latest and greatest GPU servers to enhance their data science and AI capabilities. With NVIDIA being the leading player in the GPU market, it’s challenging to determine which NVIDIA GPU server is suitable for your company. In this blog post, I will compare the PCIe and SXM5 form factors for NVIDIA H100 GPUs, the highest-performing GPUs currently available, and contrast performance and costs to help you make an informed decision.

PCIe vs. SXM5 Form Factors

NVIDIA H100 GPUs are available in two form factors: PCIe and SXM5. PCIe GPUs can be installed into standard PCIe slots on a motherboard, while SXM5 GPUs require a specialized form factor incompatible with standard PCIe slots. PCIe GPUs are easier to install since they don't require specialized hardware or connectors. They are also easier to replace or upgrade since you can simply remove the GPU, unlike the SXM5, which is attached to the chassis via thermal paste. Thanks to NVSwitch, SXM5 GPUs offer higher performance over PCIe GPUs, making them ideal for multi-node data-intensive workloads.

NVLink vs. NVSwitch + InfiniBand

NVLink is a direct GPU-to-GPU interconnect that scales multi-GPU input/output (IO) within the server and is available in both form factors. Exclusive to SXM5, NVSwitch connects multiple NVLinks to provide all-to-all GPU communication at full NVLink speed within a single node. InfiniBand enables NVSwitch connections to be extended across nodes to create a seamless, high-bandwidth, multi-node GPU cluster - effectively forming a data center-sized GPU, making it possible to solve even the most extensive AI jobs rapidly.

NVIDIA H100 PCIe with NVLink GPU-to-GPU connection + NVIDIA H100 SXM5 with NVSwitch NVLink-to-NVLink connection
NVIDIA H100 PCIe with NVLink GPU-to-GPU connection + NVIDIA H100 SXM5 with NVSwitch NVLink-to-NVLink connection

Performance and Costs

The NVIDIA H100 GPU outperforms its predecessor, the A100, by up to 10x for AI workloads. The SXM5 GPU raises the bar considerably by supporting 80 GB of fast HBM3 memory, delivering over 3 TB/sec of memory bandwidth, effectively a 2x increase over the memory bandwidth of the A100, which launched just two years prior. The PCIe H100 provides 80 GB of fast HBM2e with over 2 TB/sec of memory bandwidth.

H100 enables next-generation AI and HPC breakthroughs
H100 enables next-generation AI and HPC breakthroughs

In terms of costs, the PCIe H100 GPU server is more affordable compared to the SXM5 form factor. Here’s a breakdown of the H100 servers that Arc Compute has available:

H100 PCIe Servers:

2 x H100 GPUs (4U) - $106,000

4 x H100 GPUs (4U) - $172,000

8 x H100 GPUs (4U) - $279,000

H100 SXM5 Servers:

4 x H100 GPUs (5U) - $197,000

8 x H100 GPUs (4U) - $310,000

These prices are for servers featuring 2 x Intel® Xeon® Platinum 8458P Processors, 1,024-2,048 GB of system memory, 2 x 4 TB SSD, and 3-year hardware coverage. All servers are customizable to meet your specific use case.

Conclusion

Choosing the correct NVIDIA H100 GPU server for your company requires careful consideration between performance and cost. The PCIe form factor is easier to install and upgrade, while the SXM5 form factor offers higher performance and NVSwitch/InfiniBand for building optimized multi-node clusters. The NVIDIA H100 GPU server offers unparalleled performance compared to its predecessors, making it the ideal choice for data-intensive workloads. The specific requirements of your company’s workloads will determine the final decision of the form factor that’s suitable for you.

Looking to learn more about Arc Compute?
Read our latest white papers and case studies.
GVM Server - 100% Utilization POC
The following results are from tests we ran to demonstrate the performance benefits and limitations of GVM Server, which provides a way forward for further proof of concept tests within your organization’s infrastructure.
Thank you for your submission!
Read Now
Oops! Something went wrong while submitting the form.
GVM Server - Organization-Level Provisioning with Nested Roles
Organization-level provisioning is a nested roles feature that allows organizations to manage data and resources for teams/projects hierarchically.
Thank you for your submission!
Read Now
Oops! Something went wrong while submitting the form.
GVM Server - Solution Brief
GVM Server is Arc Compute's GPU/CPU hypervisor which is an all-in-one GPU utilization and virtualization solution.
Thank you for your submission!
Read Now
Oops! Something went wrong while submitting the form.
Arc Compute - Company Summary
Arc Compute's customers have one thing in common; they are all large consumers of GPUs who are tired of the current cloud business models and are looking for better, transparent pricing and better performance and security.
Thank you for your submission!
Read Now
Oops! Something went wrong while submitting the form.
Arc Compute Powers GPU Cloud Offering with Liqid
"Arc Compute, the only cloud service provider to offer Liqid’s revolutionary composable disaggregated infrastructure (CDI) as a service, proposed a GPU cloud option that offered the immersive video company a far more flexible and cost-effective solution".
Thank you for your submission!
Read Now
Oops! Something went wrong while submitting the form.
GVM Server - Superior GPU Utilization and Performance
As you will see in the following benchmarks, by utilizing GVM Server, your workloads can train up to 80% faster thanks to improved utilization of GPU resources.
Thank you for your submission!
Read Now
Oops! Something went wrong while submitting the form.
Connect with us
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.

The Arc Blog

Arc Compute: a custom GPU cloud provider
February 27, 2023
Read More
GPU Utilization & Total Cost of Infrastructure Ownership

GPU Utilization & Total Cost of Infrastructure Ownership

Anton Allen
March 2, 2023
One of the primary issues faced across industries is the under-utilization of computing resources, especially GPUs. 
Read More
NVIDIA H100 PCIe vs. SXM5

NVIDIA H100 PCIe vs. SXM5

Erik Kimmerer
February 27, 2023
With NVIDIA being the leading player in the GPU market, it’s challenging to determine which NVIDIA GPU server is suitable for your company. In this blog post, I will compare the PCIe and SXM5 form factors for NVIDIA H100 GPUs, the highest-performing GPUs currently available, and contrast performance and costs to help you make an informed decision.‍
Read More
Addressing Utilization Issues with GPU Job Schedulers

Addressing Utilization Issues with GPU Job Schedulers

Anton Allen
February 10, 2023
A GPU Job Scheduler is a tool that manages and schedules the allocation of GPUs in a cluster environment, although, they have drawbacks when it comes to maximizing utilization and performance.
Read More
GVM Server - Nested Roles Explained

GVM Server - Nested Roles Explained

Erik Kimmerer
January 10, 2023
Learn all about one of GVM Server's primary benefits: organization-level provisioning, a nested roles feature that allows organizations to manage data and resources hierarchically for teams/projects.
Read More
LibVF.IO: Add GPU Virtual Machine Support

LibVF.IO: Add GPU Virtual Machine Support

Arthur Rasmusson
August 24, 2022
LibVF.IO (vGPU & SR-IOV on Consumer GPUs) has added support for GPU Virtual Machine (GVM).
Read More
Experience Better GPU Performance with GVM Server

Experience Better GPU Performance with GVM Server

Erik Kimmerer
August 23, 2022
Learn how Arc's GPU/CPU hypervisor, GVM Server, increases GPU performance and utilization through exclusive configurations made possible thanks to Simultaneous Multi-Virtual GPU
Read More
The Web Browser Landscape

The Web Browser Landscape

Arthur Rasmusson
June 4, 2021
As I’m sure many people have heard over the course of the last few days Chrome’s developers have chosen to change the way Chrome’s advertising, JavaScript, XHR connection, CSS, and iframe...
Read More
Closed Investment Round with OPN & Supporters Fund

Closed Investment Round with OPN & Supporters Fund

Justin Ritchie
June 5, 2021
Typically, when a GPU cloud consumer is utilizing their provider’s GPU compute, the provider must either run single physical devices per user or instead use expensive multi-user sharing...
Read More
Why Augmented Reality is Not Ready

Why Augmented Reality is Not Ready

Arthur Rasmusson
June 24, 2021
What enabled VR to become functionally capable of inducing reliable "presence" (the qualitative threshold for experiences that convince all the cognitive systems that make up your conscious...
Read More
Learning from OpenBSD to Make Computers Better

Learning from OpenBSD to Make Computers Better

Arthur Rasmusson & Louis Castricato
December 5, 2019
This is an attempt to consolidate down a number of threads spanning separate discussions from around the 'net I have been having on the subject of operating system development models and...
Read More
Looking to learn more about Arc Compute?
Read our latest white papers and case studies.
Arc Compute GPU Cloud Infrastructure

GVM Server - 100% Utilization POC

The following results are from tests we ran to demonstrate the performance benefits and limitations of GVM Server, which provides a way forward for further proof of concept tests within your organization’s infrastructure.
Download Now
Arc Compute GPU Cloud Infrastructure

GVM Server - Organization-Level Provisioning with Nested Roles

Organization-level provisioning is a nested roles feature that allows organizations to manage data and resources for teams/projects hierarchically.
Download Now
Arc Compute GPU Cloud Infrastructure

GVM Server - Solution Brief

GVM Server is Arc Compute's GPU/CPU hypervisor which is an all-in-one GPU utilization and virtualization solution.
Download Now
Arc Compute GPU Cloud Infrastructure

Arc Compute - Company Summary

Arc Compute's customers have one thing in common; they are all large consumers of GPUs who are tired of the current cloud business models and are looking for better, transparent pricing and better performance and security.
Download Now
Arc Compute GPU Cloud Infrastructure

Arc Compute Powers GPU Cloud Offering with Liqid

"Arc Compute, the only cloud service provider to offer Liqid’s revolutionary composable disaggregated infrastructure (CDI) as a service, proposed a GPU cloud option that offered the immersive video company a far more flexible and cost-effective solution".
Download Now
Arc Compute GPU Cloud Infrastructure

GVM Server - Superior GPU Utilization and Performance

As you will see in the following benchmarks, by utilizing GVM Server, your workloads can train up to 80% faster thanks to improved utilization of GPU resources.
Download Now