Hopper architecture
Rent NVIDIA H100 PCIe cloud GPU
80GB of VRAM and the Hopper Transformer Engine at $2.30/hr on OpenRelay. The same 80GB and the same FP8 support as the SXM part, on a 350W PCIe card, for single-GPU inference and fine-tuning.
NVIDIA H100 PCIe: 80GB of Hopper, at a lower rate
The H100 PCIe is the add-in-card version of NVIDIA's Hopper data center GPU. It carries the same GH100 die and the same 80GB of VRAM as the SXM5 module, in a 350W dual-slot card instead of a 700W board. Fewer SMs are enabled and the memory is HBM2e at 2,000 GB/s rather than HBM3 at 3,350 GB/s, so it lands at roughly 70 to 75 percent of SXM throughput on a single GPU.
What does not change is what fits. Llama 3 70B at FP16, Mixtral 8x7B, DBRX, and vision-language models in the 40 to 80GB range all run natively at full precision, with no quantization compromises. FP8 and the Transformer Engine come from the Hopper architecture rather than the board, so a PCIe card gets the same automatic mixed precision that makes Hopper roughly 3x faster than Ampere on transformer models.
OpenRelay sells the PCIe and SXM boards as separate SKUs at separate prices. Pick the one your workload needs and that is the hardware you get. The H100 SXM is the one to rent when you are training across several GPUs at once and the NVLink fabric is doing real work.
H100 PCIe technical specifications
| Specification | H100 PCIe |
|---|---|
| GPU Architecture | Hopper (GH100) |
| Board | PCIe Gen5 dual-slot |
| VRAM | 80GB HBM2e |
| CUDA Cores | 14,592 |
| Memory Bandwidth | 2,000 GB/s |
| FP32 Performance | 51 TFLOPS |
| FP16 (Tensor) | 756 TFLOPS (with sparsity: 1,513) |
| FP8 (Tensor) | 1,513 TFLOPS (with sparsity: 3,026) |
| TDP | 350W |
| Memory Type | HBM2e |
| Tensor Cores | 456 (4th Gen) |
| Multi-GPU | PCIe Gen5 peer-to-peer (no NVLink fabric) |
| Transformer Engine | Yes (FP8 auto-mixed precision) |
The Transformer Engine switches between FP8 and FP16 during computation without any code changes on your side, which is where most of Hopper's advantage over Ampere comes from. The 2,000 GB/s of HBM2e bandwidth is below the SXM part but still 29 percent above an A100 80GB, and bandwidth is what governs token generation speed on memory-bound decoding.
H100 PCIe cloud GPU pricing on OpenRelay
Hyperscalers quote one H100 price and give you whichever board their instance family uses. AWS charges $4.76/hr for p5 instances, GCP $4.52/hr, Azure $3.67/hr. On OpenRelay the two boards are separate line items, so the cheaper one is something you can actually choose.
| Provider | GPU | $/hour | $/month |
|---|---|---|---|
| OpenRelay | H100 PCIe (80GB) | $2.30 | $1,656 |
| OpenRelay | H100 SXM (80GB) | $2.60 | $1,872 |
| Azure | H100 (80GB) | $3.67 | $2,642 |
| GCP | H100 (80GB) | $4.52 | $3,254 |
| AWS | H100 (80GB) | $4.76 | $3,427 |
| Lambda Labs | H100 (80GB) | $2.99 | $2,153 |
All OpenRelay pricing includes storage, load balancing, and network egress. No hidden fees.
Best use cases for the H100 PCIe
The PCIe card is the right pick whenever the work happens inside one GPU. Here is where it does the same job as the SXM part for less:
Single-GPU Large Model Inference
Run Llama 3 70B, Mixtral 8x7B, and DBRX at full precision on one GPU. The 80GB of VRAM is the same as the SXM part, so nothing about which models fit changes. Where a single GPU serves the request, the PCIe card does the same job for less.
LoRA and QLoRA Fine-Tuning
Adapter fine-tuning keeps the base model frozen and trains a small number of parameters, so it stays on one GPU and never touches the interconnect. This is the workload where the PCIe card's price advantage is pure savings with no throughput cost worth naming.
Batch and Offline Generation
Overnight embedding runs, document processing, synthetic data generation, and evaluation sweeps are throughput jobs with no latency target. Continuous batching keeps the GPU saturated, and per-minute billing means you pay for the run, not the reservation.
Vision and Multi-Modal Models
Deploy LLaVA, CogVLM, and other vision-language models in the 40 to 80GB range. Both the vision encoder and the language decoder get Transformer Engine acceleration, and the memory headroom leaves room for large image batches.
Production Inference APIs
Serve concurrent users with vLLM, TGI, or Triton behind an endpoint. Add GPUs to scale horizontally, one replica per card, which is a pattern that does not need an NVLink fabric between them.
Evaluating Hopper Before Committing
If you are deciding whether Hopper is worth it for your workload, the PCIe card is the cheaper way to measure. FP8 and the Transformer Engine behave identically, so the numbers transfer.
How to deploy an H100 PCIe on OpenRelay
Create your account
Sign up at openrelay.inc/get-started. No upfront commitments or lengthy procurement processes. Get access to the dashboard and CLI immediately upon signup.
Select the H100 PCIe
Choose H100 PCIe from the GPU catalog. It is a separate entry from H100 SXM, so what you pick is what you get: no substitution between boards. Configure GPU count, region, and container image.
Deploy with full GPU passthrough
OpenRelay provisions your H100 with VFIO passthrough in a lightweight VM. Your workload gets bare-metal GPU performance with strong security isolation. The full 80GB and all Tensor Cores are exclusively yours, with no GPU sharing or virtualization overhead.
Scale and monitor
Your deployment includes auto-failover, load balancing, and real-time monitoring. Track GPU utilization, memory usage, and inference throughput from the dashboard. Billing is per-minute, so you only pay for what you use.
H100 PCIe vs H100 SXM
Same die, same 80GB, different board. The gap is bandwidth, clocks, and what happens between GPUs:
| Feature | H100 PCIe | H100 SXM |
|---|---|---|
| VRAM | 80GB HBM2e | 80GB HBM3 |
| Memory BW | 2,000 GB/s | 3,350 GB/s |
| CUDA Cores | 14,592 | 16,896 |
| FP8 Tensor | 1,513 TFLOPS | 1,979 TFLOPS |
| TDP | 350W | 700W |
| GPU-to-GPU | PCIe Gen5 | NVLink 900 GB/s |
| OpenRelay Price | $2.30/hr | $2.60/hr |
On one GPU the PCIe card gives you most of the throughput for less money. Across eight GPUs exchanging gradients every step, NVLink is worth far more than the price difference, and H100 SXM is the SKU to rent.
H100 PCIe performance for AI workloads
Representative single-GPU figures. Actual numbers depend on your serving stack, batch size, and sequence length:
| Workload | Model | Performance |
|---|---|---|
| LLM Inference | Llama 3 70B (FP16) | ~30 tokens/sec |
| LLM Inference | Llama 3 8B (FP16) | ~135 tokens/sec |
| Training | 13B LoRA fine-tune (FP8) | ~2.2x faster than A100 |
| Image Generation | SDXL (30 steps, 1024×1024) | ~2 sec/image |
| Batch Throughput | Llama 3 8B (continuous batching) | ~2,200 tokens/sec |
Frequently asked questions
How much does it cost to rent an H100 PCIe on OpenRelay?
OpenRelay offers NVIDIA H100 PCIe cloud GPUs at $2.30 per hour with per-minute billing. That works out to approximately $1,656 per month for 24/7 usage. There are no minimum commitments, no egress fees, and no hidden costs. AWS charges $4.76/hr and GCP $4.52/hr for H100 instances.
What is the difference between H100 PCIe and H100 SXM?
Same Hopper GH100 die and the same 80GB of VRAM, different board. The PCIe card runs at 350W with 14,592 CUDA cores and 2,000 GB/s of HBM2e bandwidth. The SXM5 module runs at 700W with 16,896 CUDA cores and 3,350 GB/s of HBM3 bandwidth, and it sits on an NVLink fabric that moves data between GPUs at 900 GB/s. For a single-GPU job the PCIe card delivers roughly 70 to 75 percent of SXM throughput at a lower price. For multi-GPU training where GPUs exchange gradients every step, SXM is the one to pick. OpenRelay sells both, priced separately.
Which H100 should I rent?
Pick PCIe when your job fits on one GPU: inference on a 70B model, fine-tuning with LoRA, batch generation, or anything memory-bound that is not bottlenecked on GPU-to-GPU traffic. Pick SXM when you are training across several GPUs at once, when your model is sharded with tensor parallelism, or when you need the extra memory bandwidth. The 13 percent price difference is small next to the difference in multi-GPU scaling.
Does the H100 PCIe have the Transformer Engine?
Yes. FP8 support and the Transformer Engine are properties of the Hopper architecture, not the board, so the PCIe card gets the same automatic FP8/FP16 mixed precision. That is what makes it roughly 3x faster than an A100 on transformer models despite the lower clocks and bandwidth.
What AI models fit on an H100 PCIe?
The 80GB of VRAM runs Llama 3 70B at FP16, Mixtral 8x7B, DBRX, and large vision-language models like LLaVA-34B without quantization. Anything that fits on an SXM H100 fits here too: the VRAM is identical. Only throughput differs.
Is the H100 PCIe worth the premium over the RTX 4090?
Only if you need the memory or FP8 training. If your model fits in 24GB, the RTX 4090 at $0.29/hr is the better inference buy by a wide margin. The H100 PCIe earns its rate when you need 80GB for a large model at full precision, the Transformer Engine for training, or HBM bandwidth for memory-bound decoding.
Ready to deploy on the H100 PCIe?
80GB of Hopper at $2.30/hr. No waitlists, no minimum commitments, deploy in minutes.