What "cheapest cloud GPU" actually means in 2025
The phrase "cheapest cloud GPU" collapses three different procurement questions into one. They have three different answers:
- Cheapest $/hr. The lowest published on-demand or spot rate for a given GPU family — Vast.ai dominates for H100 and A100, RunPod community cloud for RTX 4090. Useful for marginal-hour budgeting.
- Cheapest per training step. $\$/hr \div sustained\ throughput\ at\ your\ batch$. This is the only correct unit for training procurement. A100 can win or lose against H100 depending on model size, batch, and quant scheme.
- Cheapest per million output tokens. The inference-side analogue — same formula but with tokens/second rather than steps/second. Covered in detail in the H100 vs A100 Inference Cost explainer.
All three answers change with the GPU family because the underlying silicon and VRAM tier are doing different jobs. H100 brings FP8 + Hopper tensor cores + 80GB HBM3. A100 is the predecessor Ampere part with FP16 only. A10G is a smaller Ampere inference SKU with 24GB. RTX 4090 is a consumer Ada Lovelace part with 24GB GDDR6X and no NVLink — strong on consumer-grade inference, weak on multi-GPU training jobs. The dollar number that matters depends on which axis you're optimizing, and conflating spot-floor rates with reserved-tier rates — or treating RTX 4090 as a 1:1 substitute for H100 — is the most common procurement error this year.
The one-line procurement rule
Match GPU to workload class, then minimize $/hr within that class. Cross-class substitution (e.g., "I can save money by training 7B on RTX 4090") almost always backfires because the workload-binding constraint is VRAM or interconnect, not raw $/hr. Within a workload class, Vast.ai and RunPod community cloud hold the spot floor; AWS / GCP / Azure hold the SLA.
The cheapest $/hr by GPU family (Q3 2026)
The matrix below shows live on-demand or spot rates for H100, A100-80GB, A10G, and RTX 4090 across five providers — Vast.ai, CoreWeave, Lambda Labs, RunPod, and AWS. Numbers come straight from the provider pages and our /gpu-forward-pricing provider-native rows. Cells where we have not independently verified the current rate are marked data pending — we do not interpolate.
| GPU family | Vast.ai | CoreWeave | Lambda Labs | RunPod | AWS on-demand | Workload fit |
|---|---|---|---|---|---|---|
| H100 (Hopper, 80GB HBM3) | $1.49 (spot) | $3.92 / $1.79 reserved-1yr | $4.29 | $2.49 / $1.89 community spot | $4.10+ reserved p5 | Training ≥30B · 70B+ inference |
| A100-80GB (Ampere) | $0.89 (spot) | $2.70 | $2.79 | $2.59 | $5.12+ reserved p4de | Training ≤13B · 7B inference |
| A10G (Ampere, 24GB) | data pending | data pending | data pending | data pending | $1.006 g5.xlarge | Inference only · 7-13B FP16 |
| RTX 4090 (Ada, 24GB) | $0.31 (spot) | — | — | $0.34 community spot · $0.795 on-demand | — | Small-model inference · dev/test |
data pending rather than invent a number. RTX 4090 does not appear on AWS, CoreWeave, or Lambda's published consumer/GPU line cards, so those cells are —.
Three things to take from the table:
- Vast.ai holds the spot floor for H100 and A100. $1.49/hr H100 and $0.89/hr A100-80GB are the lowest published rates in the dataset this quarter. They have held roughly flat through Q2-Q3 2026.
- AWS on-demand is the most expensive path but the most SLA-strong. The premium is 2-3× over spot for the same GPU family. For non-fault-tolerant workloads, the math often favors paying the premium.
- RTX 4090 is the cheapest on paper but the most constrained in workload fit. 24GB VRAM and no NVLink make it unsuitable for multi-GPU training; it is the right answer for ≤13B single-GPU inference and dev workloads only.
When "cheapest" is not "best value"
The cheapest GPU is rarely the cheapest-per-output. The substitution rule from the procurement explainer applies: match GPU to workload class, then optimize $/hr within that class. Three cases where "cheapest" loses:
Case 1: Training a 7B+ model on RTX 4090 spot
At $0.31–$0.40/hr, RTX 4090 looks 4× cheaper than H100 spot. But 7B FP16 weights + Adam optimizer state + activations exceed 24GB VRAM; you need DeepSpeed Zero-3 offload or parameter sharding, which adds 30–50% wall-clock time per step. The effective $/hr becomes $0.46–$0.60 — still cheaper than H100, but only marginally, and you lose tensor parallelism for anything larger. The right answer for 7B training is H100 spot at $1.49/hr or A100 spot at $0.89/hr depending on framework and quantization.
Case 2: Serving 70B inference on A100 spot
At $0.89/hr A100 looks 1.7× cheaper than H100 spot per hour. But 70B FP16 needs tensor parallelism across multiple A100s, and A100's 600 GB/s NVLink bandwidth means cross-GPU communication eats a large fraction of FLOPs. H100's 900 GB/s NVLink + FP8 tensor cores deliver ~4× the throughput per GPU at the same VRAM tier. Reusing the formula from the H100 vs A100 inference cost explainer:
$/1M tokens = ($/hr × 1,000,000) ÷ (tokens/second × 3,600,000)
At 70B-class, the per-token calculation flips — H100 wins despite the 1.7× $/hr premium. Cheapest $/hr stops being cheapest $/token.
Case 3: Reserved H100 at AWS when you only run 4 hours/week
Reserved pricing is cheap on a $/hr basis but only when you actually consume the hours. AWS p5.48xlarge reserved is $2.40/GPU-hr — slightly above Vast.ai spot. If you commit to 730 hr/yr, fine. If your actual usage is 200 hr/yr, your effective reserved $/hr is ~$8.76/GPU-hr — worse than spot by 5.9×. The Reserved Instance Advisor computes break-even utilization rate per provider per tier.
The takeaway: the "cheapest cloud GPU" answer in 2025 depends on workload class first, then on a within-class spot-vs-reserved decision. Picking the absolute-lowest $/hr without that context is the procurement trap most teams fall into.
us-east-1 than eu-west-1. Vast.ai and RunPod spot prices vary by host region within the marketplace — the "$1.49/hr H100" headline is a marketplace floor, not a uniform rate. Egress costs ($0.09/GB AWS, $0.12/GB GCP) can dominate $/hr savings when training or serving models that shuffle large checkpoints or embeddings between clouds. Always normalize to a total cost of ownership, not just the hourly rate.
Frequently asked questions
Which GPU is cheapest for training a 7B parameter model in 2025?
For a 7B parameter model, the cheapest sensible training GPU is H100 spot at $1.49/hr on Vast.ai — assuming you can saturate batches. A100-80GB spot at $0.89/hr on Vast.ai is markedly cheaper per hour but pays back the throughput penalty only when batch sizing is small. RTX 4090 at $0.31–$0.40/hr spot looks cheapest on paper but cannot fit a 7B FP16 training state in single-GPU memory without aggressive offload, which slows effective $/hr materially. A10G is not a training-class part — its 24GB VRAM and Ampere small-tensor path make it an inference-only choice.
Is RTX 4090 in the cloud good for inference?
Yes — RTX 4090 is excellent for small-to-medium LLM inference (≤13B FP16 or ≤7B FP8 single-GPU). Its 24GB GDDR6X VRAM and Ada Lovelace tensor cores deliver competitive throughput-per-dollar for short-sequence, high-QPS workloads. It is not a substitute for H100/A100 at 70B+ model scale. RTX 4090 spot at $0.31–$0.40/hr on Vast.ai and RunPod community cloud is materially cheaper than A100-80GB at the same workload when VRAM fits. /cost-per-token has the workload-modelled matrix.
Why is A100 cheaper than H100 in $/hr?
A100 (Ampere, 2020) is two generations behind H100 (Hopper, 2022) and has been in market longer with much larger aggregate supply — both new and used. Supply overhang keeps A100-80GB spot floor at $0.89/hr on Vast.ai, while H100 spot holds $1.49/hr on the same marketplace. The $/hr gap of roughly 1.7× does not translate to a cost-per-token gap of the same magnitude because H100's Hopper tensor cores (FP8 + 4× tensor throughput per SM, HBM3 bandwidth) more than compensate at ≥30B model scale. See the $/M-token matrix in the H100 vs A100 inference cost explainer.
Should I use spot or reserved for the cheapest rate?
Spot is cheapest but preemptible — Vast.ai marketplace and RunPod community cloud can drop your instance with minutes of warning. Reserved (1yr+) is cheapest on a committed-hour basis but locks a fixed $/hr. The procurement tradeoff: if your workload is fault-tolerant (checkpoint-restart, stateless inference, batch jobs) → take spot and accept preemption risk. If your workload is latency-sensitive and SLA-bound → reserved or on-demand. The Reserved Instance Advisor computes your break-even utilization rate per provider per tier.
Which provider has the cheapest H100 in 2025?
Vast.ai holds the cheapest H100 spot rate at $1.49/hr — a level it has held for 7 consecutive weeks through Q3 2026. CoreWeave on-demand at $3.92/hr and reserved-1yr at $1.79/hr are well above Vast.ai but below AWS p5.48xlarge. Lambda Labs is at $4.29/hr on-demand — the mid-tier elastic option. For hyperscaler SLA procurement, AWS reserved p5.48xlarge is the most expensive path at $4.10+/hr; for elastic spot, Vast.ai is the floor.
Compare the matrix against your workload
GridStackHub's calculator, optimizer, and reserved advisor all read from the same provider-native dataset this post is built on.