The Modern AI Stack: How to Choose Between Hyperscalers, Neoclouds, and Serverless Inference

There are three main ways to run AI. Serverless inference bills you per token and you never touch a GPU. Hyperscalers rent you GPUs inside AWS, Azure or Google Cloud. Neoclouds rent you GPUs with very little else around them. Most teams start serverless and stay there longer than they expect.

This guide also covers model-lab APIs, which sit inside serverless inference, and private cloud, which sits alongside all three. GPU prices rose across several providers this autumn, so price any comparison at current rates rather than last year’s.

1. What are hyperscalers, neoclouds and serverless inference?

Serverless inference means calling a hosted model through an API and paying per token. There are no GPUs to provision, no capacity to plan and nothing to run when traffic stops. Model-lab APIs and inference platforms are both forms of serverless inference, and the difference between them is whose models you get.

Hyperscalers are AWS, Azure and Google Cloud. They rent GPU instances by the hour alongside the storage, identity, networking and compliance you already use, and they also resell frontier models as managed services.

Neoclouds are cloud providers built to rent GPU compute for AI, without the full service menu of a hyperscaler. CoreWeave, Nebius, Lambda and Crusoe are the largest, with GPU marketplaces like RunPod and Vast.ai below them on price. You get the silicon and the networking, and much less of everything else.

2. Hyperscaler vs Neocloud vs Serverless Inference Compared

Serverless inference suits variable traffic and frontier models. Hyperscalers suit teams whose data and compliance already live in one cloud. Neoclouds suit sustained, GPU-heavy work where price per hour matters more than the services around it.

	Serverless inference	Hyperscalers	Neoclouds
What you buy	Model output	GPU-hours plus a full cloud	GPU-hours
Example providers	OpenAI, Anthropic, Together AI, Fireworks	AWS, Azure, Google Cloud	CoreWeave, Nebius, Lambda, Crusoe
Models you can use	Frontier and open-weight, whatever the host offers	Any open model, plus resold frontier models	Mostly open models you deploy yourself
How you pay	Per token	Per GPU-hour, on demand or prepaid	Per GPU-hour, usually 8-GPU nodes
Typical price	About $0.10 to $2 per 1M input for the main models, more for top tier	$5.97 to $12.29 per H100-hour	About $1.50 to $6.16 per H100-hour
Setup	An API key	Account, quota, networking	Account, often a waitlist
Scales to zero	Yes	No	No
Control over hardware	None	Instance type and region	Instance type, sometimes the cluster
Compliance and residency	Provider’s terms	Widest region and certification coverage	Fewer regions, varies by provider
Best for	Variable traffic, frontier models, fast starts	Data gravity, regulated workloads	Training and sustained inference
Main trade-off	No hardware control, rates can change	Highest list GPU rate	Fewer managed services

3. Serverless Inference with Model-Lab APIs and Inference Platforms

Both bill per token. The split is between labs selling their own closed models and platforms hosting open-weight ones.

Model-lab APIs come from OpenAI, Anthropic and Google, selling the models they built. A model lab is a company that trains its own frontier models rather than hosting someone else’s, and its API is the only route to those weights. You get one endpoint, the lab’s own models, and no way to run them anywhere else.

Model	Input per 1M	Output per 1M	Note
Claude Haiku 5.5	$0.10	$0.50	Rises above 100K-token prompts
GPT-6 Luna	$0.10	$0.50	Rises above 272K input tokens
Gemini 3.8 Flash	$0.75	$3.75	Doubles on 1 January 2027
Claude Sonnet 5.5	$2.00	$10.00	20x Haiku’s input rate

Inference platforms host open-weight models instead. Together AI, Fireworks, Groq, DeepInfra and Baseten serve models like Llama, Mistral and Qwen per token, often cheaper than mid- and top-tier models such as Claude Sonnet. They do not automatically undercut the small lab models: Llama 3.3 70B costs more on some platforms than Haiku 5.5 or GPT-6 Luna do. Baseten also sells dedicated deployments billed per minute, and NVIDIA DGX Cloud Lepton is a marketplace connecting developers to GPU capacity from partner clouds, billed for GPU use rather than per token. Groq is worth a note: NVIDIA licensed its inference technology in December 2025 and hired its founder, but Groq remains independent and GroqCloud continues to operate.

The strengths are the same for both. Traffic scales to zero, so idle services cost nothing. There is no capacity planning, no quota request and no cluster to maintain. Setup is an API key.

The trade-offs are also shared. You get no control over the hardware underneath, rates can change with little notice, long prompts often cross into a higher price tier, and models get deprecated on the provider’s schedule rather than yours.

4. AI on Hyperscalers like AWS, Azure and Google Cloud

There are two ways to run AI on a hyperscaler. You can rent GPU instances directly, such as AWS p5, Google Cloud a3 or Azure ND H100 v5, or you can use a managed model service: Amazon Bedrock, Google Vertex AI or Azure AI Foundry, which resell frontier and open models per token on your existing cloud bill.

AWS and Google Cloud also sell their own accelerators. AWS Trainium and Google TPUs cost materially less per chip-hour than an H100, but they run their own toolchains rather than CUDA, so the saving comes with porting work.

The strengths are about position rather than price. Your GPUs sit next to your data, your identity model and your compliance posture. Region and certification coverage is wider than anywhere else. Existing commitments, credits and enterprise agreements apply.

You pay the hyperscaler premium for that proximity. If your data, identity and compliance already live in one cloud, moving inference out carries egress charges, integration work and compliance review that no GPU price list shows.

The trade-offs are the highest list rate of the three options, quota requests that can take days, and limited capacity on the newest GPUs. For instance-by-instance rates, see our cloud GPU pricing comparison.

5. Neoclouds like CoreWeave, Nebius, Lambda and Crusoe

Neoclouds rent GPU capacity and not much more. They serve AI labs, model companies and platform teams running training clusters or steady high-volume inference, and marketplaces like RunPod and Vast.ai sit below them for cheaper, less predictable capacity.

The strengths are access and rate. List prices usually land below hyperscaler rates, capacity on new GPU generations can be easier to get, and the products are built around multi-GPU clusters rather than treating them as an upsell.

Cheaper is not universal, though. CoreWeave’s H100 lists slightly above AWS Capacity Blocks, so the comparison depends on which hyperscaler price you are measuring against. Our GPU pricing tracker compares current rates across providers.

The trade-offs are the services you give up: fewer managed offerings, narrower region coverage, waitlists for in-demand hardware, and commitments required for the best rates. There is also a counterparty question worth weighing. Gartner forecasts that over 80% of neocloud providers will cease independent operations by 2030, which is an argument for keeping exit paths open rather than for avoiding the category.

6. Which models can you run on each?

Closed models only run where their owner allows. Open-weight models run anywhere you can deploy them.

Model type	Serverless inference	Hyperscaler GPUs	Neoclouds	Private cloud
Closed frontier (Claude, GPT, Gemini)	Yes, via the lab API or a managed service	No	No	Rarely
Open weight (Llama, Mistral, Qwen, DeepSeek)	Yes, on inference platforms	Yes	Yes	Yes
Fine-tuned open models	Only where the host accepts custom weights	Yes	Yes	Yes
Your own trained model	Yes, on some platforms via custom model import	Yes	Yes	Yes

The one exception to that last column is Google, which offers Gemini on Google Distributed Cloud for on-premises and air-gapped deployments. Claude and GPT have no equivalent.

This is the constraint that settles most architecture debates. If you need Claude or GPT, renting GPUs is not an option, and your choice is between the lab’s own API and the same model resold through Bedrock, Vertex AI or Azure AI Foundry. If an open model does the job, everything is available to you and the decision becomes economics and operations.

Model availability on managed services changes often, so check the relevant cloud’s model catalog before committing to a provider on the strength of one model.

7. Pricing Compared by Token and by GPU-Hour

Serverless runs about $0.10 to $2 per million input tokens for the main lab models. Hyperscaler H100 capacity runs $5.97 to $12.29 per GPU-hour. Neocloud H100 capacity runs about $1.50 to $6.16.

Option	Unit	Typical range
Model-lab APIs	Per 1M tokens	$0.10 to $2 input for the models above, more for top tier
Inference platforms	Per 1M tokens	About $0.10 to $1 per 1M input for Llama 3.3 70B
Hyperscaler GPUs	Per H100-hour	$5.97 to $12.29
Neocloud GPUs	Per H100-hour	About $1.50 to $6.16

Those columns are hard to compare because they are not the same unit. A token price is a price per unit of work. A GPU-hour is a price per unit of time, and how much work you get out of an hour depends on the model, the throughput of your setup and how busy the card is.

A rented GPU also bills while it sits idle, so low utilization raises the true cost per unit of work without changing the rate you agreed.

That is why self-hosting a standard open model rarely beats the cheapest serverless host on price alone. Run your own numbers in the LLM cost calculator before assuming otherwise.

8. How to Choose the Right Option for Your Team

Five questions settle most of it.

Workload	Best fit	Why
Prototype or bursty traffic	Serverless inference	Scales to zero, no capacity planning
Frontier model required	Model-lab API or managed cloud service	Closed weights cannot be self-hosted
High-volume open model	Inference platform	Cheaper than mid- and top-tier models, no infrastructure to run
Steady 24/7 inference	Neocloud or private cloud	Custom or fine-tuned models, guaranteed capacity, or volume past premium-host pricing
Multi-week training	Neocloud	Lower hourly rate, cluster-first products
Regulated or residency-bound	Hyperscaler or private cloud	Compliance posture already in place

9. When does private cloud make sense?

When load is steady and predictable, and when data residency or single tenancy is a requirement rather than a preference.

Hosted private cloud sells a dedicated environment. A fixed monthly fee replaces the metered hour, so a busy environment costs less per useful hour and an under-used one costs the same. Single tenancy also removes the noisy-neighbor variance that makes multi-tenant inference latency unpredictable.

OpenMetal is one example. It provides single-tenant bare metal GPU servers, currently NVIDIA H200 and RTX PRO 6000, on a fixed monthly fee with no metered hours and egress included, which removes two of the line items that make hourly GPU bills hard to forecast. You get full root access to the hardware rather than a hypervisor slice, and multiple servers can be interconnected over a private network for cluster work. Pricing is quoted rather than published, and written quotes are honored for 30 days, so you can model a year against a fixed number instead of an hourly rate that moved three times in 2026.

10. How Economize Tracks AI and Cloud Costs in One View

Economize shows your cloud bill and your AI API bill side by side, so you can compare a token rate with a GPU-hour rate on real numbers.

• AI Costs tracks OpenAI, Anthropic, Amazon Bedrock, Google Vertex AI and Azure AI Foundry, broken down by provider, model and agent.
• Unit costs show cost per 1M tokens, per request and per agent run, so a token bill becomes a rate you can compare.
• Multi-cloud dashboards bring AWS, Azure and GCP spend, including GPU instances, into the same view.
• Idle and rightsizing recommendations flag oversized cloud resources with an estimated saving.
• Anomaly alerts on cloud spend go to Slack, Teams, Discord or Google Chat.

Pricing is published and flat: free up to $100,000 a month in tracked spend, then $249 a month up to $250,000. Setup takes about five minutes.

Frequently Asked Questions