Over the past two years, the US AI compute sector has staged a puzzling drama of contradictions: mainstream large-model API prices have been cut round after round, with per-call costs in some application scenarios down an order of magnitude from the early days; meanwhile, GPUs in cloud providers' data centers are nearly impossible to secure, and compute rental prices keep climbing during periods of tight supply. How can crashing prices and surging demand coexist? The answer lies in the phrase “economics of inference,” which is quietly rewriting the investment logic of the US AI compute supply chain.
1. Token Prices Plunge: AI Is Turning from a Luxury into a Utility
Looking back at the commercialization of large models, token-based billing has become the industry's mainstream model. Public industry information shows that leading model vendors have gone through multiple rounds of price cuts, and combined with cache hits, batch processing, and routing to smaller models, the effective per-unit inference cost borne by developers keeps falling. For many applications, the expense of calling large models has shifted from an early “burn money to experiment” luxury into a utility-like basic resource that can be booked as a routine cost.
The significance of this change is that it lifts the ceiling on demand. When the price of a long-document analysis, a piece of code generation, or a round of multimodal dialogue drops low enough, previously uneconomic use cases emerge en masse: customer service, marketing copy, legal first review, research assistance. Prosperity at the application layer in turn drives up total consumption of underlying compute. Falling prices did not kill demand; they released demand from a few giants into thousands of industries.
2. The Jevons Paradox Replays: The Higher the Efficiency, the Scarcer the Compute
Economics has a classic concept called the Jevons Paradox: after steam engine efficiency improved dramatically in the nineteenth century, coal consumption rose rather than fell, because more people could afford it—and used more of it. Today's compute market is replicating this logic. Every time unit token cost drops a step, a new wave of applications crosses the commercial break-even point, and the total compute consumed by these new applications often exceeds what the efficiency gains save.
More crucially, AI tasks are quickly getting heavier. A traditional single-turn Q&A request may consume only a few thousand tokens; an autonomous agent that plans, retrieves, tries, and verifies can consume tens or even hundreds of times that over one complete task. Unit costs of multimodal content such as video and audio far exceed plain text. As task forms evolve from one-question-one-answer to delegated execution, even if unit prices keep falling, the total bill can still balloon.
Moreover, the boundary between training and inference is blurring. Techniques such as reinforcement learning, synthetic data generation, and inference-time computation have made “thinking while computing” an important way to improve results; much of this compute is booked on the inference side in financial terms, further amplifying the demand curve for inference compute.
3. Supply Chain Shift: From Training-Oriented to Inference-Oriented
This change in demand structure is reshaping the value distribution across the compute supply chain. Several segments deserve close tracking:
- Chip layer: Training prizes peak performance; inference cares more about cost-effectiveness per unit and memory bandwidth. This opens up space for custom ASICs, energy-efficient inference chips, and matching interconnect solutions, and gives the “diversified compute” narrative a more realistic footing than ever.
- Memory and interconnect: Inference is extremely sensitive to data movement speed, so memory bandwidth and low-latency networks keep rising in importance; order visibility at related suppliers becomes a bellwether of industry sentiment.
- Data center form: Inference facilities need to sit close to users and data sources, showing a regional, distributed trend—clearly different from the concentrated build-out of training supercomputers. This will affect the revaluation of land, power, fiber, and other resources.
- Energy and operations: Inference workloads run around the clock at high utilization, demanding power stability and cooling efficiency different from training clusters; data center operators with high operational efficiency stand to earn excess returns.
4. Investment View: Three Themes and One Test Indicator
Theme 1: Watch Cloud Platforms' Volume-Price Breakdown
For cloud providers, whether token price cuts mean giving up margin or growing revenue depends on whether volume growth can outrun price declines. Investors should focus on the comparison between AI-related revenue growth and unit cost changes in financial reports, rather than on vague AI narratives. If inference revenue growth keeps exceeding the pace of price cuts, demand elasticity is positive and the upcycle is still strengthening.
Theme 2: Position in Cost-Effective Compute Assets
The core of competition in the inference era is tokens produced per dollar. Chip design companies with standout energy efficiency and inference-optimized designs, as well as suppliers of networking, storage, and software stacks that can significantly lower unit costs, are well placed to defend margins through price-cut cycles.
Theme 3: Don't Overlook the Certainty of the “Water Sellers”
Whichever model wins, demand for basic links such as power, cooling, and data center operations is certain. Such assets offer less valuation elasticity than chips but steadier cash flows, making them suitable ballast for a portfolio.
One Test Indicator
A simplified formula can track industry health: growth in total token consumption depends on the product of the price decline and demand elasticity. As long as total consumption maintains high growth, orders across the compute supply chain have support; once price declines come with stalled volume growth, beware of the risk of overdrawn demand.
5. Risks and Conclusion
Of course, the economics of inference is no master key. If price wars persistently outpace demand growth, gross margins at model vendors and the application layer will stay under pressure; a concentrated release of supply capacity, persistently high macro interest rates, or regulatory changes could all amplify sector volatility. In addition, the structural shift in inference demand may leave some assets betting on training scenarios with returns below expectations.
On balance, the flywheel of price cuts, demand expansion, and tighter compute keeps turning, and the core question in US AI compute has shifted from “whether there is demand” to “who can meet demand at the lowest cost.” For investors, rather than agonizing over whether this is a bubble, put research effort into the three key cards: volume-price breakdown, unit economics, and supply chain positioning. In the inference era, the decisive edge belongs to those who count most precisely.
