The CUDA moat has defined the AI boom for three years, protecting Nvidia less through its chips than through the software wrapped around them. This week two of the largest cloud companies on Earth moved to drain that CUDA moat in the open, and they did it with free software most developers already use.
A Software Coup, Not A Chip War
The real battle this week moved somewhere less visible and far more dangerous for Nvidia. On October 19, Amazon Web Services announced TorchNeuron, a tool that lets developers run their existing PyTorch programs on Amazon's in-house Trainium chips with a single configuration change. One day later, at the North American PyTorch Conference in San Jose, Google released TorchTPU, which does the same thing for its Tensor Processing Units. Both are open source. Both are free. Together they take direct aim at the CUDA moat.
Neither company said the word Nvidia out loud, and neither had to. For a decade that software layer has been the thing standing between Nvidia and every would-be rival. The chips were never the hard part. The code that makes those chips usable was. By open-sourcing tools that erase the switching cost, Amazon and Google have moved on that advantage in the open, inside the very framework that most of the AI world already uses.
The timing is not an accident. These releases landed in the same week that reports surfaced of Chinese firms queuing up more than $35 billion in Nvidia orders the company cannot fully fill. Supply is tight, prices are high, and the hyperscalers that account for most of Nvidia's revenue are also its most motivated competitors. The CUDA moat was supposed to make that conflict of interest irrelevant. This week it looked a little more relevant.
What Amazon And Google Actually Shipped
TorchNeuron is the more aggressive of the two. Amazon released it under the Apache 2.0 license and built it to hook directly into PyTorch's existing hardware interface, so a developer switches a workload from a GPU to a Trainium2 or Trainium3 chip by changing a single setting rather than rewriting application code. Amazon says PyTorch serves roughly 85 percent of AI researchers and developers, which is precisely why the company chose it as the entry point. The firm ran private tests with more than 30 companies through late 2025 and says it now runs nightly checks against more than 100,000 models hosted on Hugging Face.
The early customer numbers, all supplied by Amazon, are the part Nvidia will study hardest. The voice AI firm Deepgram reported processing about 60 percent more audio per dollar than on its previous GPU setup and has moved into production. Cartesia ported its inference pipeline with minimal changes. Amazon's own Prime Video used the stack to build a 3D basketball replay feature for NBA broadcasts. Colin Brace, a vice president at Amazon's Annapurna Labs chip division, summed up the pitch in one line to GeekWire: "If you know PyTorch, then you know Trainium."
Google's TorchTPU attacks from a different angle but lands in the same place. It connects PyTorch to TPUs through the framework's native extension system, offers three execution modes for debugging and speed, and plugs into torch.compile using Google's XLA compiler. Google claims its automatic operation-fusing mode delivers a 50 to 100 percent performance increase over the simpler path with no setup required. Crucially, standard distributed training techniques work out of the box, meaning the libraries developers already depend on do not have to be rebuilt. You can read Google's own technical account on its developer blog, and the official PyTorch project is where both efforts are being upstreamed.
Why The CUDA Moat Mattered
To understand the stakes, it helps to be precise about what the CUDA moat actually is. Nvidia's enduring advantage was never only that its chips were fast. It was that CUDA, the company's programming platform, had absorbed nearly two decades of library work, tooling, and developer habit. Nvidia points to more than five million active CUDA developers. Every major machine learning framework runs on it natively. Porting a model built around CUDA to rival hardware has historically meant months of engineering and testing, and that friction, not raw silicon, is what kept the company's share of the AI accelerator market above 90 percent.
That is the fortress Amazon and Google are now trying to flood. The strategy is not to beat CUDA on its own terms but to make the question of which chip you run on disappear beneath the framework. If PyTorch becomes genuinely hardware-agnostic, then the CUDA moat stops being a moat and becomes merely one option among several. Amazon has been explicit about the endgame, contributing code upstream to strip chip-specific dependencies out of PyTorch itself so the benefit accrues to every developer, not only its own customers. That is a long game, and it is aimed squarely at the lock-in that has defined the market.
Nvidia, for its part, still has the stronger hand in raw capability. Its CUDA ecosystem remains the default for cutting-edge research, its newest Vera Rubin generation promises large gains in inference performance and cost, and training the biggest frontier models still runs overwhelmingly on Nvidia hardware. The moat has not been drained. It has been breached, which is a different and slower thing.
The Economics Driving The Revolt
Follow the money and the revolt makes obvious sense. Inference, the work of actually running a trained model to answer queries, now accounts for roughly two-thirds of all AI compute, and inference is far less sensitive to the bleeding-edge performance where Nvidia dominates. It is sensitive to cost. AWS claims its Trainium instances can cut compute costs by up to half versus comparable GPU setups, and the Deepgram figure points the same direction. When the workload is price-driven and enormous, even a partial migration reshapes the market.
The hyperscalers also have the balance sheets to force the issue. Combined AI capital spending across Amazon, Google, Microsoft and Meta is running somewhere between $660 billion and $690 billion this year. Every dollar of that paid to Nvidia is a dollar not kept in-house, which is why each of these firms has spent years and billions building its own silicon. Amazon's custom chip operations passed a $25 billion annual revenue run rate earlier in 2026. Anthropic trains and runs its Claude models on Trainium, and OpenAI committed in February to two gigawatts of Trainium capacity. These are not experiments. They are production commitments worth tens of billions, as Blumefield has tracked across the sector.
Analysts have started to price in the shift. New Street Research projects Nvidia's share of inference compute could fall to somewhere between 20 and 30 percent by 2028, a stunning decline for a company that owns the category today. The custom-chip segment is forecast to grow several times faster than the GPU market over the coming decade. None of that happens without the software layer that Amazon and Google just made free, which is exactly why this week matters more than another earnings beat.
What Happens To Nvidia Now
The honest answer is that nothing breaks overnight. TorchNeuron is days old and TorchTPU only just reached the public. Real-world migrations are measured in quarters, the tooling will have rough edges, and Nvidia's grip on frontier training is not loosening this year. Anyone declaring the end of the CUDA moat is getting ahead of the evidence. Nvidia also retains enormous pricing power and a product roadmap that its rivals are still chasing on raw performance.
What has changed is the shape of the contest. For the first time, the path off Nvidia hardware runs through the same framework developers already know, maintained and promoted by the companies with the deepest pockets and the strongest incentive to see the CUDA moat shrink. The switching cost that protected Nvidia's margins is being deliberately engineered toward zero, in public, under open-source licenses that competitors and customers alike can adopt without permission.
Nvidia will not stand still, and its ecosystem advantage remains real. But the most valuable company in the world now has to defend its position on the merits of each chip generation rather than on the inertia of a software monopoly. That is a less comfortable place to stand. For the rest of the industry, and for anyone watching the AI trade, this week was a reminder that the deepest moats are the ones that erode quietly. For more analysis of the companies reshaping compute, follow Blumefield.