Matt Garman interned at AWS in 2005, during business school, when it was still an internal project, and went on to become EC2’s first general manager. As CEO, he now runs a business he puts at about $169 billion to $170 billion in revenue, growing 37%. The tension, as Raghuram frames it, is that frontier labs gobble up every available GPU while startups, the enterprises of tomorrow, still need capacity, and agents are replacing the developers AWS was built to serve. The conversation runs from startups and agent-ready infrastructure to GPU allocation, bubble risk, power and supply chains, AWS’s custom chips, enterprise adoption, security and AWS’s own teams. These signals are from an interview that Raghu Raghuram did with Matt Garman on the a16z podcast.
Startups and Agents
Startups Become Enterprises. Garman’s internship project at AWS in 2005 was an analysis of who AWS would appeal to most, and the answer was startups. He estimates, calling the number imprecise, that 30% to 40% of AWS revenue now comes from companies that were startups at some point in AWS’s lifetime. Startups also push AWS harder than banks, healthcare companies or governments. His reasoning: what startups ask for today, banks will want 5 years later. ↗ 2:50
Built for P999 Latency. Agents and people use many cloud services in similar ways, Garman says, but agents reach for data differently. AWS Context, now in preview, builds a context layer so agents can find data across S3, Aurora and other AWS stores. Agents also care about tail latency, and slow P999 S3 responses block them in a way people rarely notice. He credits that focus on performance for his claim that agentic workflows run better on AWS than anywhere else. ↗ 7:55
An Account in 30 Seconds. Garman says the vast majority of customers now tell a coding agent, such as Kiro, Claude or Codex, to deploy on AWS. For 20 years, a new account meant defining VPCs and IAM roles first. AWS is slowly rolling out accounts that need no credit card, accept a Gmail sign-up and are live in under 30 seconds. They are full accounts, so growing later needs no migration. His goal: “not a choice for the customer but an easy on-ramp.” ↗ 9:58
Agents Get Scoped Permissions. Agents need some brand-new building blocks, Garman says, including compute sandboxes, gateways and their own permissions. His line: “You don’t just want to give it Raghu’s permissions and let it go do whatever you can do.” An agent should get short, time-boxed, fine-grained rights for one task. For sandboxes, Raghuram notes that many startups build on Firecracker, the microVM AWS created about 10 years ago. Garman says it starts fast, isolates well and carries little overhead. ↗ 14:50
The Buildout
Capacity Held for Startups. Garman puts capex at $220 billion for 2026, possibly the largest single-year expense any company has had. Power, data centers, memory, chips and construction crews each become the constraint at different times. AWS could sell every accelerator to the big frontier labs, he says, but it deliberately holds capacity for enterprises and startups. It says yes in some form to about 60% of requests, and plans to buy 2 million Nvidia GPUs over the next couple of years. ↗ 17:25
Single-Digit Concentration. Asked about a bubble, Garman points to customer concentration. Some neoclouds have concentrations of 30% to 60% with 1 or 2 customers, while AWS’s largest customer is a single-digit percentage at most. Most AWS usage is core compute, storage and inference for production work, and enterprises tell him, almost to a person, that they see positive returns at today’s costs. His argument: as with the internet bubble, some startups will fail, but durable businesses keep spending. ↗ 22:09
Bring Your Own Power. Garman says that 15 years ago, AWS simply asked the utility for more megawatts. Now it pays for its own solar, nuclear and other power projects, on the grid and behind the meter. AWS has been one of the largest buyers of renewable power every year for 10 years. Power and transmission planning runs 20 years out, and server planning spans several years instead of a few quarters. His point: no customer can plan its 2028 memory footprint, so AWS does it for them. ↗ 25:22
Always the Next Constraint. Garman borrows from The Goal, a book he read as an undergraduate: “there’s never one constraint, there’s always just the latest constraint.” When power eases, memory, TSMC capacity, HBM, networking parts or connectors take its place. Location matters too, since spare power in Indonesia does not fix a shortage in Germany. AWS tracks tens to hundreds of thousands of components, 4 to 5 tiers down its supply chain, a discipline it began about a decade ago. ↗ 28:07
The Silicon
Graviton’s 20/20 Math. Graviton grew out of Nitro, Garman says: AWS had acquired a company whose offload cards carried ARM cores, then asked what would happen if those cores became a server. The first Graviton was small and underpowered. Today it is 20% cheaper with 20% better performance, a gap he says has held for 5 to 6 years, and 90% or more of AWS’s top 100 customers use it. Some have halved their server count. He calls it the easiest way to cut an AWS bill. ↗ 35:46
Trainium Outgrew Its Name. AWS began Trainium about 5 to 6 years ago and now sells Trainium 3, sold out until roughly the end of next year, Garman says. Most Bedrock traffic runs on it. Anthropic, OpenAI and 6 to 12 startups build on it, and Trainium 4 is announced. He concedes AWS is bad at naming: its training chip may now be the best inference chip on the market, on both absolute and cost performance. ↗ 37:39
The Enterprise
Don’t Copy Bob’s Steps. Most enterprise agents are still simple and keep people in the loop, Garman says, though they already pay off. The first barrier is design. Companies have an agent repeat what an employee he calls Bob does, steps 1 to 5, then have Bob check the work. Garman urges a blank slate, because an agent can run massively in parallel and try 50 different things. The value comes from asking how a computer would solve the problem, not copying how a person does. ↗ 40:44
FDE With an Exit Date. The second barrier is trust, Garman says, because no enterprise wants an agent that deletes a production database. Few companies know how to build evals, label data or back-test in production to catch drift, and he is not sure anyone does it well yet. AWS and its partners are leaning into forward-deployed engineering. His model is a 45-day engagement that trains the customer, then leaves, because customers do not want to depend on an outside workforce for years. ↗ 42:48
Bedrock Keeps Data Home. Raghuram cites industry peers who say enterprises should post-train open models instead of sending all their data to a frontier model. Garman agrees that enterprise data is the most valuable asset. Bedrock guarantees data never leaves the customer’s VPC, and model providers never see prompts. Most open-weights post-training happens in SageMaker today. AWS took heat 3 years ago for being slow on AI, he recalls, but the vast majority of customers moving to production now land on Bedrock. ↗ 45:33
Security at Machine Speed. CEOs mostly ask Garman how they can trust agents to do what they intend. He sees powerful models as both a real attack risk and a real opportunity. AWS recently launched Continuum, which uses those models to find vulnerabilities customers had missed and rank them by context such as permissions and compensating controls. It exposes what AWS already does internally, he says. His line: customers will need “security at machine speed, not at human speed.” ↗ 49:28
Pods of 3 to 4. Software and product development show the biggest gains inside AWS, Garman says, and launches have sped up sharply in the last year. AWS calls the new units frontier teams: agents write all the code, and engineers manage a team of agents. Work that once took 10 people can now take 3 to 4, who finish so fast that AWS wants to move them to new problems. On how organizations should change, he says he has no magic answer yet. ↗ 53:46
Watch the full Podcast Here.