Amazon Web Services
So, how DID an online book retailer end up building the infrastructure layer that powers the entire internet? (Or at least 39% of it, per latest market share data.)
So, how DID an online book retailer end up building the infrastructure layer that powers the entire internet? (Or at least 39% of it, per latest market share data.) While many myths, legends, and some downright falsehoods exist, the real answer to that question deserves a full Acquired episode of its very own. So here it is: the story of Amazon Web Services. Who’s got the truth?
Kyle’s Rating: 10/10
This AWS episode deserves a 10/10 for excellent storytelling and systematically debunking four major myths about AWS's origins, revealing the messy reality of decentralized innovation across multiple teams rather than a single visionary moment. It's the iconic episode where Bezos's "Market Size Unconstrained" phrase originated, perfectly capturing how AWS created an entirely new category rather than just replacing existing IT spending.
Company Overview
Company Name: Amazon Web Services (AWS)
Founding Year: 2002 (initial division within Amazon Associates); reoriented as cloud infrastructure in 2003, with services launched in 2006
Headquarters Location: Seattle, Washington
Core Business and Significance: AWS provides scalable cloud computing services, including storage (S3), compute (EC2), and databases (e.g., RDS, Redshift), enabling businesses to deploy applications without physical servers. It pioneered the cloud industry, powering 39% of the internet and generating profits often surpassing Amazon’s retail business, revolutionizing technology infrastructure for startups, enterprises, and governments.
Narrative
The story of Amazon Web Services is a tale of technical brilliance, strategic foresight, and relentless execution that transformed Amazon from an online bookstore into the internet’s backbone. In the early 2000s, Amazon faced a dire challenge. The dot-com crash of 2000 had pushed the company to the brink of bankruptcy, forcing a shift from costly DEC Alpha servers to HP servers running Linux, a cost-saving move that kept Amazon afloat. By 2001–2002, however, a new crisis loomed: the monolithic codebase, built by Shel Kaphan in 1995, couldn’t scale with Amazon’s retail ambitions. Adding features risked crashing the system, leading to code freezes during the critical Q4 holiday season when traffic spiked. Jeff Bezos, an embattled CEO battling Wall Street skepticism and eBay’s competition, saw technology as a growth engine, not a cost center. Leveraging Moore’s Law—expecting compute power to double every 18 months—he viewed Amazon’s technical woes as an opportunity to redefine its future.
In 2002, a pivotal meeting with Tim O’Reilly, a book publisher and Web 2.0 advocate, sparked the initial AWS. Despite past tensions—O’Reilly’s publishing clashed with Amazon’s retail dominance—his pitch for participatory, API-driven websites resonated with Bezos. Amazon launched AWS within the Amazon Associates Program, led by Collin Brier, to provide product catalog APIs for affiliates, announced at a 2002 developer conference with just eight attendees. This modest effort planted the seed of externalizing Amazon’s capabilities, aligning with its affiliate program’s goal of sharing revenue through product links. Meanwhile, internal frustrations grew. The monolithic codebase slowed innovation, as teams feared breaking the system, particularly for high-margin initiatives like Amazon’s marketplace to rival eBay. Engineers faced a “rat’s nest” of complexity, as described in Working Backwards. Bezos, joined by Andy Jassy—his new technical assistant, a Harvard MBA hired in 2001 after nearly being cut from marketing—focused on this bottleneck. Jassy, inspired by Microsoft’s technical assistant model, became Bezos’s “shadow,” absorbing his strategic vision.
In 2003, Bezos issued a transformative mandate, famously recounted in Steve Yegge’s 2011 rant: all teams must communicate via “hardened” APIs, banning direct database access or inter-team coordination, with non-compliance risking termination. This service-oriented architecture (SOA) turned Amazon into a collection of independent, API-driven “startups,” each focused on customer-facing features. Inspired by O’Reilly’s Web 2.0 principles, this shift mirrored Amazon’s documentation-obsessed culture of six-pagers and PR/FAQs, reducing communication overhead described by Metcalfe’s Law. Concurrently, two parallel efforts shaped AWS’s cloud vision. Jassy’s 2003 six-pager proposed AWS as a cloud infrastructure business, requesting 57 hires—a bold move approved by the S-Team and board. Separately, network engineers Benjamin Black and Chris Pinkham proposed virtual compute servers. Pinkham, reporting to CIO Rick Dalzell, moved to Cape Town in 2004 to lead a new subsidiary, developing EC2 with Chris Brown. This decentralized innovation, reflecting Amazon’s “Invent and Wander” ethos, saw multiple teams tackling related problems. Tensions, like Pinkham’s avoidance of Bezos’s intense focus, later evident in Amazon’s omission of his EC2 role, highlighted the messy reality of this process.
By 2006, AWS launched its first services: S3 in March for scalable storage and EC2 in August for compute, both with pay-as-you-go pricing that stunned the industry. Engineer James Hamilton’s $3.08 S3 bill for a personal project exemplified this disruption, enabling startups like Dropbox, Instagram, and Airbnb to build without massive upfront costs. Unlike Oracle and IBM, tied to 80% margin license models, AWS targeted 20–40% margins, attractive compared to Amazon.com’s 2% retail margins. The 2007 AWS Startup Challenge, featuring Justin.tv (later Twitch), cemented AWS’s startup appeal. By 2009, enterprises like NASA and Netflix joined, with Reed Hastings’s 2012 re:Invent appearance endorsing AWS’s reliability despite retail competition. AWS’s 51 price cuts by 2015, proactive without competitive pressure, drove adoption, mirroring TSMC’s strategy of sacrificing short-term profits for market share.
AWS’s growth was staggering: from $6 billion in 2015 (19.2% margins) to $80 billion in 2022, with a $100 billion backlog. Its database business, including Redshift, targeted Oracle’s $100 billion market. Innovations like Snowmobile—a semi-truck for exabyte-scale data transfers—addressed migration challenges. Yet, AWS missed the data warehouse market, allowing Snowflake’s $50 billion rise due to its complex 200+ service portfolio and focus on Oracle rather than new segments. Despite this, AWS’s scale economies, switching costs, and customer obsession made it an unregulated utility, powering 39% of the internet and often outpacing Amazon.com’s profits. Jassy’s rise to Amazon CEO in 2021, with Adam Selipsky leading AWS, underscores its strategic centrality, positioning Amazon as a high-beta capital allocator in an unconstrained market.
Timeline
1995: Amazon.com launches with a monolithic codebase by Shel Kaphan, running on DEC Alpha servers.
2000: Amazon shifts to HP servers with Linux, cutting costs to survive the dot-com crash.
2001–2002: Codebase complexity stalls innovation; Jeff Bezos, under pressure, hires Andy Jassy as technical assistant.
2002: Tim O’Reilly pitches Web 2.0 and APIs; Amazon launches AWS within Amazon Associates, led by Collin Brier, for product catalog APIs (eight developers attend the first conference).
2003: Amazon mandates service-oriented architecture (SOA) with API-based team communication; Jassy’s six-pager proposes AWS as cloud infrastructure, approved for 57 hires; Benjamin Black and Chris Pinkham propose virtual compute servers.
2004: Chris Pinkham leads a Cape Town subsidiary to develop EC2.
March 2006: S3 launches, offering scalable storage with pay-as-you-go pricing.
August 2006: EC2 launches in beta for compute instances.
2007: AWS Startup Challenge begins, with participants like Justin.tv (later Twitch); startups like Dropbox and Airbnb adopt AWS.
2008: CloudFront (CDN) launches.
2009: RDS launches, supporting databases like Postgres; NASA becomes a customer.
2012: First re:Invent features Netflix’s Reed Hastings; AWS outlines six enterprise advantages.
2014: Bezos’s shareholder letter calls AWS’s market size “unconstrained”; ~$6 billion revenue run rate.
2015: AWS financials reported in Q1 (“AWS IPO”): $6 billion run rate, 70% growth, 19.2% margins; Annapurna Labs acquired for custom chips.
2016: Jassy becomes AWS CEO; AWS hits $12 billion revenue, over 50% of Amazon’s profits.
2017–2020: Revenue grows to $17 billion (2017), $25 billion (2018), $45 billion (2020), $62 billion (2021).
2019: Amazon.com migrates off Oracle to AWS after 13 years.
July 5, 2021: Bezos retires; Jassy becomes Amazon CEO, Adam Selipsky AWS CEO.
2022: AWS reaches $80 billion run rate, $100 billion backlog; conducts Snowmobile operation for the International Space Station.
Notable Facts
Market Dominance: AWS holds 39% of the cloud market, far ahead of Azure (21%) and Google Cloud (7%).
Revenue Backlog: $100 billion in contracted revenue ensures future stability.
Pricing Disruption: S3’s 2006 launch offered storage for pennies (e.g., $0.07/month), enabling rapid startup development.
Database Share: AWS’s databases (Redshift, Aurora) compete in a $100 billion market growing 10% annually.
Snowmobile: A semi-truck for petabyte-scale data transfers highlights AWS’s innovation in migration.
Financial / User Metrics
Revenue Run Rate (2022): $80 billion, up from $6 billion (2015).
Operating Margins: 19.2% (2015) to ~30% (2022).
Revenue Backlog: $100 billion, primarily enterprise contracts.
Growth Rate: 70% (2015) to 30–35% (2022).
Market Share: 39% of cloud computing.
Customer Base: Includes startups (e.g., Dropbox), enterprises (e.g., Netflix), and institutions (e.g., CIA); exact user numbers not provided.
The Real Origin Story of AWS
Ben and David debunk four origin myths for AWS, revealing a complex, decentralized truth:
Myth 1: Excess Capacity
Claim: Amazon rented out idle Q4 servers to create AWS.
Debunked: Werner Vogels’s 2011 Quora post confirms this is false; Amazon’s servers were customized, not shareable, and Q4 demand would disrupt external customers. AWS was always a distinct business.
Myth 2: Tim O’Reilly’s Web 2.0 Vision
Claim: O’Reilly’s 2002 pitch for Web 2.0 APIs directly led to AWS.
Debunked: While influential, this led to the initial AWS (2002) for product catalog APIs within Amazon Associates, not the cloud infrastructure launched later.
Myth 3: Jassy’s Vision Document
Claim: Andy Jassy’s 2003 six-pager centrally planned AWS’s cloud services.
Debunked: The document was pivotal but omits concurrent efforts; Jassy’s 2013 Harvard i-Lab interview claims he authored it, but other teams contributed independently.
Myth 4: Pinkham and Black’s Compute Proposal
Claim: Benjamin Black and Chris Pinkham’s 2003 six-pager solely birthed EC2.
Debunked: Their work was critical for EC2, but AWS’s broader scope (e.g., S3, RDS) involved multiple teams, not a single proposal.
True Story: The real origin of AWS lies in Amazon’s decentralized innovation, sparked by internal crises and visionary leaps between 2001 and 2003. Facing a monolithic codebase that stalled growth, Jeff Bezos and Andy Jassy tackled scalability by mandating a service-oriented architecture (SOA) in 2003, requiring teams to communicate via “hardened” APIs, as detailed in Steve Yegge’s 2011 rant. This cultural and technical shift, inspired by Tim O’Reilly’s 2002 Web 2.0 pitch, aimed to speed up Amazon.com’s innovation. Concurrently, network engineers Benjamin Black and Chris Pinkham proposed virtual compute servers in a 2003 six-pager, envisioning externalized IT infrastructure. Jassy’s separate six-pager, approved for 57 hires, outlined a broader cloud vision, merging these ideas. Pinkham’s Cape Town team developed EC2, launched in 2006 alongside S3, which addressed storage needs. This decentralized approach—multiple teams iterating independently—aligned with Amazon’s “Invent and Wander” ethos, allowing AWS to evolve from internal necessity to a market-defining business. The episode emphasizes that execution, not a single idea, drove success: AWS’s primitives (S3, EC2) met developer needs with disruptive pricing, enabling startups and later enterprises. Tensions, like Pinkham’s omission from Amazon’s official history, reflect the messy reality of innovation, but Jassy’s leadership unified these efforts into a cohesive strategy, launching AWS as the internet’s operating system.
Transaction
The episode focuses on AWS’s organic growth, not a specific acquisition or IPO. However, the 2015 “AWS IPO” (Q1 earnings breakout) revealed $6 billion revenue, 70% growth, and 19.2% margins, boosting Amazon’s stock 15%. This signaled AWS’s profitability, driving Amazon’s valuation.
Grading
Ben and David award AWS an A+, praising its creation of an unconstrained cloud market and sustained leadership. The grade reflects AWS’s trajectory (2002–2022), not a transaction, with a minor critique for missing the data warehouse market (Snowflake).
Bear Case and Bull Case
Bear Case: Not explicitly stated, but implied challenges include:
Competition: Azure and Google Cloud gain share via multi-cloud strategies.
Snowflake Miss: AWS’s complex portfolio allowed Snowflake’s $50 billion rise.
Cash Flow: Data center investments drive negative free cash flow.
Bull Case: Implied by AWS’s strengths:
Market Growth: $120 billion cloud market, growing 30% annually.
Lock-In: $100 billion backlog and high switching costs ensure stability.
Margin Expansion: PaaS offerings (e.g., Lambda) boost margins to 30%.
Trust: Contracts with CIA, NASA reinforce reliability.
Tech Trends
Web 2.0 and APIs: O’Reilly’s 2002 pitch inspired AWS’s initial APIs and SOA.
Service-Oriented Architecture: 2003 mandate enabled scalable development and cloud infrastructure.
Moore’s Law: Cost reductions fueled AWS’s pricing advantage.
Cloud Primitives: S3, EC2 met developer needs, driving adoption.
Machine Learning: SageMaker, Trainium leverage data proximity for lock-in.
Powers
Counter Positioning: AWS’s IaaS model, with 20–40% margins, disrupted incumbents like Oracle and IBM, who clung to 80% margin license models. By targeting startups with pay-as-you-go pricing (e.g., $3.08 S3 bills), AWS captured an uncontested market for five years, as legacy vendors avoided cannibalizing their businesses. This allowed AWS to define cloud computing, aligning with its low-margin retail roots.
Scale Economies: Ben calls AWS “maybe the best scale business of all time.” Its massive data center investments, amortized across a vast customer base, enable lower prices and higher margins (30% by 2022). As the largest cloud provider (39% share), AWS spreads capex over more users than Azure or Google, reinforcing cost leadership. Moore’s Law further reduces compute costs, amplifying this power.
Switching Costs: High switching costs lock in customers, as seen in Amazon.com’s 13-year Oracle migration. Enterprises moving petabytes via Snowballs or Snowmobiles (e.g., semi-trucks for exabyte transfers) face significant time and cost barriers. The $100 billion backlog reflects contractual commitments, making defection impractical and ensuring long-term revenue.
Branding: AWS’s leadership and reliability (e.g., CIA, NASA contracts) create a perception of safety, encouraging customers to choose AWS over less consistent competitors. Its “fastest growing service” claims reinforce trust, reducing perceived risk despite early multi-cloud hesitancy.
Playbook
Best Scale Business of All Time: AWS is an unregulated utility for the internet, requiring massive upfront investments (e.g., data centers driving negative cash flow) but now a cash machine with $80 billion run rate and 30% margins. Like Amazon.com’s fulfillment network, AWS amortizes fixed costs (data centers vs. warehouses) across a vast customer base, leveraging scale to offer low prices while generating billions in profits. This mirrors Amazon.com’s cashflow.com model but lacks its negative cash conversion cycle, as AWS funds infrastructure upfront.
Evolution of Cloud Meaning: Initially, “cloud” meant AWS’s IaaS primitives (S3, EC2) in Amazon’s data centers with pay-as-you-go pricing. It evolved to include proprietary PaaS services (e.g., Lambda, Aurora) and hybrid/multi-cloud models, where AWS services run in customer data centers or alongside Azure/Google. This shift, driven by enterprise needs for flexibility, blurs “cloud” as API-accessed IT, creating complexity but expanding AWS’s reach.
Make Something People Want: AWS’s customer obsession, echoing YC’s slogan, focused on developer needs (e.g., S3 for startups, RDS for enterprises). Every service ties to a use case, avoiding speculative trends like blockchain until customer demand clarified. This ensured rapid adoption, from hackathons to NASA contracts.
Asymmetric Upside: Bezos’s 2015 letter highlights taking bets with “100 times payoff” despite frequent failures: “In baseball, you have a truncated outcome distribution… In business, you can score a thousand runs.” His 2014 claim of AWS’s “market size unconstrained” reflects this, with AWS as a high-beta venture bet yielding massive returns, unlike retail’s iterative failures (e.g., Fire Phone).
Service Longevity: AWS maintains services (e.g., SimpleDB) to preserve enterprise trust, unlike Amazon.com’s rapid iteration, leading to a complex 200+ service portfolio.
Carveouts
Ben: Moon Knight (Disney+), lauded for Oscar Isaac’s exceptional acting, though less well-written than Loki or WandaVision.
David: Lex Fridman’s 5.5-hour interview with John Carmack, praising his technical genius and id Software history, complementing Masters of Doom.
Additional Notes
Episode Metadata:
Number: Season 11, Episode 3
Title: Amazon Web Services
Duration: 2:43:50
Release Date: September 5, 2022
Sources: Acquired website, Apple Podcasts.
Related Episodes:
Amazon.com (S11E2, August 16, 2022)
Amazon Unbound with Brad Stone (May 26, 2021)
Amazon IPO with Tom Alberg (S1E28, December 31, 2016)