How Google Actually Works: A Systems Architecture of Planet-Scale Computing

A search query returns in a fraction of a second. In that time the request enters Google’s edge network, is balanced across front ends, fans out into dozens of services, reads state from distributed databases and storage, and may touch a machine-learning accelerator. None of that is visible, and the invisibility is the engineering achievement.

This is the companion note to a 47-page paper that reconstructs that machinery from the public record — GFS (2003), MapReduce (2004), Bigtable (2006), Spanner (2012), B4 (2013), Borg (2015), Jupiter (2015), Maglev (2016), Andromeda (2018), Orion (2021), and the current Colossus and TPU material. The paper treats the stack as one system rather than a catalogue of products, and its central claim is narrower than the usual “Google is huge” story: Google’s defining skill is not building enormous machines, but containing failure and heterogeneity behind abstractions that behave like ordinary resources.

Full paper (PDF, 47 pages): How Google Actually Works: A Systems Architecture of Planet-Scale Computing

The short version

• A Google data centre is engineered as a single computer: the individual server is a component, not the unit of reliability.
• Failure is the normal case. Detection, redundancy, recovery and abstraction do the work that reliable hardware would otherwise do.
• Every layer answers the same question — how do you turn thousands of imperfect components into one logical resource?
• The architecture is not finished. AI workloads are pulling networking and storage out of the supporting cast and into the computation itself.

One high-level map of the stack, in the paper’s own framing:

graph TD
    U["User request"] --> E["Edge cache + Maglev software load balancing"]
    E --> G["Google Front Ends: TLS, auth, routing, policy"]
    G --> S["Application services"]
    S --> B["Borg: fleet scheduling, packing, isolation"]
    S --> D["Spanner / Bigtable: structured state"]
    D --> C["Colossus: distributed storage substrate"]
    B --> T["TPU pods: 3D-torus interconnect"]
    S --> N["Jupiter fabric, B4 / Orion wide-area network"]

The map is conceptual by design — Google does not publish a packet-by-packet production blueprint — but the relationships between layers are the useful part.

Why the single-machine model breaks down

Vertical scaling on one machine eventually hits economic, physical and reliability limits. The sharper problem is the failure domain: one machine means one narrow boundary between “working” and “gone”. Horizontal scaling removes that, but changes what the system has to solve. Machines no longer share memory, communicate over links that can drop packets, run at different speeds, and fail independently.

At that size, failures stop being incidents and become a rate. So the architecture is built around a single identity:

Two consequences follow. First, coordination becomes a first-class problem: once state is distributed, correctness is no longer local, and clock skew, message loss and partial failure all have to be reasoned about explicitly. Second, latency stops being about averages. A single request fans out to many machines, so the slowest relevant component determines what the user experiences — the “tail at scale” problem that Dean and Barroso formalised in 2013. That is why Google optimises for p99/p99.9 rather than the mean, and why techniques such as hedged requests (send a duplicate, take the first acceptable answer) are worth real extra work.

The edge: software load balancing with Maglev

Before any application logic runs, traffic has to reach the right service. Google Global Cache historically served cached content close to users; behind it, Maglev turns load balancing into a distributed software service instead of a vertically scaled hardware appliance. That choice is the pattern in miniature: add capacity by adding nodes, route around node failure, and avoid a single bottleneck that would otherwise sit in front of everything Google serves.

Maglev uses consistent-distribution techniques to keep requests attached to the same backend while the backend set changes, and ECMP spreads traffic across the fleet of Maglev instances at the network layer. Behind it, Google Front Ends (GFEs) terminate secure connections, enforce routing and policy, authenticate or authorise where appropriate, and convert network traffic into service-level requests.

Compute: Borg turns the fleet into one pool

An application does not ask for a server. It asks for resources, and Borg decides where they come from — scheduling, admission control, task packing, overcommitment, isolation, monitoring, naming and fault recovery across tens of thousands of machines in the published 2015 design.

The scheduling problem is a multidimensional bin-packing problem. Given machines with 16 CPU and 64 GB, 8 CPU and 128 GB, 32 CPU and 64 GB, and workloads with different shapes, the scheduler has to pack well, honour placement constraints, and protect critical workloads. Overcommitment is deliberate: unused CPU is purchased silicon, power, cooling and floor space, so utilisation is an economic variable, not an aesthetic one.

The architectural consequence is that a workload is never tied to a failed machine. Rescheduling is part of scheduling, not an exception handler. Kubernetes inherited much of that model — declarative placement, cluster-level resource management, recovery, service discovery — though it is a re-derivation for a broader ecosystem rather than a copy of Borg.

Storage: from GFS to Colossus

GFS distributed large files across many commodity machines with a single metadata master. That was the right trade-off for large-scale batch processing over cheap disks in 2003, and it became the wrong one as workloads grew: one master is one ceiling.

Colossus keeps the GFS lineage but separates metadata operations from bulk data movement. A client asks the metadata layer where the data lives, then transfers data directly to storage servers — so the metadata authority never sits in the middle of every byte. Metadata is served by horizontally scalable curators and backed by Bigtable. Underneath, D-servers hold the physical devices, and the logical file is an abstraction over the distributed layout.

Durability is treated as an active process rather than the presence of copies. Colossus combines erasure coding (including Reed–Solomon schemes) with checksums, background scanning, proactive repair of degrading media, parity reconstruction and asynchronous garbage collection. Erasure coding lowers the storage overhead of naive replication at the cost of computation and network traffic during encode and repair — a trade the paper makes explicit rather than glossing.

The published scale figures are worth seeing, with the caveat the paper attaches to them:

These describe particular configurations, not Google’s global totals, and they are not additive. Newer material extends the same substrate toward AI: Rapid Storage on Colossus targets very high bandwidth, high request rates and low latency, because a fast accelerator waiting on storage is a wasted accelerator.

Structured data: Bigtable’s table abstraction

A filesystem answers where are the bytes. Bigtable answers what do these bytes mean to the application, presenting a sparse, distributed, sorted map as an ordinary table. Rows are partitioned into tablets and spread across many servers.

Partitioning buys horizontal scale, distributed throughput and the ability to move hot or oversized partitions. It also costs: cross-partition coordination, more complicated transaction semantics, harder failure handling, and data movement during rebalancing. It is the recurring trade in one sentence — scaling out makes capacity easier and coordination harder.

Global state: Spanner and TrueTime

Some workloads need more than scale; they need transactions that behave as if there were one database. Spanner provides externally consistent transactions across geographically distributed replicas, combining partitioning, synchronous replication, Paxos-based consensus and a time abstraction called TrueTime.

The problem TrueTime solves is ordering. If two transactions touch related state in different regions, clocks drift and network delays vary, so “which happened first” becomes ambiguous. Instead of pretending clocks agree, TrueTime returns an interval:

The interval is kept tight with GPS and atomic-clock references. Because uncertainty is modelled explicitly, Spanner can reason about order without assuming it — and it uses a commit-wait step, waiting until the uncertainty window has passed before acknowledging certain commits. The cost is latency; the payment buys the strongest property a globally distributed database can offer. Replicas then commit through quorum: correctness comes from distributed agreement, not from the hope that every replica is always reachable.

The network as a computer: Jupiter, B4, Orion, Andromeda

At this scale the network cannot stay passive plumbing, so Google programs it.

• Jupiter is a multi-stage Clos fabric inside the data centre, giving path diversity and aggregate bandwidth rather than dependence on any single route; the published generation demonstrated over 1 Pbps of bisection bandwidth, and later generations add optical circuit switching.
• B4 is the software-defined WAN between sites. With the Bandwidth Enforcer (BwE) applying a global view of demand, the operator can prioritise interactive user traffic over background transfers during congestion — relevance that only grows when AI training creates enormous long-lived flows.
• Orion moves the SDN control plane itself into a distributed, scalable service, so the network changes by software rather than by configuring devices one at a time.
• Andromeda virtualises the network for cloud workloads — isolation, provisioning, packet processing — with a Hoverboard model and an OS-bypass fast path so that virtualisation does not make the physical network unusably expensive.

AI infrastructure: the pod is the unit, not the chip

Tensor Processing Units are the clearest case of Google designing hardware and software together. The published TPU v5p specification in the paper: 8,960 chips per pod, roughly 459 TFLOPS of BF16 peak compute per chip, roughly 95 GiB of HBM per chip, and a 3D-torus-style interconnect.

The important number is not the FLOPs. A TPU is only useful as part of a distributed system: it must fetch model parameters, exchange gradients with peers, read training data, coordinate collective operations, checkpoint state, and integrate with scheduling and the network fabric. That is why AI infrastructure cannot be studied as “just a GPU problem”. The paper’s formulation:

Under that pressure, compute, storage and networking stay conceptually separable but become operationally entangled. The optimisation target shifts from maximum compute throughput to maximum useful end-to-end application throughput.

Three requests, end to end

A search query. Search does not scan the live web per request; it works from large precomputed indexes and distributed retrieval systems. Historical and publicly discussed components (SuperRoot, Ascorer, Twiddler, SnippetBrain, Glue) illustrate how one query decomposes into specialised stages — but the paper is explicit that these are illustrative examples, not a map of the 2026 production stack.

A storage write. The client asks the metadata layer for placement, receives locations, and writes data directly to storage servers. Metadata scales independently of bulk data movement. Control plane and data plane are distinct, which is the same pattern as Maglev, B4 and Borg.

An inference request. Admission, batching, routing to an accelerator pod, parameter and cache retrieval from storage, collective work across the interconnect, and a response. The specifics of Gemini serving are proprietary; the layered shape is not.

The trade-offs Google actually accepts

The paper is careful to present these as decisions with costs, not as universal improvements:

The summary is one line: Google does not eliminate complexity. It moves complexity into software, automation and fleet management, and then hides it behind a simple interface.

Six lessons that generalise

Most teams will never run a fleet this size, but the principles survive the trip down:

1. Design around logical resources. Depend on a service, file, table or queue — never on machine 17 or disk 42.
2. Assume failure. Ask “what happens if this component disappears?” before assuming it stays healthy.
3. Scale horizontally where the bottleneck is inherently parallel, and accept coordination as the price.
4. Separate control plane from data plane. Metadata and control traffic should not become the bottleneck for bulk data.
5. Measure the tail, not the mean. Average latency hides exactly the behaviour users experience.
6. Put complexity in the right layer. A reliable system can be complex inside while staying simple at its interface.

Five things almost everyone gets wrong

• “Google is one giant computer.” It is a distributed environment of large fleets — machines, disks, links, accelerators — coordinated in software.
• “Google searches the live web on every query.” It retrieves from precomputed indexes; crawling and indexing are a separate pipeline.
• “Cloud storage is just a folder on a disk.” A logical file is an abstraction over a distributed storage system with repair, tiering and reconstruction.
• “More replicas always solve reliability.” Replication costs capacity and does nothing about correlated failure or coordination.
• “The network is just a cable between servers.” It is a programmed distributed resource with scheduling, prioritisation, virtualisation and accelerator interconnects.

What the paper cannot tell you

A rigorous account has to separate evidence from inference, and the paper does:

• Google does not publish a live inventory of its fleet, so exact server counts are not established facts.
• Jupiter and B4 are published architectures, not complete physical topologies of every current data centre.
• Current Search ranking internals and end-to-end Gemini serving remain proprietary.
• A paper from 2003, 2012 or 2015 documents a design generation. It is evidence about trajectory, not a blueprint of what runs today.
• Scale figures belong to specific published configurations and should not be summed into a global claim.

That discipline is the most transferable thing in the paper. Most confident writing about hyperscale infrastructure fails precisely because it blurs historically documented systems, current public material and architectural inference into one voice.

Further reading

The paper’s bibliography is the useful reading list here: GFS (SOSP 2003), MapReduce (OSDI 2004), Bigtable (OSDI 2006), Spanner (OSDI 2012), B4 (SIGCOMM 2013), The Tail at Scale (CACM 2013), Borg (EuroSys 2015), Jupiter Rising (CACM 2016), Maglev (NSDI 2016), Andromeda (NSDI 2018), Orion (2021), and The Datacenter as a Computer.

Citing this paper

Bagtharia, H. R. (2026). How Google Actually Works: A Systems Architecture
of Planet-Scale Computing. https://blog.harshrb.in/posts/how-google-actually-works/

Read the full paper (PDF, 47 pages): How Google Actually Works: A Systems Architecture of Planet-Scale Computing — the request walkthroughs, failure analysis, architectural trade-offs and evidence notes are all expanded there.

The other half of this research looks at demand systems rather than infrastructure: Commercial Real Estate Lead Generation: An End-to-End Systems Architecture.