The Invisible Trillion · Data Centers: The Cathedrals Nobody Visits

The Building

Let’s go inside one. A medium-large modern data center — about 50,000 square feet, or roughly the size of a big-box hardware store.

From the outside, there is nothing to see. No logos, usually. No windows. The walls are concrete, reinforced. There are loading docks and transformers and sometimes giant diesel tanks on the side for backup generators. A chain-link fence with barbed wire. A single entrance with a guard. Security cameras everywhere. Some facilities have vehicle barriers designed to stop a ramming truck.

Inside the entrance, you go through a series of airlocks — security checkpoints where badge access is tested, sometimes with biometrics, sometimes with mantrap doors that only let one person through at a time. You sign NDAs. You are escorted.

Then you come to the main floor.

The first thing you notice is the noise. Data centers are loud. Not crashing, alarming loud — a continuous, deep, enormous hum, plus a higher whine from thousands of fans, plus the lower rumble of air conditioning. In the hot aisles (more on those in a moment) you actually need hearing protection; the noise level reaches 90 dB, which is roughly a lawnmower that never stops. If you stand still for five minutes, your ears start to ache.

The second thing you notice is the cold. The data center floor is kept at around 18–22°C (65–72°F), sometimes colder. There is a breeze — a real breeze, constant — from the cooling system. If you wear a short-sleeved shirt, you will be uncomfortable within twenty minutes.

The third thing you notice is how empty it is of people. A modern data center that occupies a football-field-sized room might have zero humans in it most of the time. Technicians come in for specific tasks — replacing a failed drive, installing new equipment, fixing a broken fan — and then leave. The servers run themselves. If you walked in unannounced on a Tuesday at 2 pm, you would very likely be the only human being in the room.

The Racks

The main floor is full of racks. A rack is a metal frame, about the size of a tall, thin refrigerator, designed to hold standardized computer equipment.

Racks are arranged in long rows, often 50 or 100 feet long, sometimes longer. The rows are oriented so that the fronts of the racks face each other across a “cold aisle” — the aisle from which cold air is supplied — and the backs of the racks face each other across a “hot aisle” — the aisle into which the hot exhaust is dumped. Air flows from cold aisle, through the servers, out into the hot aisle, and then back up to the cooling system, where it is re-cooled and recirculated. This “hot aisle / cold aisle” architecture is one of those small but important innovations that revolutionized data center efficiency in the 2000s. Before it, cooling was a mess of mixed air flows. Now it’s a clean loop.

Each rack holds somewhere between 20 and 50 servers. A server is a specialized computer — no monitor, no keyboard, designed to be stacked in a rack and remotely managed. Each server is about the thickness of a pizza box (and they’re measured in “U,” where 1U is about 1.75 inches; a 2U server is the thickness of two pizza boxes). A typical server has some combination of CPUs, GPUs, memory chips, storage, and network interfaces. It has a lot of fans, because it throws off a lot of heat.

A rack might have: - 20 servers, each with 2 CPUs, 16 memory sticks, 4 network cards, and a bunch of storage. - Or, for AI workloads, 8 servers, each with 8 of the most powerful GPUs plus a couple of CPUs and a lot of high-speed memory. An 8-GPU AI server can cost $200,000 to $400,000 on its own. - Or, for memory-intensive workloads, a handful of “fat” servers with 4 terabytes of RAM each.

The rack itself has power distribution, networking switches, and sometimes special cooling plumbing at the top or bottom. The front door of the rack has a cable-management system that looks like an elegant spiderweb if the data center is well-run and like a hairball if it isn’t.

A typical medium-large data center has somewhere between 1,000 and 5,000 racks. That’s 20,000 to 250,000 servers, depending on density. The very largest hyperscale facilities — the Google and Meta and Microsoft campuses that are essentially dedicated to one company — have over a million servers across multiple buildings on one site.

All 19 chapters of The Invisible Trillion · Library · All courses