How Data Centers Actually Work, Explained Simply

Every time you push code, run a search, or ask an AI to write your commit message, your request does not float off into some abstract “cloud.” It lands in a real building. A building with the power draw of a small city, security that makes a bank vault look relaxed, and enough fiber inside it to wrap around the planet.

That building is a data center. This guide opens one up: what it is, the four types you will run into, how power and cooling and security actually work, what the Tier ratings mean, how your request travels through it, and why AI has forced the biggest redesign of these buildings since the cloud arrived.

What is a data center?

A data center is a building designed to house servers, storage systems, and networking equipment, plus the power and cooling that keep them running without interruption. It is the backbone of everything digital, from loading a web page to training an AI model. Whenever you do anything online, a data center somewhere is doing the work.

Calling it “a room full of servers” undersells it badly. That is like calling a hospital “a building with beds.” The servers are the easy part. The engineering is in everything around them: the electrical system that never blinks, the cooling that hauls out the heat, the layered security, and the network links to the rest of the internet. Get any of those wrong and the servers are useless.

What are the four main types of data centers?

There are four kinds of data center worth knowing, and they differ mostly by who owns them and how big they are. Enterprise centers serve a single company. Colocation centers rent space to many. Hyperscale centers are the cloud giants. Edge centers are the small ones near you. Most of the internet you touch runs on the last two.

Enterprise One owner

Built and run by a single company for its own workloads, often for compliance. A bank running its own hall.

Colocation Rented racks

A shared building where you lease rack space and the operator supplies power, cooling, and security. Co-working for servers.

Hyperscale 20–100+ MW

The giants behind cloud and AI. AWS, Microsoft, Google, and Meta run campuses of thousands of servers across many buildings.

Edge Under 100 kW

Small, distributed sites placed close to users to cut latency. Cell towers, retail back rooms, factory floors.

The shift over the past decade has been dramatic. According to Synergy Research, hyperscale operators ran 1,189 of these giant facilities by early 2025, up from a few hundred a decade earlier, and they are on track to hold more than 60% of all data center capacity by 2030. Companies that once ran their own halls now rent from AWS, Azure, or Google instead. The building did not disappear. It moved, and it got much bigger.

Inside the server hall: racks, power, and cooling

Walk into the server hall and the first thing you see is order. Servers sit in standardized 19-inch racks, usually 42 slots tall, lined up in rows. Those rows alternate between cold aisles and hot aisles. Servers pull cool air in the front and blow hot air out the back, and containment panels stop the two streams from mixing. That simple airflow trick is the foundation of keeping the room efficient.

Then there is power, and this is where the engineering gets serious. Utility electricity arrives at high voltage, gets stepped down by transformers, and flows through switchgear toward the racks. The critical link in that chain is the UPS, the uninterruptible power supply. The moment the grid drops, its batteries carry the load within milliseconds while diesel generators start up to run for hours or days.

1. Utility Grid feed, 10–35 kV
2. ▶
3. Transformer Steps down voltage
4. ▶
5. Switchgear Routes + protects
6. ▶
7. UPS Battery bridge
8. ▶
9. PDU Splits to racks
10. ▶
11. Rack Servers draw power

Backup When the grid drops, UPS batteries hold the load within milliseconds while diesel generators spin up to run for hours or days.

How much of this gets duplicated depends on the facility’s Tier, which we will get to. The point is that a serious data center never has a single power path. There is always a second route, and often a backup for the backup, so a technician can service one side while the other carries the load.

Cooling is the part being reinvented right now, and AI is the reason. Traditional air conditioning works fine up to about 15 to 20 kilowatts per rack. But an NVIDIA GB200 NVL72 GPU rack draws around 132 kW, and the next generations may need far more. Air simply cannot move that much heat, so the industry is switching to liquid.

Air cooling 15–20kW / rack

CRAC and CRAH units push cold air under raised floors. Hot and cold aisles keep the airstreams apart. Fine for general compute, then it runs out.

vs

Liquid cooling 132kW / rack

Direct-to-chip cold plates and immersion tanks move heat with fluid, not air. An NVIDIA GB200 NVL72 GPU rack needs it. Next-gen racks may hit 250–900 kW.

Two liquid approaches lead. Direct-to-chip cooling pipes coolant onto cold plates sitting right on the CPUs and GPUs. Immersion cooling goes further and submerges whole servers in a non-conductive fluid. Both can cut cooling energy by close to half and slash water use compared with old evaporative systems, which is why they are moving from experiments to standard kit fast.

How do data centers stay secure?

Physical security in a data center works in concentric layers, a model called defense in depth. You do not cross one barrier to reach a server. You cross four, and each one is harder than the last. The idea is simple: even if someone defeats the outer ring, three more stand between them and the hardware.

1

Perimeter Fences, bollards, AI cameras, 24/7 guards

2

Facility access Badge readers, mantraps against tailgating

3

Computer room Biometrics: iris, fingerprint, palm vein

4

Rack cabinet Electronic locks, logged door openings

Server

The outer ring is the perimeter: fences, bollards, lighting, cameras with AI analytics, and guards on site around the clock. Next is facility access, with badge readers and mantraps, those interlocking double-door booths that stop someone tailgating in behind a badged employee. The third ring guards the computer room itself with biometrics, often an iris or palm-vein scan. The innermost ring is the rack cabinet, with its own electronic lock and a camera tied to every door opening.

All of it is audited. Every entry is tracked, timestamped, and recorded, so there is a full trail of who touched what. The same care goes into fire suppression and leak detection, because a flood or a spark in the wrong place is just as dangerous as an intruder.

What are data center tiers (Tier 1 to Tier 4)?

Data center tiers are a four-level rating from the Uptime Institute that describe how much redundancy a facility has, and therefore how much downtime it risks. The scale runs from Tier I, a single path with no backup, to Tier IV, fully fault tolerant with a backup for every component. Each tier includes everything the lower ones require.

Most production cloud workloads live in Tier III facilities, which hit the sweet spot: you can do maintenance without taking anything offline, without paying for the full fault tolerance of Tier IV. One caveat worth knowing is that the old “99.982% uptime” figures you see attached to tiers are legacy marketing, not official guarantees. Real availability depends as much on how a site is run as on how it was built.

Anatomy of a request: how your code actually runs

So what happens when your code serves a real request? The path is shorter than you would think and more redundant than you would guess. DNS points the request at the nearest healthy data center. It enters over redundant fiber, passes border routers and firewalls, and a load balancer picks a server that is free and healthy. The app runs, hits storage, and the answer heads back out.

1. 01 DNS Resolves you to the nearest healthy data center.
2. 02 Ingress Redundant fiber, border routers, firewalls.
3. 03 Load balancer Picks a free, healthy server for the request.
4. 04 Compute Your app runs, calling microservices as needed.
5. 05 Storage Reads and writes hit SSDs or object stores.
6. 06 Response Travels back out, often cached at a CDN edge.

The part you never see is the internal traffic. A single request to your app might fan out into dozens of calls between microservices, all flowing sideways between servers. That lateral traffic is called east-west, and modern data centers are built around it with a layout called spine-leaf. Every leaf switch connects to every spine switch, so any server can reach any other in exactly two hops.

That two-hop guarantee is the whole point. It gives predictable latency no matter which servers are talking, and if any single switch dies, traffic just takes another path. Scaling up means adding more spines or leaves rather than redesigning the network. The same redundancy runs through every layer: servers cluster, data replicates across availability zones, and the system routes around failure before a human even gets paged.

Why is AI breaking data centers?

AI is breaking the old data center design because it changed the one number everything else is built around: power per rack. Ordinary servers draw 5 to 10 kW. AI GPU clusters draw ten to twenty times that. That single shift cascades into new cooling, new power delivery, and a buildout so large it is reshaping national energy markets.

CapEx 2026 $600B+

projected combined data center spend by the top hyperscalers, up about 36% on 2025

Power by 2030 945 TWh

IEA base case for global data center electricity, roughly double today and near 3% of all power generated

Per GPU rack 132 kW

draw of one NVIDIA GB200 NVL72 rack, versus 5–10 kW for traditional compute

The spending is hard to fathom. The largest hyperscalers, Amazon, Microsoft, Google, Meta, and Oracle, are projected to spend over $600 billion on data center infrastructure in 2026, up about 36% on the year before. Put differently, tech’s capital spending as a share of the economy now rivals the great infrastructure projects of the last century.

The power demand is just as steep. In its 2025 Energy and AI report, the IEA projected global data center electricity roughly doubling to around 945 TWh by 2030, close to 3% of all the electricity generated on Earth. That is why hyperscalers are signing deals straight with nuclear operators. Microsoft is backing the restart of Three Mile Island, Amazon is funding new small modular reactors, and Meta locked in a 20-year deal to keep an Illinois nuclear plant running. The appetite for compute is now an appetite for power.

Where data centers go next

A few trends are converging fast. Cooling is moving past liquid-on-chip toward microchannels etched into the silicon itself, with TSMC and Microsoft both working on it. Edge data centers keep spreading to serve latency-sensitive work like autonomous vehicles and real-time gaming. And digital twins, full virtual copies of a facility, are letting AI manage power and cooling with less and less human input.

The throughline is that sustainability has turned into a hard engineering constraint rather than a marketing line. New designs target water-free cooling, waste-heat reuse, and scheduling that shifts heavy jobs to when clean energy is cheapest. The pressure of the AI buildout is forcing the industry to get efficient, because at this scale even small percentage gains add up to whole power plants.

So the next time you run git push, fire an API call, or watch a model answer in real time, picture the building behind it. Power that never blinks, cooling reinvented for GPUs, four rings of security, a two-hop network, and an increasing number of very expensive chips. That is a data center, devsplained.