Follow the Money

1Follow the money

You already know this week's physics from your own pocket. Your phone gets warm when you game on it. Its battery dies by dinner on a busy day and lasts a weekend when it sits in a drawer — wait, no: it dies in the drawer too, just slower. Your laptop's fan spins up the moment a game starts. And the data centers that run the internet are built next to rivers and dams, because what comes out of a computer, at the end of every computation, is heat — megawatts of it, enough that the building's biggest engineering problem is plumbing.

Every bit of that is the same event, repeated beyond counting. Notes 08 taught you what a signal physically is: a bucket of charge, filled from the tank through a tap. A chip is billions of those buckets, and computing means filling and draining them, billions of times a second. Each fill is a withdrawal from the tank — from the battery, from the wall socket — and every withdrawal, once spent, becomes heat. The warm phone, the dead battery, the spinning fan, the river: one story, four costumes.

So tonight we open the ledger. The headline is simple: every joule a chip spends goes to one of exactly two destinations.

Section 2 prices the first: what one flip withdraws from the tank, and why one of the two knobs on that price is worth far more than the other. Section 3 prices the second: why a chip that computes nothing at all still drains its battery — the drawer mystery, solved. Section 4 is the designer's menu of fixes, and Section 5 is the law the whole menu obeys: joules and picoseconds are two currencies with an exchange rate, and you are always — always — trading one for the other.

2The price of changing your mind

Watch one node flip, in slow motion, with Notes 08's plumbing. The node is a bucket. Somewhere above it stands the tank — the supply, V_DD — kept full by the battery. When the node flips 0→1, a tap opens and the bucket fills from the tank: charge flows, and energy is withdrawn. When it flips back 1→0, a different tap opens at the bottom, and here is the detail the whole week turns on: the water is dumped down the drain. It is not pumped back up to the tower. The charge goes to ground, its energy becomes a puff of heat, and the next 0→1 flip starts the withdrawal from scratch. A chip is not a water cycle. It is a water bill.

What one fill costs — and the two knobs on the price

So how big is the withdrawal? Say it in words first, because the words carry the design wisdom of the whole industry: the energy to fill a bucket grows with the bucket's size, and with the square of how high you fill it. A bigger bucket costs more — no surprise. But raising the water level is punished twice: higher level means more water, and each drop is lifted higher. Twice the height, four times the energy. Once, for the record, in symbols — and this is the only formula tonight needs:

Read the two knobs straight off it. Knob one: make the buckets smaller — smaller gates, and above all shorter wires, because Notes 09 told you a wire is a brigade of buckets at 200 fF per millimeter; the biggest buckets on a chip are not gates at all. Every lesson about locality was secretly an energy lesson too. Knob two: lower the fill height — the supply voltage. And because V is squared, this knob is the whole ballgame: drop the supply from 1.0 V to 0.8 V and every fill costs 0.8 × 0.8 = 64% of what it did — a 36% discount for a 20% turn of one dial. No other lever in this course pays off that steeply. Hold that thought; section 4 reads its fine print.

From one flip to a power bill

A power bill is just the flip price multiplied up: power = energy per fill × how many buckets flip × how often they flip. Three factors — bucket size, fill height squared, and flips per second — and every power-saving trick ever invented attacks one of the three. Let us check the multiplication is honest:

Who actually flips? The city at night

The "how many flip" factor deserves one honest look, because it is smaller than you'd guess. Fly over a city at night: the city has a million windows, but only some lights are on, and only some of those are being switched this minute. A chip is the same. On any given tick, most buckets hold the value they held last tick — the adder is idle, that cache row wasn't touched, this block is between jobs. Engineers call the flipping fraction the activity factor, and for ordinary logic it is a few percent.

With one glaring exception. There is a network of wires whose entire job is to flip: the clock — Notes 10's metronome, physically delivered. The tick must reach every register on the die, so the clock is a tree of wires spanning the whole chip, and it flips up and down every single tick, by definition — activity factor as bad as it gets, on some of the longest, biggest-bucket wires on the die. The result, roughly, on many real chips: the metronome alone eats around a third of the whole power budget. Not the arithmetic — the beat. Keep that scandal in mind; it makes one of section 4's fixes obvious.

3The price of merely existing

Now the drawer mystery. The phone in the drawer computes nothing. No buckets are supposed to be flipping. Dynamic power says the bill should be zero — and yet the battery drains. Destination two:

Modern taps do not fully close. A transistor is a tap, and when we say it is "off," we mean almost off: a tiny trickle of charge seeps through anyway, from tank to drain, around the clock, computing nothing. Engineers call it leakage. The drip from one tap is absurdly small — but your chip is not one tap. It is billions of taps, every one of them dripping at once, and a billion drips is not a drip. It is a running faucet, open 24 hours a day, whether the chip is proving theorems or sitting in a drawer.

Why the taps got leakier — the barrier comes home

Why did it get so much worse? You already own the answer; you built it in Project 1. Back in week 2 a bit was a ball in one of two valleys, held in place by an energy barrier, and "Torture a Bit" showed you the rule: lower the barrier and the bit flips more easily — by accident as well as on purpose. A transistor's "off" state is exactly such a barrier, holding the electrons back on the tank side. Decades of making transistors smaller and faster meant, unavoidably, making that barrier lower and thinner — a lower barrier switches quicker and works at a lower fill height, which is everything section 2 wants. But electrons do not read datasheets. Give them a lower, thinner barrier and more of them seep over and through it while the tap is supposedly shut. Easy to flip and leaky are the same coin, week 2's trade-off wearing a transistor costume: the faster we made the taps, the worse they got at being closed.

Put the two destinations together and you get the wall Notes 10's lore named. As chips packed in more buckets and flipped them faster, dynamic power climbed; as the taps shrank, the faucet of leakage opened wider underneath it — until the total heat could no longer leave the package, and the gigahertz race stopped. The sequel is visible on every modern die: there is not enough power budget to run all the circuitry flat-out at once, so at any moment whole neighborhoods of the chip are deliberately kept dark and cold — designers literally call it dark silicon. Your laptop has eight cores instead of a 12 GHz clock, and it cannot even light all eight for long. The ledger on this page is why.

4The menu of fixes

Every chip design team stares at the same two-line bill — flips and drips — and reaches for the same short menu. You now own the physics behind every item on it.

Fix 1 — turn down the pressure

The squared knob first, because it pays best: lower the supply voltage. From 1.0 V to 0.8 V, every fill costs 64% of what it did; halve the voltage outright and every fill costs a quarter. No other move touches that. But Notes 08 already wrote the fine print: a tap fills its bucket slower at lower pressure — less push, a longer fill, a fatter R×C. Lower the supply and every gate on every route slows down, the critical path's bill grows, and Notes 10's budget arithmetic forces the metronome to slow with it. There it is, appearing all by itself: the joules you save are paid for in picoseconds. Sections 5's law, met in the wild. Worse, near the bottom there is a cliff: push the fill height down toward the barrier height and the taps barely open at all — delay doesn't just grow, it blows up. Bench 4.1 puts this cliff under your thumb.

Fix 2 — stop the metronome where nothing is happening

Section 2's scandal: the clock flips every tick, everywhere, and eats roughly a third of the bill — including in blocks that are doing nothing, dutifully grabbing the same unchanged values tick after tick. The fix is almost insultingly simple: switch off the clock wire to any block with no work. The block's registers stop flipping, its logic stops twitching, and that neighborhood's dynamic bill drops to zero. Engineers call it clock gating, and it is on essentially every chip made this century; blocks wake the instant the clock returns, so it costs almost nothing. What it does not stop: the drip. A clock-gated block still leaks — the taps are still attached to the tank.

Fix 3 — shut the water main

To stop the drip you must do something more drastic: disconnect the sleeping block from the tank entirely — one big tap on the block's supply, shut hard. That is power gating, and it kills both lines of the bill at once: no fills, no drips, true zero (near enough). The price is at wake-up: the block's plumbing is empty, every bucket must be refilled before the first honest tick, and that takes real time — not picoseconds but microseconds or more. So power gating is for real naps, not for blinks: the camera block while you type, the radio between check-ins, seven of eight cores overnight. Your phone plays this game constantly, and the drawer mystery's full answer is here: the drawer phone is mostly power-gated, but not entirely — something must stay awake to hear the alarm, and that something drips until the battery dies, weeks later instead of hours.

Fix 4 — race to sleep

Now the counterintuitive one. Suppose a job must be done, and when it is finished the chip can power-gate — truly off, no drip. Which is cheaper: amble through the job at low voltage, saving on every flip — or sprint at high voltage, pay more per flip, and shut the faucet sooner? Your instinct says the ambler wins; the squared knob says so too. But the ambler is awake longer, and the faucet runs the entire time it is awake. On a leaky chip, the drip while dawdling can out-cost the sprint's expensive flips. Honestly, with numbers from Bench 4.1's toy chip:

Fix 5 — turn the dial while driving

Finally, notice that Fix 1 need not be a design-time decision. Voltage and clock speed are both just settings, and your phone changes them on the fly, many times a second: reading email — drop to a low gear, slow and low, sip the battery; launch a game — snap to high voltage and full clock, burn joules for picoseconds; screen off — downshift to nearly nothing and start gating. Engineers call it dynamic voltage and frequency scaling (DVFS), but it is simply this section's whole menu, automated, with the energy–delay exchange rate recalculated every few milliseconds. The dial below is the one your phone is turning right now.

The chip under the dial is Notes 10's specimen: at 1.0 V its bill is the familiar 360 ps (max clock 2.78 GHz). Three things to find. One: slide from 1.2 V to 0.6 V and watch flip energy fall 144 → 36 pJ — exactly a quarter, the square — while the bill balloons from 261 to 1176 ps. Two: with the metronome at 2 GHz, creep the voltage down: at 0.84 V the route misses the tick, and you are back at Notes 10's Bench 5.1 — garbage, grabbed and held. Three: the solid curve's dip near 0.68 V — below it, operations take so long that the drip out-costs the flip savings. The cheapest voltage is not the lowest. That dip is race-to-sleep drawn as a curve.

Both phones must finish one billion operations, then power-gate (drip off). Watch the energy bars: the sprinter's flip bar jumps up fast but stops early; the ambler's grows slowly while its drip bar keeps creeping the whole time it is awake. At 0.10 W/V the ambler wins by a third; drag the slider past about 0.35 W/V and the verdict flips. "Hurry up and go to sleep" is a real strategy, printed in your phone's power-management firmware.

5The exchange rate

Step back and look at what every section tonight kept tripping over. Raise the voltage: faster taps, costlier flips. Lower it: cheaper flips, slower taps — until the tick is missed. Sprint: expensive but brief. Amble: cheap but leaky. It is always the same two currencies, and the law of the course's third act is this: you can buy speed with joules, and you can buy battery with picoseconds, but you cannot be handed both. Every chip ever shipped is a point on one curve, and designing a chip is mostly choosing where on it to stand.

One plain sentence on keeping score. If someone brags their chip uses half the energy, ask what happened to the delay; if they brag it is twice as fast, ask what happened to the joules. The honest scoreboard multiplies the two — energy per operation times time per operation — because sliding along the curve barely moves that product, while a genuinely better design (smaller buckets, tighter taps, shorter routes) shrinks it. Marketing slides along the curve; engineering moves it.

The term so far, re-read under this law

• Notes 09's energy chart — an add ≈ 1 unit, across the chip ≈ tens, out to memory ≈ hundreds to thousands — was this week in disguise: a wire is a brigade of buckets, and moving a bit means filling and spilling every one. Locality saves picoseconds and joules; it is the rare move that wins both currencies, which is why the whole industry is organized around it.
• Notes 10's pipelining now shows its price tag. Every relay station is a row of registers — real buckets, flipped by the clock every single tick, plus more clock wire to reach them. Pipelining buys gigahertz with joules: the deeper the pipeline, the bigger the metronome's cut of the bill. Nothing on the menu is free.
• The power wall, properly explained at last: the 2000s GHz race raised frequency (more flips per second) and voltage (squared!) and pipeline depth (more registers) all at once, while shrinking taps leaked ever harder underneath — until the package could not shed the heat, around the hundred-watt mark of our §2 worked example. The industry did not hit a cleverness wall. It hit this page's arithmetic, and swerved into multicore: several slower, cooler points on the curve instead of one incandescent one.

6Check yourself

Five questions, answers hidden behind each. Commit to your answer out loud before you tap.

→Work before Session 21

Session 21 is Monday, November 16. Before then:

• Review these notes. The memory test: name the two destinations every joule goes to; say why the voltage knob is squared and what the squaring costs you; and explain, in one sentence each, clock gating vs. power gating vs. race-to-sleep.
• Ten minutes per bench. On Bench 4.1, predict before you drag: what should flip energy do from 1.0 V to 0.8 V? (Commit to a percentage.) Then find the voltage where the 2 GHz metronome first misses, and the voltage where total energy per op bottoms out. On Bench 4.2, find the crossover leakiness to two decimal places.
• Redo the §2 worked example with your own machine: look up your CPU's advertised power budget and clock, and back out roughly how many 1 fF buckets flipping per tick it implies. Bring the number — everyone's will differ, and the comparison is the fun part.

And the tease. We have spent three weeks making the chip fast and cheap — trimming budgets, lowering voltages, shaving margins. Next week the bill for cutting margins arrives: noise. How low can the voltage go before a bucket's ordinary sloshing looks like a bit flip? Why does a wire's flip shake its neighbor? Week 2's barrier model comes home, and Notes 12 counts the errors.

§Symbols and constants