The Physical Cost of Reliability

1The adversary: kT

We have been using k_BT as "the size of the thermal kicks" for two weeks. Time to say where it comes from and exactly what it promises. Statistical mechanics delivers a result of astonishing generality called equipartition: in thermal equilibrium at temperature T, every independent quadratic way a system can hold energy — each velocity component of each particle, each mode of vibration, the voltage on each capacitance — carries, on average,

Read that sentence again with an engineer's suspicion, because it says something outrageous: the environment funds every degree of freedom in your device, all the time, without being asked. You cannot decline the funding. Every storage node, every wire, every mechanical element in a system at 300 K is continuously credited about half of 26 meV of random energy, refreshed on the timescale of the system's own dynamics — picoseconds for electrons in silicon. Temperature is not an occasional disturbance to be filtered out; it is a standing bankroll for the adversary, deposited into the very degrees of freedom you are using to store information.

The average is only half the story. Fluctuations around it have no upper cutoff: occasionally, by pure chance, one degree of freedom briefly concentrates many times k_BT. Rarely — but a gigabit of memory examined a billion times a second buys an enormous number of lottery tickets, and this course's reliability arithmetic is really the arithmetic of rare large fluctuations. How rare is a fluctuation of size E? That is the next section, and the answer is the most consequential exponential in engineering.

2Boltzmann statistics and the barrier

Here is the honest two-line argument, and it is worth having seen once rather than taking the exponential on faith. Put a small system in contact with a large thermal reservoir at temperature T and ask: how probable is it to find the small system holding energy E? Every joule the system holds is a joule the reservoir gave up, and a reservoir that gives up energy E loses entropy E/T — that is what temperature means. Fewer ways for the reservoir to arrange itself means proportionally fewer ways for the joint arrangement to occur, and Boltzmann's counting turns that entropy deficit into a probability ratio:

Each additional k_BT of energy you ask the reservoir to concentrate in your degree of freedom makes the request a factor of e less likely to be granted. That is all the exponential is: the reservoir's bookkeeping. It is not a property of silicon, or of electronics, or of any technology — which is why the same factor will govern every device in this course, and every device in your capstone, whatever it is made of.

Now aim it at the double well of Notes 02. A bit rattles in its valley, testing the barrier f₀ times per second; each test succeeds with the Boltzmann probability of holding E_b at the moment of the attempt. The escape rate and its reciprocal, the mean time the bit survives, follow at once:

This attempt-and-succeed picture is called the Arrhenius law, and its careful derivation for a particle diffusing over a smooth barrier is due to Kramers — the refinements move f₀ around by small factors and never touch the exponential. That division of labor is worth internalizing as a modeling habit: prefactors are technology, exponents are physics. When you build the Project 1 model you will estimate f₀ crudely and it will not matter; get the exponent wrong and nothing else you do will save the answer.

3The price of a reliable bit

We can now derive the course's first genuine design equation. The specification a storage designer is actually handed reads like this: N bits must survive for a hold time t, and the probability that even one of them flips must stay below p_err. Each bit escapes at the rate in (2.2), so over time t a single bit flips with probability 1 − e^−rate·t, which for any rate worth shipping is excellently approximated by rate × t. With N independent bits the expected number of failures multiplies by N:

Solve for the barrier and the exponential becomes a logarithm:

Equation (3.2) is Project 1 in one line, and it repays staring at. Everything the customer can demand — more bits, longer retention, stricter error budget — enters through a logarithm, so heroic-sounding requirements cost almost nothing: a million times more bits is 13.8 k_BT of extra barrier, about a third of a volt's worth of electron energy. Meanwhile the one thing the customer cannot demand away, temperature, multiplies the whole right-hand side linearly. Reliability is cheap against scale and brutal against heat — one equation, both facts.

Equation (3.2) drawn as a curve: required barrier against hold time, for your chosen fleet size, error budget, and temperature. f₀ = 10¹³ Hz. The line is straight because the time axis is logarithmic — ten times the hold time is always the same 2.3 k_BT surcharge, whether that is nanoseconds to tens of nanoseconds or years to decades.

One more reading of (3.2) before we move on. It is a statement about the minimum barrier, and minimums in this course are two-sided: Notes 02 section 4 showed that every k_BT of barrier must be re-crossed at write time. A designer therefore wants to sit near the bound, not comfortably above it — and knowing exactly where the bound sits, for a given device model rather than for the idealized double well, is precisely what Project 1's simulation is for.

4Landauer: the entropy cost of forgetting

Barriers protect bits; now for the strangest and most fundamental result in this unit, which concerns destroying them. Consider the operation RESET: whatever the bit was, make it 0. Before, the system could be in either of two states; after, only one. The information-bearing degrees of freedom have had their number of possible arrangements cut in half, and Boltzmann's definition of entropy makes that a real, physical entropy decrease of k_B ln 2. The second law does not permit entropy to simply vanish: it must be exported to the environment, and exporting entropy ΔS to a reservoir at temperature T means delivering heat TΔS. Erasing one bit therefore dissipates, at minimum,

This is Landauer's principle (1961), and notice what makes it different in kind from everything else in these notes: it contains no f₀, no barrier, no capacitance, no material property at all. It is not a limitation of a technology; it is a property of the operation. Any physical system whatsoever that erases a bit at temperature T pays it — your laptop, a neuron, a hypothetical computer built from carefully labeled billiard balls. And it is not folklore: in 2012 Bérut and colleagues measured it directly, erasing a bit held by a micron-sized colloidal particle in a double-well optical trap and watching the dissipated heat approach k_BT ln 2 as the erasure slowed. Their apparatus is, almost literally, the model you are building in Project 1.

Be precise about what the principle does and does not bound, because loose versions of it circulate widely. It bounds erasure — the logically irreversible collapse of two states into one. Ordinary logic erases constantly: a NAND gate maps four input states onto two output states, and the discarded distinction is entropy that must go somewhere. But a computation arranged so that no information is ever discarded — every step invertible, inputs recoverable from outputs — evades the bound in principle, which is why reversible computing is a real body of theory and not a perpetual-motion scheme. What reversible schemes pay instead is speed, area, and exquisite sensitivity to noise, which is a trade you now have the vocabulary to evaluate; several capstone topics live exactly there.

For this course, Landauer's limit plays the role of sea level: the altitude from which every real energy figure is measured. The interesting engineering question is never "can we reach the floor" — it is "we are five orders of magnitude above the floor; itemize the bill." The next two sections do exactly that.

5The energy of switching

Holding a bit is one cost. Changing one is another, and it is the cost that shows up on your electricity bill. In CMOS the dominant term is beautifully simple, and we can get it with nothing but bookkeeping. To make a node a 1, the supply pushes charge Q = CV_DD onto its capacitance, doing work Q·V_DD = CV_DD² — the supply rail sits at fixed voltage, so every coulomb it sources costs the same. The capacitor, though, ends up storing only ½CV_DD² (the first charge arrived to an empty node, the last against full opposition — the average is half). The missing half was dissipated in whatever resistance the charge flowed through. Discharge the node to make a 0 and the stored half is dissipated too. A full 0→1→0 cycle therefore turns exactly CV_DD² into heat, and the conventional per-transition figure is

Two things about this expression deserve to be underlined. First, the resistance does not appear. Notice that the bookkeeping argument never asked what R was — the half is lost regardless (exercise 2 makes you confirm it with the explicit integral, and exercise 3 shows the one honest loophole). How fast you charge the node changes the delay, but not the energy; that is why the speed story and the energy story can be told separately for several weeks before they collide in week 11. Second, the voltage is squared. Halving V_DD cuts switching energy by a factor of four. This is the single most powerful energy lever in digital design, and the reason supply voltages fell from 5 V to under 1 V over three decades. It is also why they stopped falling — and we can now say precisely why, instead of waving at "noise."

Why the lever jammed: E_switch meets Δ/σ

Take the charge-held bit and bring together the two threads of Notes 02. The separation between its read levels is the full swing, Δ = V_DD. The fluctuation is the thermal noise on the node itself, which the next section shows is σ = √(k_BT/C). Form the distinguishability ratio and something remarkable happens:

The square of the signal-to-noise ratio is the switching energy, measured in units of k_BT. The two costs of Notes 02 — distinguishability and stability — and this section's energy cost are one number wearing three hats. Demand Δ = 12σ so that misreads are astronomically rare, and (5.2) says a voltage-sensed bit costs at least 72 k_BT per switch — a floor a hundred times above Landauer that you hit before speed, margins, or manufacturability have said a word. Lower V_DD with C fixed and Δ/σ falls linearly while errors rise Gaussian-fast; lower C to save energy and the same thing happens. That is the jam: below roughly 0.7 V, equation (5.2) and the barrier arithmetic of section 3 begin to bite simultaneously, and the industry's forty-year free ride on the V² lever ended.

Chip power assumes 10⁹ gates, activity factor α = 0.1, f = 1 GHz, and counts dynamic switching only. Real processors add leakage, clock distribution, and interconnect on top of this. Slide V_DD down and watch the point race toward the floor — then remember what (5.2) says is happening to the error rate as it goes.

Now the comparison that motivates half of modern research in this field. A 1 fF node switched at 0.8 V costs 320 aJ, which is about 77,000 k_BT, or roughly 110,000 times the Landauer limit. We are five orders of magnitude above the floor. That gap is not stupidity; it is the itemized bill for margins, restoration, fan-out, speed, and manufacturability at scale — section 7 breaks it down line by line. But it is also the room in which every post-CMOS proposal claims to operate, and evaluating those claims honestly is what your capstone is for.

6Noise, first contact: kT/C and Johnson

Twice now we have used σ = √(k_BT/C) on credit. Here is the payment, and it is one of the cleanest facts in all of electronics. A capacitance C connected to anything resistive is one quadratic degree of freedom — energy ½Cv² — so equipartition (1.1) applies to it directly: ½C⟨v_n²⟩ = ½k_BT, and therefore

No resistance in the formula, no bandwidth, no device parameters — C and T, nothing else. For a 1 fF node at 300 K that is about 2 mV rms. Against a 1 V supply, comfortable: the noise floor sits roughly 500 times below full swing. Shrink the node to 1 aF and the same formula gives 64 mV, and suddenly thermal noise is a serious fraction of your noise margin. This is a hard floor. You cannot filter it away — it lives on the very node you are sensing — and you cannot design around it; you can only spend capacitance (which by (5.1) costs energy and, later, speed) or lower the temperature (which costs a refrigerator). Finding where that trade lands for a specified error rate is, again, Project 1.

Where the fluctuation comes from: Johnson noise

Equipartition tells you the size of the fluctuation but not its source. The source is the resistor. Every resistance at temperature T is a thicket of thermally agitated charge carriers, and their agitation appears at the terminals as a fluctuating voltage — measured by Johnson and explained by Nyquist in 1928 — with a spectral density that is flat across every frequency we will care about:

A 50 Ω resistor observed over a 1 GHz bandwidth fluctuates by about 29 µV rms — small, but sensing circuits routinely fight for less. And here is the satisfying closure: put that noisy resistor in front of a capacitor and the RC filter passes only a noise bandwidth of 1/(4RC). Multiply: 4k_BTR × 1/(4RC) = k_BT/C. The R cancels — a big resistor is noisier but filters more aggressively, and equipartition's verdict (6.1) is recovered exactly. The two views agree because they must; one is thermodynamics, the other is the mechanism.

This is deliberately only first contact. Noise in real digital systems is a budget with many line items — coupling between wires, supply and ground bounce, shot noise, flicker noise, and the Johnson floor under all of it — and week 12 is devoted to that full accounting, once we have real circuits to account for. What you should carry forward now is the shape of the thing: white, Gaussian, inescapable, and scaled by √kT — exactly the σ that has been standing under every distinguishability argument since Notes 02.

7Numbers that matter

This unit generates the reference points that the rest of the course — and your project reports — will be measured against. Here they are in one table, worth genuinely knowing by heart to the nearest order of magnitude.

Read the k_BT column downward and the story of the course appears in the ratios. From the Landauer floor to the physics-mandated minimums is a factor of ten to a hundred: the honest cost of reliability, derived in sections 3 and 5, not removable by any cleverness. From those minimums to a real CMOS switch is another factor of a thousand — and that gap is engineering, itemizable line by line: real nodes carry the capacitance of the wires and the fan-out they drive, not just one gate (a factor of ten); real devices are driven hard because we want answers in picoseconds, not adiabatically (another healthy factor); and real chips must work across a billion devices, −40 to 125 °C, and ten years of aging, so every margin is set for the worst device on the worst day (the rest). Nothing in the column is mysterious. That is the point of the course so far: the bill is large, but every line on it can be derived.

And the last two rows — where the multiplier leaps from 10⁵ to 10⁹ and beyond — are not about switching at all. They are wires and distance, the subject of weeks 8 and 9, and the reason the phrase "data movement dominates" appears in every modern accelerator paper. Keep the table; we will keep adding rows to it all term.

8Modeling it in Python

Everything above is closed-form, which means everything above is a claim your Project 1 model can check. Two snippets set the pattern for how this course treats theory: first compute what the equations predict, then build a simulation that never saw the equations and see whether it agrees.

The design equation as a picture

Equation (3.1) in eleven lines of NumPy. Run it and you get the failure probability against barrier height for hold times spanning seventeen orders of magnitude — the logic-node and flash-cell worked examples of Notes 02 are two points on these curves.

import numpy as np
import matplotlib.pyplot as plt

f0 = 1e13                                   # attempt frequency, Hz
Eb = np.linspace(0, 60, 601)                # barrier height, units of kT
hold = [1e-9, 1e-3, 1.0, 3.16e8]            # 1 ns, 1 ms, 1 s, 10 years

for t in hold:
    p = 1 - np.exp(-f0 * t * np.exp(-Eb))   # P(bit lost within t)  -- eq (3.1)
    plt.semilogy(Eb, p, label=f"t = {t:.0e} s")

plt.xlabel("E_b / kT"); plt.ylabel("P(bit lost)")
plt.ylim(1e-18, 1); plt.legend(); plt.grid(True, which="both", alpha=0.3)
plt.show()

Each curve is a cliff: harmless to the left, certain failure to the right, with the transition only a few k_BT wide. Digital reliability is not a slope you descend gracefully; it is an edge you stay away from. Note where the four cliffs sit — 9, 23, 30, and 50 k_BT — and check them against equation (3.2).

A bit that never saw the formula

Now the other half of the method. This simulates the physical system directly: an overdamped particle in the double well V(x) = E_b(x²−1)², kicked every timestep by Gaussian thermal noise of the equipartition size, watched until it escapes. No Boltzmann factor appears anywhere in the code — the exponential must emerge.

import numpy as np
rng = np.random.default_rng(255)

def mean_escape_time(Eb_kT, n=400, dt=5e-4, tmax=2e4):
    """Mean first-passage time out of the left well, in units of the
    system's own relaxation time. Eb_kT is barrier height over kT."""
    x     = -np.ones(n)                     # n copies, all starting at state 0
    t_esc = np.full(n, np.nan)
    alive = np.ones(n, dtype=bool)
    for k in range(int(tmax / dt)):
        F = -4.0 * Eb_kT * x * (x**2 - 1)   # -dV/dx
        x += F * dt + np.sqrt(2 * dt) * rng.standard_normal(n)
        just = alive & (x > 1.0)            # crossed into the right well
        t_esc[just] = k * dt
        alive &= ~just
        if not alive.any():
            break
    return np.nanmean(t_esc)

for r in (2, 3, 4, 5, 6):
    print(f"Eb = {r} kT : mean escape {mean_escape_time(r):9.2f}")

Escape times grow roughly e-fold per added k_BT — Arrhenius, emerging from nothing but F = −dV/dx and random kicks. Measure the ratios and you will find them a little under e at these small barriers: the Kramers prefactor shifts with barrier height too. The exponent is the physics; the prefactor is detail. And the last row's wall-clock time is the exponential, felt directly.

Notice also what the simulation cannot do, because the limitation is instructive. At E_b = 40 k_BT the mean escape time is ~10¹⁷ attempt periods; no Monte Carlo will sit through that. The working method — in Project 1 and in the industry — is to validate the law where simulation can reach it, then let the law carry you the remaining orders of magnitude. Your model earns the right to extrapolate by agreeing with physics where they can both be checked.

9Work before Session 5

Session 5 is Monday, September 21, and it changes register: semiconductor physics, on the way to the transistor. Before then:

• Review these notes. Sections 3 and 5 are the analytical spine of Project 1 and will be assumed cold from here on.
• Project 1 work begins now. This week: repository created from the spec's template, both section 8 snippets running, and a first-passage-time plot of your own committed. Aim to have the core model working by the end of week 4 — week 5 is for analysis and writing, not debugging.
• Reproduce worked examples 3.1 and 5.1 with your own code rather than a calculator. If your numbers disagree with mine, one of us is wrong and finding out which is the whole game.
• Work the exercises below; exercise 3 opens the door to adiabatic charging, a standing capstone direction in this course.
• If Project 1's scope is unclear after reading the spec twice, say so before Wednesday — not in week 5.

10Exercises

Not graded. Bring answers or stuck points to either session this week.

1. A DRAM cell holds its bit as charge on a 25 fF capacitor charged to 1.0 V. How many electrons is that? What is the rms thermal noise voltage on that node by equation (6.1), and what fraction of full swing is it? What Δ/σ does this cell enjoy, and what misread probability does that imply?
2. Show by explicit integration that charging a capacitor C to voltage V through a resistor R dissipates ½CV² in the resistor regardless of the value of R — confirming the bookkeeping argument of section 5. Then explain in one sentence why the energy is independent of R but the delay is not.
3. Charge the same capacitor in two steps — first from a supply at V/2, then from a supply at V. Show the total dissipation is now ¼CV², and that N equal steps dissipate CV²/2N. What is this strategy's cost, what circuit resource does it demand, and what is it called in the literature? (Look up "adiabatic charging" after you have your answer, not before.)
4. A processor dissipates 95 W and performs 4 × 10¹¹ floating-point operations per second. Estimate the energy per operation, and express it as a multiple of the Landauer limit at 300 K. Using section 7's table, itemize where you believe the factor goes, to the nearest order of magnitude per line.
5. Compute the rms Johnson noise of a 50 Ω resistor at 300 K over a 1 GHz bandwidth, and of a 1 MΩ resistor over the same bandwidth. Compare both against a 100 mV noise margin and comment: in which regimes of impedance and bandwidth does Johnson noise begin to threaten digital signaling directly?
6. Using equation (3.2): a satellite memory must hold 10¹² bits for 15 years with total failure probability 10⁻⁹, but the electronics run at 358 K. Find the required barrier in eV. Then find how much barrier the same specification needs at 300 K, and express the penalty of the 58 K as a percentage. Which term of the equation did the temperature attack?

§Symbols and constants