Sessions 5–6 · Notes 04
Where Your Bytes Live
Object, block, and file storage — and the two promises, durability and availability, that make a rented disk more trustworthy than one you can touch.
01The Question of the Week
Two weeks of this course have been about muscle. You rented machines, sized them, cloned them from images, and let the autoscaler hire and fire them by the minute. And Notes 03 ended on a rule that should have left a small knot in your stomach: instances are cattle. You are supposed to delete them casually — terminate, replace, never look back.
Here is the knot. When an instance dies, its disk usually dies with it. The scratch disk that comes with the machine is called ephemeral — a polite word for “gone when the instance is gone.” And remember what the autoscaler you built last week does when traffic drops: it scales in by terminating instances. Every quiet afternoon, your own automation is out back burning machines. Anything that lives only on one of those machines burns with it.
That is fine for the machine itself. Notes 03 taught you to rebuild a machine from an image in under a minute — muscle can always be rebuilt from a recipe. But it is not fine for the photo a user uploaded ten minutes ago. There is no recipe for that photo. Compute is rented muscle; what we need now is rented memory — and memory has one job the muscle never had: outlive everything.
So the cloud makes storage its own service. Not a part inside a machine, but a thing machines connect to over the network — with its own lifetime, its own price (billed per GB per month, the way compute is billed per second), and its own promises. Instances will come and go all week; the storage stays put. This week is about the three shapes that service comes in, and the promises it makes about your data once it has it.
An instance should hold nothing you would miss. If terminating any machine in your system would make you flinch, some bytes are living in the wrong place. By the end of these notes you will know the three right places.
02Three Shapes of Storage
Open any provider’s storage page and you will find a bewildering pile of products. Ignore the names for a minute. Underneath, every one of them is one of exactly three shapes, and you already know all three from ordinary life: a bookshelf, a filing cabinet, and a coat check.
2.1 Block storage — the bookshelf bolted to one machine
Block storage is an empty bookshelf bolted to the side of one machine. The shelf arrives with nothing but numbered slots — no labels, no folders, no idea what a “file” even is. Its entire contract is: read slot 4,183; write slot 4,184. All the organization — the filesystem, the folders, the database’s pages — is brought by the machine it is bolted to.
That dumbness is the point. Because the shelf promises so little, it can be very fast and sit very close — which is why block storage is the disk under your instance’s operating system and the disk under every serious database. The catch is in the word bolted: a block volume attaches to one instance at a time. It is a personal bookshelf, not a library.
And here is the part that unties Section 01’s knot: the shelf is a separate piece of furniture. Terminate the instance and you can keep the volume, then bolt it onto the replacement — the data never noticed the machine change. (Watch one default, though: boot volumes are often set to be deleted along with the instance. The furniture survives only if you say so.) The names: EBS on AWS, Persistent Disk on Google Cloud, Managed Disks on Azure.
2.2 File storage — the shared filing cabinet
File storage is the office filing cabinet in the hallway. Folders inside folders, papers with names — and, the important part, many machines can open the same drawer at the same time and all see the same papers. When one machine files a page, everyone else finds it there a moment later.
This is the shape your laptop has trained you to expect, which is exactly why the cloud sells it: for software written to expect a shared folder. A render farm where twenty workers read the same textures. Home directories. A fleet of web servers that all need to see the same uploads folder. The names: EFS, Filestore, Azure Files.
The honest trade-off: shared drawers are convenient and comparatively expensive, and “everyone can edit the same paper at once” is a coordination headache the other two shapes never have. Reach for file storage when many machines genuinely need the same folders by path — not by default.
2.3 Object storage — the coat check
Object storage is a coat check. You hand over a coat; you get a claim ticket; from then on the ticket is the only way to get the coat back. Behind the counter there may be a thousand coats or a hundred billion — you cannot browse the racks, and you do not care how they are arranged. Ticket in, coat out. That is the entire interface.
The ticket is called a key — a name you choose, like photos/2026/cat.jpg. The coat is the object: any file at all, from a one-kilobyte config to a multi-terabyte video. The coat room itself is a bucket. That is the whole vocabulary.
Two desktop habits to unlearn at the coat check. First, there are no folders. The slashes in photos/2026/cat.jpg are just characters in the ticket; the console draws little folders around them to be comforting, but nothing is “inside” anything. Second, there is no editing in place. You cannot ask the attendant to hem a sleeve; you hand in a whole new coat under the same ticket. Objects are replaced, never patched.
In exchange, the coat room is effectively bottomless, costs pennies per gigabyte, hands any coat to anyone on the internet you permit (every object can have a URL), and — as the next section shows — is very nearly impossible to destroy. This is where the internet keeps its stuff: backups, logs, photos, video, ML training sets, and the static files of your Project 1 service. The names: S3, Cloud Storage, Azure Blob Storage.
| Shape | The analogy | AWS | Google Cloud | Azure |
|---|---|---|---|---|
| Block | Bookshelf bolted to one machine | EBS | Persistent Disk | Managed Disks |
| File | Shared filing cabinet | EFS | Filestore | Azure Files |
| Object | Coat check | S3 | Cloud Storage | Blob Storage |
Self-checkYour app stores a 2 GB video and later needs to change ten seconds in the middle of it. Which shape are you fighting?
Object storage. There is no editing in place at the coat check — changing ten seconds means uploading a whole new 2 GB coat. A block volume lets the machine rewrite just the slots that changed. (Whether you should edit videos in place is a different question — but know which shape makes it cheap.)
03Two Promises: Durability and Availability
Storage services make you two separate promises, and mixing them up will muddle every reliability conversation you ever have. Durability asks: does my data still exist? Availability asks: can I reach it right now? One is an insurance policy; the other is the open-hours sign on the door.
The two can split. A bank vault during a blackout is perfectly durable and completely unavailable — your valuables are fine; come back tomorrow. A service that answers instantly but has quietly lost your file is available and not durable, and that is the unforgivable one. Downtime is an apology. Data loss is a funeral.
Now, how is durability manufactured? Because no single disk deserves the promise. At cloud scale, hardware failure is not a risk, it is weather: with millions of disks in a fleet, some die every single day, guaranteed. The trick is never to let the only copy of anything sit on the thing that breaks. So the recipe has one ingredient: copies. When you hand an object store your file, it quietly writes several copies before it even says “done” — on different machines, in different buildings (the availability zones from Notes 01), sometimes in different cities. When a disk dies, the store notices, and re-copies from the survivors onto fresh hardware. Your file outlives any particular disk the way a story outlives any particular teller.
Providers state these promises in nines. Object stores advertise eleven nines of durability — 99.999999999%. Translated honestly: store ten million files for ten thousand years and expect to lose one. Availability promises are far humbler — typically 99.9% to 99.99% — because things go unreachable (a power cut, a mistyped router config) vastly more often than they get destroyed. Closed-for-an-hour happens. Burned-down almost never does.
| The promise | The number | In everyday terms |
|---|---|---|
| Availability, “three nines” | 99.9% | Closed about 9 hours a year |
| Availability, “four nines” | 99.99% | Closed about 52 minutes a year |
| Availability, “five nines” | 99.999% | Closed about 5 minutes a year |
| Durability, “eleven nines” | 99.999999999% | Ten million files, ten thousand years — expect to lose one |
Numbers are one thing; feel it in your hands. The toy below is a file uploaded to a tiny cloud: three buildings, five machines. Choose how many copies exist and how spread out they are, then make trouble. Watch the two promises come apart — a dark building and a dead disk are very different kinds of bad day.
- Copies that exist
- 3 of 3durability
- Copies you can reach
- 3availability
- Your file is
- SAFEand reachable
Three copies, three buildings — the default posture of a real object store. Try killing a disk, then cutting the power, and watch which number moves.
Play until both halves are second nature. The only way to lose the file is to lose every copy. The only way to be locked out while the file is still safe is for the surviving copies to sit somewhere you cannot reach. And notice that the spread matters as much as the count: three copies on one machine looks like redundancy and is actually one disk death away from oblivion.
Self-checkAn entire region’s network goes down for an hour. Your bucket lives there. Which promise got broken?
Availability only. The copies still exist, patiently, in the dark — durability was never in danger. The nines that got spent were the 99.99, not the eleven. This is also why the durability number can be so much larger than the availability number: existing is easier than answering.
04When Copies Disagree
Copies gave us durability. Now the bill arrives. You post a photo to the group chat. It is on your screen. Your friend across town glances at her phone — nothing yet. For a few seconds, the two of you are looking at different versions of reality. Then her phone catches up. That gap is a consistency problem, and it is not a bug in her phone. It is the unavoidable price of the same information living in more than one place.
Your file in the last section was three copies in three buildings. A write lands on one copy first and takes real time — usually milliseconds, occasionally longer — to travel to the rest. If a read hits a copy the write has not reached yet, it gets the old version back. That is a stale read: not wrong forever, just behind. The disagreement window exists because the copies exist — the very thing manufacturing your durability is what allows two readers to briefly see two truths.
Storage systems handle the window in one of two ways, and both have plain names. Read-after-write consistency (you will also hear “strong”) promises: once your write is confirmed, every read, from every copy, sees it — the system quietly makes readers wait, or routes them, until reality agrees. Eventual consistency promises less: reads may briefly return the old version, but stop writing and all copies converge on the same answer. Eventual sounds sloppy, but it buys speed and resilience, and for plenty of data — a like-count, a view counter — briefly-behind is completely harmless.
Where do object stores stand today? Better than the folklore says. For years, S3 was eventually consistent when you overwrote or deleted an object, and a generation of engineers carries scar tissue from photos that “un-updated” themselves. Since late 2020 that is history: S3, Cloud Storage, and Azure Blob all give read-after-write consistency now — write it, and the very next read sees it. So why teach the group-chat problem at all? Because the moment you put anything in front of the bucket — a CDN, a browser cache, a copy in another region — you have rebuilt the friend’s unrefreshed phone. Those layers are eventual, and no provider upgrade will change that.
Three machines each hold a copy of sunset.jpg. Post a new version, then hit the read buttons while the amber bars are still moving — that is the disagreement window, stretched from milliseconds to seconds so you can stand inside it.
One practical rule falls out of all this, and it is the only consistency rule Project 1 needs:
Never silently change a file behind a URL someone may have cached. Upload the changed file under a new name — style.css becomes style.v2.css — and point your pages at the new name. A cache cannot serve you a stale copy of a file it has never heard of; the old one dies of old age on its own.
Self-checkA user uploads a new profile picture but keeps seeing the old one for ten minutes. Your object store is read-after-write consistent. Who is the likely culprit?
A copy you forgot you had: the CDN or the browser cache, still holding the old image under the same URL. The bucket told the truth immediately; the cache in front of it is eventual. The fix is the rule above — give the new picture a new key, and the stale copy can never be asked for again.
05Picking Storage for a Job
You now hold all three shapes and both promises. Time to make some calls. Three everyday jobs, decided out loud:
5.1 User-uploaded photos
Written once, read many times, never edited in place (a “crop” is really a new photo), served straight to browsers by URL, and the pile grows forever. Every clue points the same way: the coat check. Object storage — cheap per coat, bottomless, URL for every object, eleven nines watching over irreplaceable memories. This is precisely why Project 1 puts your static assets there.
5.2 The database’s disk
One machine — the database instance — owns it exclusively. It makes thousands of tiny in-place edits per second and cares about every microsecond. That is the bolted bookshelf: block storage. Running a database on object storage would mean handing in a whole new coat a thousand times a second; running it on a shared cabinet adds coordination cost it does not want. Fast, close, private: block.
5.3 A shared drive for a render farm
Twenty worker machines all need /textures and /scenes by path, at the same time, through software that expects ordinary folders. That is the filing cabinet: file storage. Object storage would force you to rewrite the renderer’s file handling; block storage cannot be opened by twenty machines at once. When the software demands a shared folder, give it one.
And when a new job shows up that is not on this list, ask the same three questions in order:
- Who opens it — one machine or many? One machine, intimately: block. Many machines, as folders: file. Anyone, by name or URL: object.
- Is it edited in place, or replaced whole? Constant small edits want block. Whole-thing replacement is object storage’s native gait.
- Does it grow without bound? Bookshelves and cabinets have sizes. The coat check does not. Anything that accumulates forever wants a ticket, not a slot.
06Before Session 7 — September 28
Project 1 is due at the end of Week 5 — that is next week. This week’s work moves your project onto the right storage before the deadline pressure starts, which is much easier than moving it after. The list remembers your progress on this device.
The URL of one object served from your Project 1 bucket, and your README’s “if this instance is terminated, we lose ___” sentence. Next week the bytes start moving: networks, load balancers, DNS, and the CDNs that put your coat check on every street corner.
07Key Terms
- Block storage
- Raw numbered slots attached to one machine at a time; the machine brings all the organization. The disk under the OS and the database. EBS / Persistent Disk / Managed Disks.
- File storage
- Shared folders many machines open at once, by path, like a network drive. EFS / Filestore / Azure Files.
- Object storage
- The coat check: whole files stored and retrieved by key, never edited in place, effectively bottomless. S3 / Cloud Storage / Blob Storage.
- Bucket
- One named coat room in an object store; the container all your objects and their keys live in.
- Key
- The claim ticket: the full name of an object, like
photos/2026/cat.jpg. Slashes are just characters — there are no real folders. - Ephemeral disk
- The scratch disk that lives and dies with an instance. Fine for temp files; fatal for anything you would miss.
- Durability
- The promise that your data still exists — the insurance policy. Manufactured out of copies.
- Availability
- The promise that you can reach your data right now — the open-hours sign. Always more nines short than durability.
- The nines
- How promises are quoted: 99.99% availability is ~52 minutes closed per year; eleven-nines durability is ten million files for ten thousand years, expect to lose one.
- Replication
- Keeping copies of the same data on different machines, buildings, or regions, and re-copying when one dies. The engine of durability.
- Read-after-write consistency
- Once a write is confirmed, every read everywhere sees it. What the big object stores promise today.
- Eventual consistency
- Reads may briefly return an old version, but all copies converge. Still the rule for caches, CDNs, and cross-region copies.
- Stale read
- A read answered by a copy the latest write has not reached yet. Not wrong forever — just behind.