The delivery-guarantee myths: exactly-once & the two generals

Read the marketing for any message broker and you'll see "exactly-once delivery." It is the most sold and least understood guarantee in distributed systems, and the literal claim is impossible — not "hard," not "expensive," but provably unachievable over an unreliable network. Yet the systems selling it aren't lying so much as being loose with words, because there is a real, achievable guarantee hiding under the phrase. This lesson is the capstone of the distributed track because pulling those two things apart — the impossible version and the achievable version — retroactively explains every reliability pattern you've met: idempotency keys, webhook dedup, the outbox, message dedup. They are all one answer to one problem.

The two generals problem

Here's the thought experiment that proves the impossibility. Two generals on opposite hills must attack a city at the same time to win; attacking alone loses. They can only communicate by sending messengers through the valley — where messengers are sometimes captured (messages are lost). General A sends "attack at dawn." But A can't attack unless sure B received it — the messenger might have been captured. So B sends an acknowledgment. But now B can't be sure A received the ack — that messenger might have been captured — so A doesn't know B knows, so A won't commit. So B acks the ack. But that might be lost too…

The chain never terminates. No finite number of acknowledgments can make both generals certain they agree, because the last message sent — whichever it is — might be the one that's lost, and its sender can't know whether it arrived. This is the two generals problem, and it's a proof: guaranteed agreement over an unreliable channel with a bounded number of messages is impossible. It is the theoretical bedrock under everything that follows.

Why exactly-once delivery is impossible

Apply the generals to a message and a consumer. A broker sends a message and needs to know the consumer processed it. The consumer processes it and sends an ack. That ack can be lost (two generals). So the broker, having received no ack, faces the same undecidable question the general did: did the consumer get it, or not? It has exactly two options, and neither is "exactly once":

• Resend it (assume it was lost). If the consumer actually did get it and only the ack was lost, the consumer now processes it twice → at-least-once delivery, possible duplicates.
• Don't resend it (assume it arrived). If it actually was lost, the consumer never processes it → at-most-once delivery, possible loss.

There is no third choice, because the broker can never know which case it's in — that's the two generals result. So over an unreliable network you get at-least-once (never lose, might duplicate) or at-most-once (never duplicate, might lose), and "exactly-once delivery" — never lose and never duplicate — is not on the menu. Anyone claiming it either means something narrower or is wrong.

Run the same three payments under each semantics. At-least-once never loses m2 but charges it twice; at-most-once never duplicates but silently loses it; only effectively-once — at-least-once plus an idempotent dedup — lands on the right ₹1500.

The achievable thing: exactly-once effects

Here's the escape, and it's the sentence the whole curriculum has been building toward. You can't control how many times a message is delivered, but you can control how many times it takes effect. Accept at-least-once delivery (never lose anything, tolerate duplicates) and make the processing idempotent — so that handling the same message twice produces the same result as handling it once. Then:

at-least-once delivery + idempotent processing = exactly-once effects

The message might arrive five times; the money moves once, the row is inserted once, the notification fires once — because the consumer recognizes the duplicates and neutralizes them (dedup on a message id, a WHERE NOT EXISTS, a unique constraint, a compare-and-swap). This is exactly what everyone means when they say they want exactly-once, and it's entirely achievable. The impossible part (delivery) is sidestepped by making the count of deliveries irrelevant to the count of effects.

This is even what the vendors' "exactly-once" is, once you read the fine print. Kafka's exactly-once semantics, for instance, are an idempotent producer (dedup on the broker) plus transactional writes within Kafka — genuinely exactly-once for the effects inside Kafka's own boundary. Which sets up the trap.

The whole curriculum, in one idea

Step back and notice that a single principle has appeared, unlabeled, in nearly every lesson — and it's all this one:

• Idempotency keys on money writes — at-least-once HTTP retries + idempotent effect.
• Webhook receivers deduping on event id — at-least-once webhook delivery + idempotent effect.
• The outbox relay publishing at-least-once to idempotent consumers.
• Chat's client_dedup_key and the settlement idempotency constraint — at-least-once sends + idempotent effect.
• Even TCP itself: retransmit lost segments (at-least-once) + sequence numbers to discard duplicates (idempotent reassembly) = the reliable, exactly-once byte stream you take for granted.

They are not five patterns. They are one pattern — don't try to make delivery perfect; make effects idempotent — applied at five layers. That's the takeaway of the whole distributed track: perfect delivery is a fantasy the two generals forbid, so you stop chasing it and build systems where duplicate delivery is harmless.

Go deeper

• The Two Generals' Problem — The clean statement and proof of the impossibility result that underlies everything here — worth reading slowly until the "the last message might be lost" argument is obvious.
• Confluent — "Exactly-once semantics ... here's how Kafka does it" — Read it precisely to see what the guarantee actually covers (idempotent producer + transactions inside Kafka) — and therefore where it stops, which is the callout's whole point.
• Tyler Treat — "You Cannot Have Exactly-Once Delivery" — A blunt, clear treatment of why delivery can't be exactly-once and why idempotent processing is the real answer — the thesis of this lesson, argued hard.

Check yourself

Answer out loud, as if an interviewer asked. If you hand-wave, reread that section.

1. State the two generals problem and explain precisely why no number of acknowledgments makes both parties certain they agree. What real-world guarantee does this rule out?
2. From the broker's perspective, an unacked message forces a choice with only two options. Name them, the risk of each, and why "exactly-once delivery" is not among them.
3. Write the equation that produces exactly-once effects, define each term, and explain why controlling the count of effects sidesteps the impossibility of controlling the count of deliveries.
4. Kafka advertises exactly-once. Explain what it actually guarantees and, precisely, where that guarantee stops — including how a consumer writing to an external database gets a duplicate anyway.
5. The lesson claims idempotency keys, webhook dedup, the outbox, chat dedup, and TCP are "one pattern." State the pattern in one sentence and explain how TCP is an instance of it.
6. Fable never relies on exactly-once delivery. Explain how it achieves correct money and chat over lossy networks instead, and why its effects living outside any transport boundary makes idempotency the only defense that works.