Infrastructure evolution: when ₹0/month started costing 70 milliseconds

Every frontend developer has felt this bug from the outside: you tap something, and the screen just… hangs. Not broken — the spinner is honest, the data eventually arrives — but slow in a way that makes the whole app feel cheap. On the web you'd open the Network tab, see a request sitting at 800ms, and start asking why. Fable had exactly that problem on every screen, and the answer wasn't in the code. It was on a map. The server was in Mumbai; the database was in Singapore; and the request waterfall you learned to read in the browser was quietly paying for that distance several times over, per request.

This chapter is about how Fable's infrastructure started, why it started wrong on purpose, and what it cost to fix. It's the most-linked page in the networking track for one reason: it's where the latency table stops being a table and starts being a bill.

The problem, stated precisely

Fable's target user is in India. The whole product assumes it — +91 is hardcoded in v1, the default currency is INR, the launch market is one country. So the API runs on a small VM in Mumbai (GCP's asia-south1). That part was always right.

What wasn't right was where the data lived. The planned stack, straight out of the original architecture spec (TDD-001), was aggressively free: Neon for managed Postgres, Upstash for Redis, Cloudflare R2 for media, all on free tiers, with a stated cost target of ~₹0/month for the first few hundred users. Neon and Upstash's free tiers, at the time, ran in Singapore — asia-southeast1. There was no India region on the free plan to choose.

So the steady-state shape of a single API request was:

  phone (India)
     │  ~40–60ms  (user → Mumbai API)
     ▼
  API  (Mumbai, asia-south1)
     │  ~60–70ms round trip  ┐
     ▼                       │  ×N queries per request
  Postgres (Singapore)  ─────┘

The user-to-Mumbai hop is unavoidable and fine. The Mumbai-to-Singapore hop is the killer, because it's inside the request, and it happens more than once. A single screen load — open a group, hydrate the balance, list the last fifty messages, resolve sender profiles — is not one query. It's several, sometimes sequentially dependent. Mumbai↔Singapore is roughly a 60–70ms round trip; the connection-pooling lesson explains why that number is a floor you cannot optimize away — it's the speed of light through fiber plus switching, not something a faster query planner touches. Do it five times in a request and you've spent a third of a second doing nothing but waiting for packets to cross the Bay of Bengal and come back.

Then it got worse. Neon's free tier scales to zero — it suspends the compute when idle to save money, which is exactly how it affords a free tier. The first request after an idle period pays a cold start: the migration notes record the symptom bluntly as a 3–5 second request lag. So the app had two overlapping latency problems wearing one disguise: a steady cross-region tax on every request, and a brutal cold-start spike on the first request after quiet.

Options considered

Four ways out, and three of them were traps.

Move the API to Singapore, next to the data. This makes the request-internal round trips fast — but it moves the problem onto the user. Every user is in India, so now every request pays the India↔Singapore hop on the outermost, most latency-sensitive leg: the one the user actually waits on. You'd trade many cheap internal hops for one expensive user-facing one. For a single-region, single-country product, co-locating compute away from your users is backwards.

Find a managed Postgres/Redis free tier in India. There wasn't one. Neither Neon nor Upstash offered an India region on their free plans. This option simply didn't exist to pick.

Self-host Postgres and Redis on the Mumbai VM. Co-located, effectively free — but now I own Postgres operations: backups, point-in-time recovery, failover, major-version upgrades, disk-full incidents. For a money ledger where the entire product promise is that the balances are exactly right forever, hand-rolling database durability is precisely the wrong corner to cut. Redis is a different story (more on that below), but the database is not where you want to be your own on-call DBA in month one.

Managed Cloud SQL in Mumbai, self-hosted Redis on the VM. The winner. Postgres moves to GCP Cloud SQL in asia-south1 — same region as the API, always-on (no scale-to-zero, so no cold start), and managed, so Google owns the backups and failover. Redis, being a cache and pub/sub layer whose contents are reconstructable, gets self-hosted as a container right next to the API. Media stays on R2 — it was never the problem, because media is fetched through a CDN edge, not on the request's critical path to the database.

The split is the whole insight: the stateful thing that must not lose data gets managed; the stateful-ish thing that can be rebuilt gets self-hosted and co-located.

Decision and why

The honest accounting: the free tier was never actually free. It cost ~60–70ms × several round trips on every request plus a 3–5 second cold-start spike, paid in the one currency users notice — the feel of the app. The migration converts that hidden latency cost into a visible, small money cost: a db-f1-micro instance at roughly $11.5/month (about ₹950) once the trial credits run out, and ₹0 while they last. Self-hosted Redis adds nothing — it's a container on a VM that's already paid for. TDD-001 predicted this exact moment: "cost rises as we cross free-tier thresholds — at that point we've validated the product enough to spend a few hundred dollars a month." Free tiers were the right first call precisely because they let the product prove itself before it cost anything; the mistake would have been paying for locality before knowing anyone would show up.

How it works

The shipped topology is one Docker Compose stack on the Mumbai VM: Caddy (TLS + reverse proxy), the NestJS api container, a cloud-sql-proxy sidecar, and a redis container. The API never talks to Cloud SQL directly. It talks to the proxy at cloud-sql-proxy:5432, and the proxy tunnels to the managed instance over Google's backbone.

The proxy's authentication is the elegant part: no key file anywhere. On a GCE VM, the Cloud SQL Auth Proxy authenticates using the VM's attached service account (granted roles/cloudsql.client at provisioning), fetched from the metadata server. There's no database credential JSON sitting on disk to leak. The API's DATABASE_URL points at cloud-sql-proxy:5432 — a hostname on the Compose network — and Prisma is none the wiser that the actual Postgres is a managed instance reached through an encrypted tunnel.

Redis runs deliberately small: --maxmemory 96mb with --maxmemory-policy volatile-lru. That policy matters on a ~1GB box — under memory pressure it evicts only keys with a TTL (idempotency keys, session cache) and never the un-expiring Socket.IO adapter state, so realtime fan-out can't be starved by a flood of short-lived keys. AOF persistence (appendonly yes) means a container restart doesn't wipe everything. One consequence, documented in the runbook: at cutover Redis started empty, so everyone had to re-log-in once — acceptable because this happened pre-launch, with no real users to inconvenience.

The migration, and why it took four commits

The plan was one commit: dump Neon, restore into Cloud SQL, flip the connection string. The reality was four, and each extra one is a lesson.

Commit 1 — the migration. A pg_dump from Neon, a pg_restore into Cloud SQL, a cloud-sql-proxy sidecar added to the prod Compose file, the self-hosted Redis service, and a row-count check (source vs. target) as the go/no-go gate. Straightforward. Wrong.

Commit 2 — the Postgres version surprise. The plan assumed Neon was Postgres 16, matching TDD-001. It wasn't — Neon was running Postgres 18.4. A pg_dump from 18 restored into a 16 target is a downgrade, and it fails: an 18 dump emits settings like transaction_timeout that a 16 server rejects outright. The fix had two parts. First, recreate the (still-empty) Cloud SQL instance as Postgres 18 to match the source — you keep the source and target on the same major version, always. Second, and more robust: stop depending on whatever pg_dump version happens to be installed on the migrating laptop, and instead run all the Postgres tooling inside a postgres:18 Docker container, joined to the proxy over a private Docker network. The host's local psql version became irrelevant. The migration script also drops and recreates the target database on each run, so a failed attempt leaves nothing half-restored to clean up.

Commit 4 — the health check that wouldn't start. After cutover, the API refused to boot. The api container depends_on the proxy with condition: service_healthy, and the proxy's health check ran cloud-sql-proxy wait --max=0s. That command polls the proxy's own readiness endpoint — but with --max=0s it gives itself zero time to confirm and exits non-zero every time. Docker marked a perfectly healthy proxy as unhealthy, so the dependency gate never opened and the API never started. The proxy was fine the whole time; the health check was lying. Fix: --max=5s and a timeout of 8s, so the check has time to actually confirm readiness. A dependency gate is only as trustworthy as the health check behind it — a check that's too strict fails closed and takes the whole stack down with it.

Interview takeaway

If a system-design interview touches regions, latency, or "the app feels slow," this chapter is what to say:

• "Data locality beats compute locality for a single-region product." Put the database next to the server that queries it most, and put the server next to the users. When those pull apart — users here, managed data there — co-locate compute with data for the internal round trips, but never at the cost of moving compute away from users. For Fable, both wanted to be in Mumbai; the free tier was the only reason the data wasn't.
• "Measure RTT × trips-per-request before you pick a region." A single number — the round-trip time between two regions — times the number of times a request crosses that gap is usually most of your latency budget. It's arithmetic you can do on a napkin before writing a line of code, and it's invisible to a profiler because the CPU is idle, waiting on the network.
• "Free tiers price latency in, not out." Scale-to-zero cold starts and far-away regions are how a provider affords ₹0/month. That's a genuinely good trade while you're validating — you pay in latency you don't yet have users to annoy. The skill is knowing the moment to convert that hidden latency cost into a small, explicit money cost.
• "Manage the system of record; self-host the reconstructable." Postgres holds the ledger, so it goes to managed Cloud SQL where someone else owns backups and failover. Redis holds cache and pub/sub state you can afford to rebuild, so it's a cheap local container. Match the operational burden to the cost of losing the data.
• And the meta-point of the whole series: the ₹0/month architecture wasn't a mistake I corrected — it was the correct first move whose expiry date I planned for. The migration off it took four commits, not one, because production infrastructure fails on the boring specifics: a major version you didn't check, extensions you didn't know were enabled, a health check that's too strict. None of that is in the runbook until you write the runbook.