Connection pooling & keep-alive

You already know, from what happens when you fetch(), that the second request to a host is dramatically cheaper than the first. This lesson is about why, and about the machinery — in your browser, your server, and your database driver — that exists entirely to make sure you almost never pay the "first request" price. It's one idea applied at three layers: opening a connection is expensive, so don't throw it away — reuse it.

The bill for a cold connection

Before a single byte of your actual request goes out on a brand-new HTTPS connection, you pay, in series:

• DNS — turn the name into an IP. On a cache miss, its own round trips (DNS lesson).
• TCP handshake — SYN / SYN-ACK / ACK, one full round trip (TCP lesson).
• TLS handshake — one round trip on TLS 1.3, two on 1.2 (TLS lesson).

That's roughly 3–4 round trips before byte one of your request. Put the latency numbers on it: to a server 70ms away, that's easily 200ms+ of pure setup, spent before your handler runs or your query executes. And recall from TCP's slow start that even after setup, a fresh connection can't use your full bandwidth immediately — its congestion window hasn't ramped up. So a cold connection is expensive twice over: round trips to open it, then more round trips to get up to speed.

The whole game of pooling and keep-alive is to pay that bill once and then ride the warm connection for every request after.

Keep-alive and the browser's connection pool

HTTP's answer is keep-alive: after a response completes, don't close the TCP connection — leave it open and send the next request down the same pipe. The DNS lookup is cached, the TCP handshake already happened, the TLS session is established, and the congestion window is already wide. The second request skips all of it and starts sending immediately. This is the default in HTTP/1.1 and the entire premise of HTTP/2's single multiplexed connection.

Your browser manages this automatically with a connection pool — a set of live, warm connections per host it keeps around and hands out to new requests. Under HTTP/1.1 that's the ~6-per-host set from the HTTP evolution lesson; under HTTP/2 it's typically one connection carrying everything. Either way, you get connection reuse for free in the browser. Where it stops being free — and starts being your job — is the database.

Why databases are so much worse

A warm HTTP connection is cheap to keep around. A database connection is not, and the reason is architectural. Postgres uses a process per connection. Every time a client connects, the Postgres server forks an entire OS process to serve just that one connection. That process carries real weight — on the order of 5–10 MB of memory each, before it does any work, and more once queries allocate working memory (work_mem is per-operation, per-query, so one connection running a complex query with several sorts can multiply its footprint).

Two things follow. First, connections are a scarce, heavy resource — a Postgres instance configured for max_connections = 100 means at most 100 of these processes, and pushing that number up costs real RAM whether or not the connections are busy. Second — and this is the one that takes down production — is the connection storm. Imagine a serverless or autoscaling setup where each app instance opens its own database connections. Traffic spikes, the platform spins up 200 instances, each opens 10 connections... and now 2,000 processes stampede a database sized for 100. It doesn't slow down gracefully; it falls over, refusing connections, and often takes a while to recover even after the spike passes. The cure is a pool: a fixed, reused set of connections that requests borrow and return, so the database never sees more than N processes no matter how many requests are in flight.

Client pools vs server-side poolers

Pooling shows up in two places, and they solve different halves of the problem.

Client-side pool — lives inside your application. Your ORM or driver (Prisma, HikariCP for the JVM, pg-pool for Node) keeps a small set of open connections and lends one to each query, returning it to the pool when done. Requests that arrive when all connections are busy wait in a queue for one to free up rather than opening new ones. Prisma, for instance, defaults its pool size to num_physical_cpus * 2 + 1. This caps the connections one app instance opens — but if you run many instances, each has its own pool, and their sum can still storm the database.

Server-side pooler — a separate process that sits between your apps and Postgres, e.g. PgBouncer. Every app instance connects to PgBouncer; PgBouncer maintains a small set of real connections to Postgres and multiplexes everyone's queries across them. This is what actually caps the total against the database, regardless of how many app instances exist — the answer to the connection storm. But how aggressively it multiplexes is a choice with sharp consequences:

Transaction mode is where the efficiency is, and where the footguns are. Because a client gets a different underlying connection for each transaction, anything that relies on connection-scoped state across transactions breaks:

• Prepared statements — a statement your driver PREPAREd on one physical connection doesn't exist on the different connection you get next time. Drivers that prepare-then-execute (many do, silently) can throw "prepared statement does not exist."
• Advisory locks, session variables (SET), LISTEN/NOTIFY, temporary tables — all connection-scoped. Take an advisory lock in one transaction and it's on a connection you won't see again; the lock is effectively lost or stuck.

None of these are bugs in PgBouncer — they're the direct, logical cost of not owning your connection between transactions. Transaction mode trades connection-scoped features for the ability to serve thousands of clients on a handful of real connections.

The counterintuitive part: smaller is faster

Every instinct says a bigger pool serves more requests. For databases, past a point, a bigger pool is slower — and the argument (made canonical by the HikariCP project) is worth internalizing.

A database server has a fixed amount of real parallelism: a finite number of CPU cores and a finite number of disk spindles. Ten thousand connections do not mean ten thousand queries running at once — they mean ten thousand queries fighting over the same handful of cores, and the machine spends its time context-switching between them and thrashing caches instead of finishing work. Fewer connections, each running to completion before the next starts, finishes the total workload faster — the same reason a supermarket with 4 lanes and a queue serves people faster than 40 half-staffed lanes.

HikariCP's rule of thumb makes it concrete: for many workloads the right pool size is close to

connections ≈ (core_count × 2) + effective_number_of_disk_spindles

which for an 8-core box is roughly 20-ish, not 200. A pool of a few dozen, fully utilized, will out-throughput a pool of hundreds that's drowning the database in contention. The queue in front of a small pool isn't waste — it's what keeps the database in its efficient regime.