Sockets & the kernel: what the server sees

Every lesson so far followed a request outward from the browser. This one stands on the other side and asks: when those bytes arrive, what is the server program actually doing to receive them? You've written server handlers — an Express route, a NestJS controller — but underneath your handler is a small, ancient set of operating-system calls that have looked the same since the 1980s. Understanding them is what turns "the backend" from a black box into a machine you can reason about.

We touched this boundary at the end of what happens when you fetch(): the OS hands request bytes to a listening process through a socket. Now we open that up — and the payoff is understanding, concretely, why your entire Node server can be frozen by one line of code.

A socket is a file descriptor

Here's the idea Unix is built on: almost everything is a file. An open file is a file. So is your terminal. So is a network connection. When a program opens any of these, the kernel hands it back a small integer — a file descriptor (fd) — that stands in for that resource. You've seen the famous three: fd 0 is stdin, fd 1 is stdout, fd 2 is stderr. The next thing you open gets fd 3, then 4, and so on.

A socket is just a file descriptor that happens to represent a network connection instead of a file on disk. That's the whole trick. Once you have the fd, you read() bytes from it and write() bytes to it with the same calls you'd use on a file. The kernel knows fd 5 is a TCP connection and does the packet machinery from the TCP lesson behind the scenes; your program just sees a numbered thing it can read and write. This is why fd exhaustion is a real server failure mode — every connection is an fd, and there's a limit.

The five syscalls a server makes

A syscall (system call) is how your program asks the kernel to do something it isn't allowed to do itself — talk to hardware, open a connection, allocate a socket. A server accepting connections makes essentially five, in a fixed order. Here it is in annotated pseudocode (this is real C socket API, lightly simplified):

// 1. Ask the kernel for a socket. Get back a file descriptor.
int server_fd = socket(AF_INET, SOCK_STREAM, 0);   // SOCK_STREAM = TCP

// 2. Claim an address+port so the kernel knows what traffic is yours.
bind(server_fd, "0.0.0.0:3000");                    // "port 3000, any interface"

// 3. Switch the socket into listening mode and size the accept queue.
listen(server_fd, backlog: 128);                    // backlog = queue depth

// 4. Block until a client connects; get a NEW fd for that one client.
while (true) {
    int client_fd = accept(server_fd);              // <-- sleeps here until a connection arrives

    // 5. Talk to this client over its own fd.
    read(client_fd, buffer);                        // pull the request bytes
    write(client_fd, response);                     // push the response bytes
    close(client_fd);                               // release the fd
}

Walk it once. socket() allocates the fd. bind() attaches it to a port — this is what "address already in use" complains about when another process already owns port 3000. listen() flips it from a plain socket into one that accepts incoming connections. Then accept() is the heart of it: it pulls one waiting client off the queue and returns a brand-new fd for that single client. The listening socket keeps listening; each accepted connection gets its own fd. From there read()/write() move the actual HTTP bytes.

The one to notice is accept(). By default it blocks — the process sleeps, using no CPU, until a connection shows up. That single fact drives the entire rest of this lesson.

The accept queue and what "backlog" means

Connections can arrive faster than your program calls accept(). The kernel doesn't drop them on the floor — it holds completed connections in an accept queue (also called the backlog), waiting for your next accept() to pick one up. The backlog argument to listen() sizes that queue.

If the queue is full — your program is too busy to call accept() fast enough and connections keep arriving — new connections get refused or dropped. This is exactly what a "connection refused" or mysterious timeout under heavy load can be: not your handler being slow, but your process not returning to accept() quickly enough, so the accept queue overflows. The backlog is a shock absorber, and like any buffer it only hides bursts, not a sustained rate you can't keep up with.

Blocking I/O and thread-per-connection

Look back at that loop: it handles one client at a time. While read(client_fd) waits for a slow client to send its request, the whole loop is stuck — nobody else gets served. Useless for a real server.

The classic fix is intuitive: give each connection its own thread (or process). accept() a connection, hand its fd to a fresh thread, and immediately loop back to accept() the next one. Each thread blocks on its own read()/write() independently; the OS scheduler juggles them. This is the thread-per-connection model, and it's how Apache's classic mode and many older servers worked. It's simple and it reads naturally — each thread is just straight-line "read request, do work, write response" code.

It has a hard ceiling, though, and the ceiling is memory. Every thread needs its own stack — commonly 1–8 MB reserved each. A thousand connections is a thousand threads is potentially gigabytes of stack alone, plus the CPU cost of the scheduler context-switching between thousands of threads that are mostly just sitting in read() waiting. At ten thousand idle-but-open connections, this model falls over — and that number has a name.

C10K: the wall

Around 1999, Dan Kegel wrote up the C10K problem — "how do you handle ten thousand concurrent connections on one machine?" The insight that named an era: thread-per-connection doesn't scale to that, and the bottleneck isn't the work. Ten thousand connections that are each mostly idle — a chat app, long-polling, slow mobile clients — do almost nothing most of the time, yet thread-per-connection charges you a full thread (megabytes + a scheduler slot) for each one just to wait. You run out of memory holding connections that aren't even busy.

The realization: you don't need a thread per connection. You need one thread that watches all the connections and only does work on the handful that are actually ready to be read or written right now. But to do that, the thread needs the kernel to tell it which fds are ready — so it isn't blocked on any single one.

Readiness: select → epoll/kqueue

That "which fds are ready?" question is answered by a readiness API, and its evolution is the story of scalable servers.

select() was the first. You hand it your whole set of fds and it blocks until at least one is ready to read or write, then tells you which. One thread can now manage many connections: ask select() what's ready, service exactly those, loop. The problem is that select() is O(n) — every call scans all your fds, and it caps out around 1024 fds. At 10,000 connections you're rescanning 10,000 fds on every single loop iteration just to find the few that changed. The bookkeeping to find the ready connections costs more than serving them.

epoll (Linux) and kqueue (macOS/BSD) fixed this. Instead of handing the kernel your entire fd set every call, you register your fds once, and the kernel maintains a ready list — it hands you only the fds that actually became ready, in O(1) relative to the total count. Ten thousand connections, three of them have data — you get back exactly those three, no scanning. This is the mechanism that made C10K (and far beyond) routine. The pattern built on it — one thread, blocked in epoll/kqueue, waking only to service ready fds, then looping — is the event loop.

How Node maps onto all of this

Now the part that matters for your actual job. Node.js is that event-loop pattern, wrapped in JavaScript. Under the hood, a C library called libuv runs one event loop on epoll (Linux) or kqueue (macOS) — exactly the readiness machinery above. When people say Node is "non-blocking," this is precisely what they mean: Node never sits blocked in a read() on one connection. It registers every socket with the kernel, and the loop wakes only when the kernel says "these fds are ready," runs your JavaScript callback for each, and goes back to waiting.

This is why a single Node process serves thousands of concurrent connections on one thread without thread-per-connection's memory blowup: the I/O wait — most of a request's life — is handled by the kernel's readiness list, not a parked thread each. But the same design carries a sharp edge, and it's the single most important thing to internalize about Node: