Proxy vs Load Balancer vs API Gateway: One Box, Three Jobs

0:00The Box You Didn't Know About

Open an app like Netflix. Before you see a single title, your TV fires off a request: show me my home screen. You'd think it goes straight to Netflix's servers, the machines that know what's on tonight. It almost never does. Between you and those machines sits a box you didn't know about, and everything you send stops there first. Every big app has one — you passed through several today without noticing. And that box goes by three names — reverse proxy, load balancer, API gateway — which people mix up constantly. In design meetings. In job interviews. In diagrams that are quietly wrong. So let's follow one request through the box, and build a simplified Netflix as we go. And a hint for the ending: the names cause trouble — but not because there are three boxes.

0:53Reverse Proxy — The Public Face

Start with the word. A proxy is anyone who acts on your behalf. Say you're famous and want to buy vegetables without being recognised — so you send an assistant to the shop. The shopkeeper deals with your assistant and never sees you. On a network, that's a forward proxy: you point your laptop at it, and websites see the proxy, not you. You chose it, and it hides you. Now flip it. Walk into a restaurant: you never enter the kitchen. You order from the waiter, who passes it to the kitchen and brings the food back. The waiter works for the restaurant, not for you — and from your seat, the waiter is the restaurant. That's a reverse proxy. So picture Netflix on day one: one server, the kitchen — and a waiter in front of it. Your TV opens a connection to the waiter and asks for the home screen. The waiter opens a connection of its own to the real server, asks the same question, and hands the answer back over the first connection. Two connections. One request. And as far as your TV can tell, the waiter is Netflix. Why bother, with one server? Three reasons. The waiter is where the encryption gets unwrapped, so the kitchen never faces the open internet. It hides what's behind it, so you can swap the server tonight and nobody outside will know. And it keeps copies of popular answers, so the kitchen isn't asked the same thing twice. That's the position: the public face. It's the job NGINX is famous for.

2:30Load Balancer — Somebody Has to Choose

Now it's Friday night, and one server can't keep up. So there are several, all running the same app. Every request has to go to one of them — and somebody has to choose. That job is load balancing, and the box that chooses is a load balancer. How does it choose? The simplest rule: take turns — first request to the first server, next to the second, round and round. Or pick whichever server has the fewest connections open right now. Or keep the same customer on the same server — handy when a server remembers something about you. You can tip the scales, too: give a big machine a weight of five and a small one a weight of one, and five requests in six go to the big one. And one rule outranks all of that. Every few seconds, the balancer asks each server: are you still there? A server that stops answering is quietly dropped from the rotation, and eased back in when it returns. Your TV never sees any of this. It just gets an answer.

3:42Envelope or Letter — Layer 4 vs Layer 7

So far the balancer chose a server without really reading your request. And that's the question that splits load balancers: how much of the request is it allowed to read? Think of the request as a letter inside an envelope. Some balancers read only the envelope — the addresses on the outside — and never look in. They forward it sealed — fast, because there's nothing to understand. Others open the letter. They end your connection right there, read what's inside — which page you want, which cookie you carry — and open a fresh connection to the server they pick. That's how artwork requests go to the artwork servers and payments to the payment servers — one front door, different machines. The jargon for this is layers. The internet is built in numbered layers, like wrappings, and the number just says how far in the balancer reads: the envelope is layer four; the letter is layer seven. Amazon sells one of each — a network load balancer for the envelope, an application load balancer for the letter. Now notice what the letter-opener had to do. It ended your connection, answered you itself, then went and asked a server. That's exactly what the reverse proxy did in chapter two. So a balancer that opens the letter is — in the industry's own words — often referred to as a reverse proxy server. It has to be. To read the letter, it must stand in as the server. Same box. Second job.

5:23API Gateway — The Front Door

Now the app grows up. Behind the box there's no longer one program. There's a catalogue service, a recommendations service, a playback service, a billing service — each run by its own team. And the callers aren't browsers loading pages; they're TVs and phones calling the app's API — the fixed set of requests those services have agreed to answer. That's when the box earns its third name: an API gateway. Watch your home-screen request again. The gateway takes it, asks catalogue for the rows, asks recommendations what order to put them in, asks billing whether this account is paid up — and stitches the three answers into one reply for your TV. One request in, several services consulted, one answer out. Amazon calls it, plainly, a front door. And the checks at that door are about the caller, not the traffic. Who are you — authentication. Are you allowed — authorization: a kids profile shouldn't open this title. How many calls a second do you get — rate limiting: an app gone haywire is told to slow down. Work every service would otherwise repeat, done once, at the door. And under all those policies, the gateway still routes, still balances — it's standing in front of the servers, same as ever.

6:57One Box, Three Jobs

So here's the untangling, on the same picture. Reverse proxy is a position: the box in front of your servers, standing in for them. Load balancer is a job you give that position when there's more than one server behind it — the HTTP spec itself lists load balancing as one of the jobs a reverse proxy is for. API gateway is the job when what's behind it is an API, and the door needs policies. One position. Two jobs you can hand it. Which is why one program can do all three: NGINX is documented as a reverse proxy, documented as a load balancer — and its commercial edition is sold as an API gateway. So the question is never which one to buy. One server? Put a reverse proxy in front of it. Several copies of the same server? Give that box the balancing job. Many services behind an API, with rules to enforce? Give it the gateway job. And when an interviewer asks — reverse proxy or load balancer? — that was never a choice between machines. It's a question about what the box in front of your servers has to do. Follow the request. It stops at the box either way.