Load balancing and stateless services
Put one address in front of many interchangeable copies of a service, and move whatever makes a copy special out of it. Then one box going down is a slower minute, not an outage.
Round robin or least connections: which should I say in an interview?
Round robin when requests cost about the same and finish quickly, as page renders and small API calls do. It keeps no state beyond a turn and cannot get its bookkeeping wrong. Least connections when request lengths vary a lot, as with websockets, uploads or slow queries, because round robin keeps handing a replica new work while it is still holding long requests. Say the reason with the choice: it is the reason the interviewer is listening for.
What does stateless mean for a service behind a load balancer?
It means a replica keeps nothing between requests that another replica would need to serve the next one. Sessions, carts and uploads in progress live in a shared store or in a signed token the client carries. The replica can still cache things, as long as losing that cache only makes it slower. Once that holds, the balancer may send any request anywhere, and losing a replica loses no user state.
Are sticky sessions bad?
They are a trade, not a mistake. Pinning a user to one replica lets that replica keep the session in memory, which saves a round trip to a store on every request. The costs are uneven load, because some users are much heavier than others, and lost sessions when the replica dies or is redeployed. Use them when the state is expensive to move and cheap to lose; otherwise put the session in a store.
Is the load balancer itself a single point of failure?
It would be if there were one. In practice there are at least two, sharing a virtual address that moves to the survivor when one fails, or several behind DNS that returns more than one address. A balancer does very little work per request, so a pair handles a very large fleet. In a round it is enough to say that the balancer is redundant and how, then move on.
What is the difference between a layer 4 and a layer 7 load balancer?
A layer-4 balancer routes by address and port, so it forwards whole connections without reading them. That makes it cheap and protocol-agnostic. A layer-7 balancer terminates the connection and reads each request, so it can route by path, header or cookie, retry a failed request on another replica, and terminate TLS. The price is the work of parsing every request, and it has to understand the protocol.