What is FaaS? A Simple Walkthrough
The Idea in One Sentence
FaaS (Function as a Service) lets you deploy just your code, a single function, and the platform handles everything else: servers, scaling, patching, capacity planning. You write def hello(): return "hi", upload it, and forget the infrastructure exists.
Where This Problem Actually Came From (Meta)
Meta’s engineers used to write infrastructure code every time they shipped a new service. That’s fine for steady traffic. It falls apart for spiky work, things like sending scheduled notifications or generating thumbnails, where load jumps hard at certain times and goes quiet the rest of the day.
Pre-allocated servers can’t flex for that. You either over-provision and waste money, or under-provision and get slow, or fall over entirely. So they built a FaaS layer: you hand over a function, and the platform decides where and when to run it.
Think of FaaS as a job queue, except someone else owns the servers behind the queue.
The Basic Flow
client → gateway → function server → function database
↓
metadata database- A client sends a request.
- The gateway routes it to a function server.
- The function server runs your code and returns the result.
- Metadata (which function, which version, resource limits) lives in a separate database so the routing layer knows how to handle each function.
That’s the whole idea at small scale. The hard part is making this work when you have millions of functions and no idea which ones will get called next.
Now Let’s Build One
Step 1: Decide how functions actually run
You have two real choices here: virtual machines or containers, with functions running inside as isolated Linux processes. Isolation matters because you’re running code from many different teams (or customers) on shared hardware, and one function crashing shouldn’t take down another.
Step 2: Solve the idle-server problem (cost)
Here’s a fact that shapes the whole design: most functions are called rarely and finish fast. In Meta’s data, 81% of functions are invoked at most once a minute, and most run in under a minute.
If you keep servers warm waiting for these rare calls, you’re burning money on idle capacity. 144 idle servers for 10 minutes is a full server-day wasted, and at Meta’s scale that gap adds up to tens of thousands of extra machines.
So the design goal becomes minimizing idle time without making the caller wait too long. That tension is really the core engineering problem in FaaS.
Step 3: Solve the cold-start problem (latency)
When there’s no warm server ready, the platform has to spin one up from scratch. That’s called a cold start, and it means:
- Start the VM
- Download the container image and function code
- Initialize the container
- Initialize the language runtime
- JIT-compile the code
Every one of those steps adds latency the caller feels directly. And if a container sits unused too long and gets shut down to save cost, the next call pays the cold-start tax all over again.
A few tricks engineers use to cut this down:
- Universal worker model. A worker can run any function written in that language, not just one specific function. This means a Python worker is generically useful instead of dedicated to a single job, which shrinks how often you need a true cold start.
- Locality groups. Group workers that tend to run the same functions together. This keeps their JIT-compiled code cache small and reused, instead of every worker compiling everything from scratch.
- Pre-push code to disk. Instead of downloading the function code and container image on every cold start, push them onto the worker’s SSD ahead of time.
- Cooperative JIT compilation. Instead of every worker compiling new code independently, one worker compiles it once and shares the compiled result with the rest. Compile once, reuse everywhere.
Step 4: Protect your backend services (availability)
Functions can scale up almost instantly. Your database or downstream services can’t. A traffic spike in your functions can easily overload whatever they’re calling.
The fix here is backpressure-based throttling, borrowed straight from TCP congestion control:
- The backend signals congestion (error rate, latency, capacity).
- A rate limiter slows down requests hitting that backend.
- Once things calm down, it gradually ramps requests back up.
- If congestion returns, it throttles again.
This only throttles functions that call external services. You don’t want to slow down functions that are self-contained and don’t need it.
Step 5: Handle peak traffic gracefully
Even with throttling in place, you need hard limits: cap how many times a function can run, or how much resource it can consume, before it gets rate-limited outright.
Beyond that, spread the work out:
- Distribute execution across data centers so no single one absorbs the whole spike.
- Delay low-priority functions until off-peak hours using a separate priority queue. A scheduler handles high-priority work immediately, while a lower-priority queue waits for free capacity.
Step 6: Design your state carefully
Most services in a FaaS platform should be stateless and replicated. That’s what makes horizontal scaling easy. Anything that must hold state gets partitioned and replicated separately, because state is exactly what makes scaling hard in the first place.
The Core Trade-off to Remember
FaaS doesn’t remove the systems engineering problem, it just relocates it. You’re no longer thinking about how many servers to run. You’re thinking about:
- How fast can I go from cold to warm?
- How do I avoid paying for idle time?
- How do I keep a traffic spike in my functions from turning into an outage in my database?
Everything above, universal workers, locality groups, cooperative JIT, backpressure, exists to answer those three questions at scale.