Hi all, Dave Cline here, Software Engineer at Orum.
Serverless systems are tough to wrap your head around. There are a billion different variables to adjust, and figuring out what value to set for each one can be a chore. This post attempts to demystify at least one of these variables, called reserved concurrency, and to give suggestions on how to use it. Reserved concurrency is perhaps the most dangerous variable to leave unset, so if you are running lambdas in AWS and don’t know what it is, you should definitely read this post.
How do lambdas work?
If you already know how lambdas work feel free to skip this section. I’ll try to keep it brief.
To understand reserved concurrency, you first need to understand a little bit about how lambdas work. Basically, a lambda function is a very small container that starts up when it is asked to do some kind of calculation. Allow me to demonstrate with an extremely accurate and highly technical diagram.
The cold start / initialization process is pretty fast, usually taking up to one second. This opens up a lot of possibilities that weren’t really available until now. Namely, you can spin up a new container every time you need to process something, then spin it back down again immediately afterwards. That used to take minutes rather than milliseconds.
So now if you want to do something like, say, processing 10,000 sales events as fast as possible, you can literally create 1 little lambda for each sales event and then have them all do their processing at the same time. It’ll end up taking about 1 second total to spin up all 10,000 lambdas, and then say 1 second for each one to finish its work in parallel, for a total processing time of 2 seconds for all 10,000 events. So how long would it take for 20,000 events? 100,000? As long as you let enough lambdas run at once, then your processing times will remain a constant 2 seconds no matter how much input is coming down the pipeline.
Does this mean we have unlimited processing power because we can spin up an infinite number of lambdas to solve any number of problems in constant time?!? Well, not quite. As always, with great computing power comes great computing responsibilities. In order to protect your own hardware, you’re going to need to put guard rails on these little guys by adding “reserved concurrency” limits.
What is reserved concurrency?
Reserved concurrency is a variable that determines how many lambdas your AWS account can run at once. If you have 100 inputs in a queue, and you want to process them all at once, you’ll want a reserved concurrency of 100. If you want to limit the number of inputs you’re processing in parallel to say, 50, then you’ll want a reserved concurrency of 50.
Why is reserved concurrency important?
Why not just use unlimited reserved concurrency? The short answer is: lambdas are too powerful to leave to their own devices, and they will destroy your infrastructure if left unchecked. Let me outline a couple of ways leaving reserved concurrency unset, or too high, can cause serious problems.
Problem number one: finite resources
Imagine a lambda function that runs a simple query against a database. Sounds like… one lambda, one database, right? In reality it’s very different, because each function will spin up as many copies of itself as it has inputs. Here’s a visual:
You can see that a database or similar resource can very quickly get overwhelmed, even if it is only attached to a “single” lambda function. Setting a reserved concurrency of one would make the diagram on the right look like the diagram on the left, essentially telling the lambda to only spin up one copy of itself regardless of the number of inputs. Most likely your database can handle more than one connection at a time, though, so in practice you should do some testing to find a good balance between processing power and database load. By setting a reserved concurrency on your lambdas, you can control the load they place on their ancillary resources.
Problem number two: lambdas competing for shared concurrency
Each account in AWS actually comes with a default maximum concurrency (called unreserved concurrency) for your lambdas. So even if you don’t set an explicit reserved concurrency, your lambdas are getting one from this default. This is ok from a processing perspective, because the limit is high enough that you aren’t going to have trouble with not-enough lambdas spinning up to process inputs efficiently. But, there is another issue, which is that all of the lambdas in your account share this default limit.
This means your lambdas may be competing with each other for unreserved concurrency, and every lambda that is running is one less slot for a different lambda to run in. If you don’t have a lot going on in your system overall, this probably doesn’t matter. But, if 1 of your lambdas needs to process 10,000 events, and your account reserved concurrency is only 1,000, then you can be certain that the lambda with all the events is going to be doing its best to hog all 1,000 of those concurrency slots for itself no matter how many other lambda functions you have set up. Because I like illustrations so much, here’s one more:
If lambda function one is sending emails, and lambda function two is your login processing lambda, then no one is going to be able to log in in the diagram on the right. When a lambda function is attempting to process an input but AWS isn’t giving it space to run in, it will fail to spin up, and the input will be returned to wherever it came from. This is called a “throttle” and is normally something you want to avoid, especially if a lambda is customer-facing and/or not backed by a queue.
To avoid throttles, you should put a reasonable reserved concurrency on every one of your lambdas. That way no matter how much work a lambda has to do, it won’t hog the entire account’s concurrency in order to do it, and your other lambdas will be able to do their jobs in parallel as well.
Lambdas are great, but are also very powerful and often misunderstood. Hopefully after reading this you can at least avoid the biggest pitfall—not setting reserved concurrency.