We run some Python functions on AWS Lambda, and we originally used serverless. We set up the config files, node packages, etc. for that to deploy to our different environments. It worked pretty well. The main benefit is that serverless packaged up Python dependencies and deployed them with our main function code.

Believe it or not, those functions just ran for 5 years without needing upgrades or changes. Once it was time to upgrade our functions from Python 3.6, which is EOL on Lambda, to 3.9 we had to dig back into how the deployments worked. Unsurprisingly, serverless and other node packages were out of date and that deploy process didn’t work anymore. I had the choice between diving back into serverless (and probably starting from scratch) or looking at alternatives.

I decided to look at alternatives and found out about a newer feature in Lambda: layers.

Layers consist of pieces of your functions that you want to package together. They can be used across functions to share code or dependencies. Layers are analogous to Docker layers (maybe it’s what Lambda uses under the hood?)

The way we’re now using layers is to build our Python dependencies once, create a layer, then use that layer as the basis for the three Lambda functions we have. With serverless, each function contained all the Python dependencies independently. This was heavier, more complicated, and also meant I couldn’t make quick edits/tests in the Lambda code editor when necessary. Now, I create the layer once and use it in each function. When I look at the functions in the AWS web console, I just see the code I wrote for that function.

This is the blog post where I initially learned about layers: https://dev.to/mmascioni/using-external-python-packages-with-aws-lambda-layers-526o. It’s also a great tutorial on how to use local docker to run pip to install and package Python dependencies for the version of Python you are using on Lambda. Then, you create a layer our of that.