Python and AWS Lambda – A match made in heaven

In recent months, I’ve begun moving some of my analytics functions to the cloud. Specifically, I’ve been moving them many of my python scripts and API’s to AWS’ Lambda platform using the Zappa framework.  In this post, I’ll share some basic information about Python and AWS Lambda…hopefully it will get everyone out there thinking about new ways to use platforms like Lambda.

Before we dive into an example of what I’m moving to Lambda, let’s spend some time talking about Lambda. When I first heard about, I was a confused…but once I ‘got’ it, I saw the value. Here’s the description of Lambda from AWS’ website:

AWS Lambda lets you run code without provisioning or managing servers. You pay only for the compute time you consume – there is no charge when your code is not running. With Lambda, you can run code for virtually any type of application or backend service – all with zero administration. Just upload your code and Lambda takes care of everything required to run and scale your code with high availability. You can set up your code to automatically trigger from other AWS services or call it directly from any web or mobile app.

Once I realized how easy it is to move code to lambda to use whenever/wherever I needed it, I jumped at the opportunity.  But…it took a while to get a good workflow in place to simplify deploying to lambda. I stumbled across Zappa and couldn’t be happier…it makes deploying to lambda simple (very simple).

OK.  So. Why would you want to move your code to Lambda?

Lots of reasons. Here’s a few:

  • Rather than host your own server to handle some API endpoints — move to Lambda
  • Rather than build out a complex development environment to support your complex system, move some of that complexity to Lambda and make a call to an API endpoint.
  • If you travel and want to downsize your travel laptop but still need to access your python data analytics stack move the stack to Lambda.
  • If you have a script that you run very irregularly and don’t want to pay $5 a month at Digital Ocean — move it to Lambda.

There are many other more sophisticated reasons of course, but these’ll do for now.

Let’s get started looking at python and AWS Lambda.  You’ll need an AWS account for this.

First – I’m going to talk a bit about building an API endpoint using Flask. You don’t have to use flask, but its an easy framework to use and you can quickly build an API endpoint with it with very little fuss.  With this example, I’m going to use Lambda to host an API endpoint that uses the Newspaper library to scrape a website, pull down the text and return that text to my local script.

Writing your first Flask + Lambda API

To get started, install Flask,Flask-Restful and Zappa.  You’ll want to do this in a fresh environment using virtualenv (see my previous posts about virtualenv and vagrant) because we’ll be moving this up to Lambda using Zappa.

Our flask driven API is going to be extremely simple and exist in less than 20 lines of code:

Note: The ‘host =’ and ‘port=50001’ are extranous and are how I use Flask with vagrant. If you keep this in and run it locally, you’d need to visit to view your app.

The last thing you need to do is build your requirements.txt file for Zappa to use when building your application files to send to Lambda. For a quick/dirty requirements file, I used the following:

Now…let’s get this up to lambda.  With zappa, its as easy as a couple of command line instructions.

First, run the init command from the command line in your virtualenv:

You should see something similar to this:

zappa init screenshot

You’ll be asked a few questions, you can hit ‘enter’ to take the defaults or enter your own. For this eample, I used ‘dev’ for the environment name (you can set up multiple environments for dev, staging, production, etc) and made a S3 bucket for use with this application.

Zappa should realize you are working with Flask app and automatically set things up for you. It will ask you what the name of your Flask app’s main function is (in this case it is Lastly, Zappa will ask if you want to deploy to all AWS regions…I chose not to for this example. Once complete, you’ll have a zappa_settings.json file in your directory that will look something like the following:

I’ve found that I need to add more information to this json file before I can successfully deploy. For some reason, Zappa doesn’t add the “region” to the settings file. I also like to add the “runtime” as well. Edit your json file to read (feel free to use whatever region you want):

Now…you are ready to deploy. You can do that with the following command:

Zappa will set up all the necessary configurations and systems on AWS AND zip up your libraries and code and push it to Lambda.   I’ve not found another framework as easy to use as Zappa when it comes to deploying…if you know of one feel free to leave a comment.

After a minute or two, you should see a “Deployment Complete: …” message that includes the endpoint for your new API. In this case, Zappa built the following endpoint for me:

If you make some changes to your code and need to update Lambda, Zappa makes it easy to do that with the following command:

Additionally, if you want to add a ‘production’ lambda environment, all you need to do is add that new environment to your settings json file and deploy it. For this example, our settings file would change to:

Next, do a deploy prod and your production environment is ready to go at a new endpoint.

Interfacing with the API

Our code is pushed to Lambda and ready to start accepting requests.  In this example’s case, all we are doing is returning “hello world” but you can see the power in this for other functionality.  To check out the results, just open a browser and enter your Zappa Deployment URL and append /hello to the end of it like this:

You should see the standard “Hello World” response in your browser window.

You can find the code for the lambda function here.

Note: At some point, I’ll pull this endpoint down…but will leave it up for a bit for users to play around with.


If you want to learn more about Lambda, there are two fairly good books on the topic – check them out (Amazon links):


Eric D. Brown , D.Sc. has a doctorate in Information Systems with a specialization in Data Sciences, Decision Support and Knowledge Management. He writes about utilizing python for data analytics at and the crossroads of technology and strategy at

3 thoughts on “Python and AWS Lambda – A match made in heaven

  1. There’s one important architectural drawback of using Zappa: it’s terribly slow. It took 4.26 seconds to access your simple “Hello world” endpoint, 3.99 s was spent waiting on Zappa (Time To First Byte). It can be much faster (300~400 ms) on subsequent requests, but the delay is pretty significant and some of your users may have this bad experience with your service. Sure, using AWS Lambda for infrequent stuff is fine, but it isn’t for hosting a web service via Zappa. A VPS (which you can get as cheap as €1 at Aruba Cloud, and even $5 is not that much cash) will work much better and make your users happier.

    1. First: This is using AWS Lambda not “Zappa”. Zappa is just the framework to send the code up to Lambda and has nothing to do with the execution of the code on the Lambda platform.

      Second: While using Zappa, it automatically sets a ‘keep warm’ so your lambda code is ready to serve immediately. Sure, it may take some time to spin up on this particular example (although there shouldn’t because I have the keep warm turned on) but in a ‘production’ environment, this can easily be solved. I’ve hit the URL for this example a number of times over the last 24 hours and execution has been immediate.

      Third: Your recommendation of a VPS is EXACTLY the problem that I am trying to solve by moving to Lambda. I DON”T WANT TO MANAGE MY OWN SERVER – which I would have to do with a VPS. Additionally, Spending €1 or $5 a month or whatever number someone wants to throw out there is still much more expensive than spending 10 cents a month for a lambda function that I call a few times a month (or even a few thousand calls a month). Sure, that VPS might make economic sense if I have 1,000 scripts that I hit thousands of times a month and each script runs for multiple minutes per execution vs lambda. A script that takes 30 seconds to execute and is run 10,000 times will cost 63 cents a month to run on lambda (with 128MB of Allocated Memory). Combine that with hundreds / thousands of other scripts with similar run numbers and times and lambda can definitely add up. But…everyone has to consider system administration overhead in any economic calculation for this type of thing…and for me, no sysadmin time per month is worth it.

      Fourth: For large scale production systems, Lambda works wonderfully (when configured correctly). How would a €1 VPS running Aruba Cloud work for that? I can tell you exactly how well Lambda works in this instance…because I use it daily for multiple API endpoints that get a million+ hits a day with absolutely no issues and zero system administration headache. Lambda (when configured correctly) will handle load balancing and scalaability for your (unlike a €1 VPS).

Leave a Reply

Your email address will not be published. Required fields are marked *