Lambda Basics with Python using Github Webhooks or API Gateway
Nov 4, 2015 · Commentscodecontinuous delivery pythonawslambdagithubgit
I recently needed to call a command whenever there was a push on my Github repo.
Since this was related to AWS tasks, I figured aws lambda
is a good candidate.
Here I will talk about the steps I took to enable all of this using aws lambda, python.
As a side note, I will also elaborate on using API gateway to call upon the lambda itself.
In a nutshell, what I will talk about:
- Create SNS Topic
- Create IAM user who can Publish
- Setup GitHub webhook
- Create Lambda function
- Setup API Gateway url that can call lambda function
Create SNS Topic
Go to AWS Console and click on Create Topic.
![Creating an API](/images/lambda_basics_sns_1.png)
Creating an API
Note down the ARN
because you will need that.
In my case it is something like arn:aws:sns:ap-northeast-1:XXXXXXXXXXXX:commits_1
Create IAM user
We need to setup an IAM user who can publish onto this SNS we just created. As a shortcut you can just create a simple user and initialize it with full access rights for testing purposes.
Once on IAM page click on “Create New Users” button at the top.
Creating new IAM user
It will then allow you to download credentials if you want
Creating new IAM user
Add permissions
Once an IAM user is created, by default there are no permissions attached to the account.
You can add permissions by going to permissions tab and clicking on “Attach Policy” button.
Attach Policy
Setup GitHub webhook
We need to assign a webhook for each Git commit. So let’s do the following:
- Navigate to your GitHub repo
- Click on “Settings” in the sidebar of that repo
- It has to be an actual repo settings, not profile settings
- Click on “Webhooks & Services”
- Click the “Add service” dropdown, then click “AmazonSNS”
- You will need the account details for the IAM user you just created
- The trigger will be delegated through this given IAM user
Add SNS Webhook
Add Account details from IAM user
Create Lambda function
Now that we have creater a user, assigned a trigger on Github, we need to create a function that will be run on an actual trigger. Let’s create a lambda function by going to AWS Lambda and clicking on “Create a Lambda Function”.
Filter by SNS in the given samples because we want to see SNS functions.
Create a Lambda function with the following code:
import json
print('Loading function')
def lambda_handler(event, context):
print("Received event: " + json.dumps(event, indent=2))
message = event['Records'][0]['Sns']['Message']
print("From SNS: " + message)
return message
![Python code](/images/lambda_basics_lambda_2.png)
Python code
Testing the setup so far
Testing GitHub webhook
Now that we have setup Lambda, SNS, and Github so far, it is time to test the setup.
Go to the “Webhooks and Services” under you repo settings and click on “Amazon SNS” that is
viewable at the bottom of that page. You will then be able to test the service.
Checking AWS
Now that we have been able to send the message, we also need to confirm that the lambda was actually called.
This can be done by looking at CloudWatch logs. Any lambda run will be logged under CloudWatch.
Even normal logging is also available there.
Going to CloudWatch Checking the lambda logs
Setup API Gateway
After all this is done, we can even create an API along with a public facing URL where all these services can be called (apart from just GitHub), if you’d like. So let’s do that.
Create API
![Creating an API](/images/lambda_basics_api_1.png)
Creating an API
Create Resource
![Creating Resource](/images/lambda_basics_api_2.png)
Creating Resource
Create Method
![Creating Method for GET](/images/lambda_basics_api_3.png)
Creating Method for GET
![Assign Lambda function to GET](/images/lambda_basics_api_4.png)
Assign Lambda function to GET
Test your GET Method
Now this API can be tested easily by just calling upon this URL:
https://XXXXXXXXXXXX.execute-api.ap-northeast-1.amazonaws.com/test/resource_1/
![Check your Response from Lambda using GET](/images/lambda_basics_api_5.png)
Check your Response from Lambda using GET
If you wanted to test from a URL, it would be actually going to this URL itself from the browser.
Misc
Using CLI
Try the following command to get the list of your lambda functions:
$ aws lambda --profile lambda.s3.1 --region ap-northeast-1 list-functions
{
"Functions": [
{
"CodeSize": 317,
"FunctionArn": "arn:aws:lambda:ap-northeast-1:XXXXXXXXXXXX:function:lambda_1",
"MemorySize": 128,
"Role": "arn:aws:iam::XXXXXXXXXXXX:role/lambda_basic_execution_1",
"Handler": "lambda_function.lambda_handler",
"Runtime": "python2.7",
"CodeSha256": ".....",
"FunctionName": "lambda_1",
"Timeout": 183,
"LastModified": "2015-11-15T07:49:28.367+0000",
"Version": "$LATEST",
"Description": "An Amazon SNS trigger that logs the message pushed to the SNS topic."
},
{
"CodeSize": 316,
"FunctionArn": "arn:aws:lambda:ap-northeast-1:XXXXXXXXXXXX:function:lambda_2",
"MemorySize": 128,
"Role": "arn:aws:iam::XXXXXXXXXXXX:role/lambda_basic_execution_1",
"Handler": "lambda_function.lambda_handler",
"Runtime": "python2.7",
"CodeSha256": ".......",
"FunctionName": "lambda_2",
"Timeout": 3,
"LastModified": "2015-11-14T14:03:00.083+0000",
"Version": "$LATEST",
"Description": "An Amazon SNS trigger that logs the message pushed to the SNS topic."
}
]
}
Custom Policy for Lambda Access
Validate Policy You can either just give “AmazonSNSFullAccess” to the user lambda.s3.1 or add the following Policy onto User->Permission->Add Inline policy->custom policy->Select->Policy Document
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"sns:Publish"
],
"Resource": [
"arn:aws:sns:ap-northeast-1:YOUR_NUMBER:Commits_1"
],
"Effect": "Allow"
}
]
}