- Core Features
- API Documentation
- AWS Lambda Authorization workflow
- API Components built through Terraform
- API Versioning through URI path
- API Configuration (rate limiting & throttling)
- API Testing
- CI/CD (Github Actions -> Terraform -> AWS)
- Observability, Error Tracking & Cost Monitoring
- REST APIs vs HTTP APIs
- API Development Lifecycle
- Further improvements
REST API should contain 3 endpoints:
-
hello
is a public endpoint. All requests are delivered intohello
Lambda function. -
goodbye
is a private endpoint. Access validated withAuthorization: <token>
presence in request header viaLambda Authorizer
function. Validated requests are delivered intogoodbye
Lambda function. -
welcome
is a private endpoint. Access validated throughx-api-key
presence in request header. Validated requests are delivered intowelcome
Lambda function.
OpenAPI Specification
(formerly Swagger Specification) is an API description format for REST APIs:
- https://github.com/qct/swagger-example/blob/master/README.md#introduction-to-openapi-specification
- https://swagger.io/blog/api-documentation/what-is-api-documentation-and-why-it-matters/
- https://swagger.io/docs/specification/paths-and-operations/
-
Swagger / OpenAPI
YAML
documentation file (format easier to read & maintain) created following standard guidelines: https://github.com/juanroldan1989/terraform-with-rest-api-gateway-and-lambda-functions/blob/main/terraform/docs/api/v1/main.yaml -
YAML
file converted intoJSON
(sinceSwagger UI
script requires aJSON
file):
docs/api/v1% brew install yq
docs/api/v1% yq -o=json eval main.yaml > main.json
-
JSON
file can be accessed through:3.a.
Github repository
itself as: https://raw.githubusercontent.com/github_username/terraform-with-rest-api-gateway-and-lambda-functions/main/docs/api/v1/main.yaml or3.b.
S3 bucket
that will containmain.yml
. Bucket created and file uploaded through Terraform.3.c. Terraform
output
command will show this value underapi_v1_docs_main_json
variable.
- Both file accessibility options available within this repository.
-
static
API Documentationstandalone
HTML page generated withindocs/api/v1
folder in repository: https://github.com/swagger-api/swagger-ui/blob/master/docs/usage/installation.md#plain-old-htmlcssjs-standalone -
Within
static
API Documentation page, replaceurl
value with your ownJSON
file's URL from point3
above:
...
<script>
window.onload = () => {
window.ui = SwaggerUIBundle({
// url: "https://<api-id>.execute-api.<region>.amazonaws.com/main.json",
dom_id: '#swagger-ui',
...
- A
static website
can also be hosted withinS3 Bucket
: https://docs.aws.amazon.com/AmazonS3/latest/userguide/WebsiteHosting.html
- To upload files
aws sync
command is recommended. E.g.:aws s3 sync docs/api/v1 s3://$YOUR_BUCKET_NAME
-
The client calls a method on an API Gateway API method, passing a bearer token or request parameters.
-
API Gateway checks whether a Lambda authorizer is configured for the method. If it is, API Gateway calls the Lambda function.
-
The Lambda function authenticates the caller by means such as the following:
-
Calling out to an OAuth provider to get an OAuth access token.
-
Calling out to a SAML provider to get a SAML assertion.
-
Generating an IAM policy based on the request parameter values.
-
Retrieving credentials from a database.
-
If the call succeeds, the Lambda function grants access by returning an output object containing at least an IAM policy and a principal identifier.
-
API Gateway evaluates the policy.
-
If access is denied, API Gateway returns a suitable HTTP status code, such as 403 ACCESS_DENIED.
-
If access is allowed, API Gateway executes the method. If caching is enabled in the authorizer settings, API Gateway also caches the policy so that the Lambda authorizer function doesn't need to be invoked again.
https://docs.aws.amazon.com/apigateway/latest/developerguide/apigateway-use-lambda-authorizer.html
-
A Lambda authorizer (formerly known as a custom authorizer) is an API Gateway feature that uses a
Lambda function to control access to your API
. -
A Lambda authorizer is useful if you want to implement a
custom
authorization scheme that uses a bearer token authentication strategy such as OAuth or SAML, or that uses request parameters to determine the caller's identity. -
When a client makes a request to one of your API's methods, API Gateway calls your Lambda authorizer, which takes the caller's identity as input and returns an IAM policy as output.
-
REST API Gateway implemented via Terraform (Infrastructure as Code)
-
Reference: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/api_gateway_rest_api
-
Resource:
aws_api_gateway_rest_api
-
Manages an API Gateway REST API.
-
The REST API can be configured via importing an OpenAPI specification in the
body
argument (with other arguments serving as overrides) OR -
via other
Terraform resources
to manage the resources (aws_api_gateway_resource
resource), methods (aws_api_gateway_method
resource), integrations (aws_api_gateway_integration
resource), etc. of the REST API. -
Once the
REST API
is configured, theaws_api_gateway_deployment
resource can be used along with theaws_api_gateway_stage
resource to publish the REST API. -
With a REST API we can apply Usage Plans, Rate Limits and Throttle Configuration.
-
Reference: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/apigatewayv2_api
Resource: aws_apigatewayv2_api
Manages an Amazon API Gateway Version 2 API.
Note:
-
Amazon API Gateway Version 2 resources are used for creating and deploying WebSocket and HTTP APIs.
-
To create and deploy REST APIs, use Amazon API Gateway Version 1 resources.
Authorization logic applied through Lambda Authorizer
function:
% curl https://<api-id>.execute-api.<region>.amazonaws.com/v1
{ "message" : "Missing Authentication Token" }
# 3-rest-api-gateway-integration-hello-lambda.tf
...
resource "aws_api_gateway_method" "hello_method" {
rest_api_id = aws_api_gateway_rest_api.main.id
resource_id = aws_api_gateway_resource.hello_resource.id
http_method = "GET"
authorization = "NONE"
}
curl https://<api-id>.execute-api.<region>.amazonaws.com/v1/hello
{ "message" : "Hello, world!" }
Authorization logic applied through Lambda Authorizer
function:
# 3-rest-api-gateway-integration-goodbye-lambda.tf
...
resource "aws_api_gateway_method" "hello_method" {
rest_api_id = aws_api_gateway_rest_api.main.id
resource_id = aws_api_gateway_resource.goodbye_resource.id
http_method = "GET"
authorization = "CUSTOM"
authorizer_id = aws_api_gateway_authorizer.main.id
}
$ curl https://<api-id>.execute-api.<region>.amazonaws.com/v1/goodbye
{ "message" : "Unauthorized" }
$ curl https://<api-id>.execute-api.<region>.amazonaws.com/v1/goodbye \
-H "Authorization: allow"
{ "message" : "Goodbye!" }
Authorization logic applied through API_KEY
:
# 3-rest-api-gateway-integration-goodbye-lambda.tf
...
resource "aws_api_gateway_method" "welcome_method" {
rest_api_id = aws_api_gateway_rest_api.main.id
resource_id = aws_api_gateway_resource.goodbye_resource.id
http_method = "GET"
authorization = "NONE"
api_key_required = true
}
$ curl https://<api-id>.execute-api.<region>.amazonaws.com/v1/welcome
{ "message" : "Forbidden" }
$ curl https://<api-id>.execute-api.<region>.amazonaws.com/v1/welcome \
-H "x-api-key: XXXXXXXXXX"
{ "message" : "Welcome :)" }
https://aws.amazon.com/blogs/compute/accepting-api-keys-as-a-query-string-in-amazon-api-gateway/
How API Gateway handles API keys
- API Gateway supports API keys sent as headers in a request. It does not support API keys sent as a query string parameter. API Gateway only accepts requests over HTTPS, which means that the request is encrypted.
- When sending API keys as query string parameters, there is still a risk that URLs are logged in plaintext by the client sending requests.
API Gateway has two settings to accept API keys:
- Header: The request contains the values as the X-API-Key header. API Gateway then validates the key against a usage plan.
- Authorizer: The authorizer includes the API key as part of the authorization response. Once API Gateway receives the API key as part of the response, it validates it against a usage plan.
Long term considerations
-
This temporary solution enables developers to migrate APIs to API Gateway and maintain query string-based API keys. While this solution does work, it does not follow best practices.
-
In addition to security, there is also a cost factor. Each time the client request contains an API key, the custom authorizer AWS Lambda function will be invoked, increasing the total amount of Lambda invocations you are billed for.
-
Versioning achieved through Terraform
stage
resource. -
https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/api_gateway_stage
-
Manages an API Gateway Stage. A stage is a named reference to a deployment, which can be done via the
aws_api_gateway_deployment
resource.
resource "aws_api_gateway_stage" "production" {
# To avoid issue:
# "CloudWatch Logs role ARN must be set in account settings to enable logging"
depends_on = [
aws_api_gateway_account.main
]
deployment_id = aws_api_gateway_deployment.main.id
rest_api_id = aws_api_gateway_rest_api.main.id
stage_name = "v1"
Limit Exceeded (429 Throthling Error Response)
- Configuration applied for
welcome
endpoint viaFREE
Usage Plan:
# terraform/4-rest-api-gateway-free-plan.tf
quota_settings {
limit = 10 # Maximum number of requests that can be made in a given time period.
offset = 2 # Number of requests subtracted from the given limit in the initial time period.
period = "WEEK" # Time period in which the limit applies. Valid values are "DAY", "WEEK" or "MONTH"
}
- After exceeding weekly limit of 10 requests:
$ curl https://<api-id>.execute-api.<region>.amazonaws.com/v1/welcome \
-H "x-api-key: XXXXXXXXXX"
{"message":"Limit Exceeded"}
- AWS CloudWatch Logs showing error:
Testing is conducted on 3 steps within Github Actions workflow:
- Lambda Functions (Unit testing) - Hello Lambda Function
- API Testing (Integration) - Welcome Lambda Function
- API Testing (Load) - Welcome Lambda Function
Artillery used for load testing and gathering results on different endpoints.
-
https://www.artillery.io/docs/guides/getting-started/installing-artillery
-
https://www.artillery.io/docs/guides/integration-guides/github-actions
-
Artilley Config/Scenarios references: https://www.artillery.io/docs/guides/guides/test-script-reference
-
https://dev.to/brpaz/load-testing-your-applications-with-artillery-4m1p
-
AWS references: https://aws.amazon.com/blogs/compute/load-testing-a-web-applications-serverless-backend/
- Reports present in
ZIP
file withinArtifacts
section, generated by Github Actions workflow:
- Files included within report:
Load testing results for hello
endpoint:
response time
for 95% of requests (p95
parameter) is close to200ms
:
response code
for requests:
-
It's possible to configure Artillery to return a non-zero exit code if the test run doesn't comply with specified conditions based on a set of parameters like error rate, minimum, maximum and percentile based latency or response time.
-
The following configuration will ensure that at least 95% of requests are executed below 50ms (helps to include some buffer for warm up time), otherwise, the command will exit with an error.
config:
ensure:
p95: 50
- This is really useful in a CI environment as you can make the test fail if it doesn't meet your performance requirements.
AWS Reference for 401
errors: https://aws.amazon.com/premiumsupport/knowledge-center/api-gateway-401-error-lambda-authorizer/
If Lambda Event Payload
is set as Token
, then check the Token Source
value. The Token Source
value must be used as the request header
in calls to your API:
-
Deployment can be triggered from
GIT commit messages
by including[deploy]
. -
Deployment can be triggered
manually
using Terraform CLI withinterraform
folder. -
Pre Deployment
linting
andunit_tests
steps triggered through Github Actions. -
Post Deployment
integration_tests
andload_tests
steps triggered through Github Actions.
- Github Actions workflow can be customized here:
# .github/workflows/ci_cd.yml
name: "CI/CD Pipeline"
on:
push:
paths:
- "terraform/**"
- ".github/workflows/**"
branches:
- main
pull_request:
...
End-to-end observability for serverless: https://dashbird.io/serverless-observability/
Error tracking across all serverless services:
AWS Lambda Calculator: https://dashbird.io/lambda-cost-calculator/
Optimizing AWS Lambda functions: https://aws.amazon.com/blogs/compute/optimizing-your-aws-lambda-costs-part-1/
AWS tags are key-value
labels you can assign to AWS resources
that give extra information about them.
Reference: https://engineering.deptagency.com/best-practices-for-terraform-aws-tags
https://docs.aws.amazon.com/tag-editor/latest/userguide/find-resources-to-tag.html
-
https://docs.aws.amazon.com/apigateway/latest/developerguide/http-api-vs-rest.html
-
https://docs.aws.amazon.com/apigateway/latest/developerguide/http-api-develop-routes.html
-
REST APIs and HTTP APIs are both RESTful API products.
-
REST APIs support more features than HTTP APIs, while HTTP APIs are designed with minimal features so that they can be offered at a lower price.
-
Choose REST APIs if you need features such as API keys, per-client throttling, request validation, AWS WAF integration, or private API endpoints.
-
Choose HTTP APIs if you don't need the features included with REST APIs.
- Clone repository.
- Validate Terraform <-> Github Actions <-> AWS integration: https://developer.hashicorp.com/terraform/tutorials/automation/github-actions
- Adjuste
0-providers.tf
file to your own Terraform workspace specifications.
- Create a new branch from
main
. - Create a new
NodeJS
function folder. Runnpm init
&npm install <module>
as you need. - Create a new
Lambda function
throughTerraform
. - Create a new
Terraform Integration
for said Lambda function. - Create
unit
,integration
,load_test
tests for said Lambda function. - AWS Lambda functions can be tested locally using
aws invoke
command (https://docs.aws.amazon.com/lambda/latest/dg/API_Invoke.html). - Apply
linting
best practices to new function file. - Add
unit
,integration
,load_test
steps into Github Actions (ci_cd.yml
) following the same pattern as other lambda functions. - Commit changes in your
feature branch
and create aNew Pull Request
. - Pre Deployment
Github Actions
workflow will be triggered in your new branch:
- Validate
workflow run
results. - Once everything is validated by yourself and/or colleagues, push a new commit (it could be an empty one) with the word
[deploy]
. - This will trigger pre deployment and post deployment steps within the entire github actions workflow:
-
Once everything is validated by yourself and/or colleagues, you can merge your branch into
main
. -
Once Github Actions workflow is successfully completed, a valuable addition is sending a notification with workflow results into Slack channel/s:
# .github/workflows/ci_cd.yml
...
send-notification:
runs-on: [ubuntu-latest]
timeout-minutes: 7200
needs: [linting, unit_tests, deployment, integration_tests, load_tests]
if: ${{ always() }}
steps:
- name: Send Slack Notification
uses: rtCamp/action-slack-notify@v2
if: always()
env:
SLACK_CHANNEL: devops-sample-slack-channel
SLACK_COLOR: ${{ job.status }}
SLACK_ICON: https://avatars.githubusercontent.com/u/54465427?v=4
SLACK_MESSAGE: |
"Lambda Functions (Linting): ${{ needs.linting.outputs.status || 'Not Performed' }}" \
"Lambda Functions (Unit Testing): ${{ needs.unit_tests.outputs.status || 'Not Performed' }}" \
"API Deployment: ${{ needs.deployment.outputs.status }}" \
"API Tests (Integration): ${{ needs.integration_tests.outputs.status || 'Not Performed' }}" \
"API Tests (Load): ${{ needs.load_tests.outputs.status || 'Not Performed' }}"
SLACK_TITLE: CI/CD Pipeline Results
SLACK_USERNAME: Github Actions Bot
SLACK_WEBHOOK: ${{ secrets.SLACK_WEBHOOK }}
CI/CD Slack Notification example:
REST API Gateway integration
files for all Lambda Functions could be refactored within a lambda
module to concentrate shared infrastructure code.
- In the same way
deployment
can be triggered via GIT Commit messages, we can apply a similar behavior to eachlinting
,unit_tests
,integration_tests
and/orload_tests
steps within Github Actions workflows:
linting:
name: "Lambda Functions (Linting)"
if: "contains(github.event.head_commit.message, '[linting]')"
linting:
name: "Lambda Functions (Unit Testing)"
if: "contains(github.event.head_commit.message, '[unit_tests]')"
integration_tests:
name: "API Testing"
if: "contains(github.event.head_commit.message, '[integration_tests]')"
load_tests:
name: "API Load Testing"
if: "contains(github.event.head_commit.message, '[load_tests]')"
-
This will provide developers with more granular control over which types of tests to run as they see fit. E.g.: if a
hot fix
is applied tomain
branch, it might be really useful to just run specific set of tests given time is a priority. -
Also, a default tests list (e.g.:
linting
,unit_tests
,integration_tests
andload_tests
) could be set to run every time a new feature is added tomain
branch.
Once token
is received within Authorizer Lambda function, there are a couple of ways to validate it:
- Call out to OAuth provider
- Decode a JWT token inline
- Lookup in a self-managed DB