Part 1: Building a Serverless Backend with AWS Lambda and DocumentDB

One of the most exciting advancements in web development is the advent of serverless architecture, which allows developers to build and run applications without managing servers. This article will focus on building a serverless backend using AWS Lambda and DocumentDB. We'll create a Virtual Private Cloud (VPC), VPC Endpoints, and a DocumentDB cluster and then configure a Lambda function to handle CRUD operations.

Please follow along with all the code used in the README here in my GitHub repository.

AWS Lambda and DocumentDB

AWS Lambda is a serverless computing service that runs your code in response to events and automatically manages the underlying compute resources for you. With AWS Lambda, you can run your applications virtually maintenance-free.

AWS DocumentDB (with MongoDB compatibility) is a fast, scalable, highly available, and fully managed document database service that supports MongoDB workloads. As a document database, AWS DocumentDB makes it easy to store, query, and index JSON data.

Setting up the VPC and VPC Endpoints

A VPC (Virtual Private Cloud) is a virtual network dedicated to your AWS account. It's logically isolated from other virtual networks in the AWS Cloud, providing a secure environment for your resources. VPC Endpoints allow private connectivity between your VPC and supported AWS services.

We are building 100% of our infrastructure with Infrastructure as Code (IaC) using Terraform. We will also deploy both our front end and back end with Terraform.

module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "~> 3.14.0"

  enable_ipv6                     = true
  assign_ipv6_address_on_creation = true
  database_subnet_ipv6_prefixes   = [6, 7, 8]
  private_subnet_ipv6_prefixes    = [3, 4, 5]
  public_subnet_ipv6_prefixes     = [0, 1, 2]

  azs                                             = local.availability_zones
  cidr                                            = local.vpc_cidr
  create_database_subnet_group                    = false
  create_flow_log_cloudwatch_iam_role             = true
  create_flow_log_cloudwatch_log_group            = true
  database_subnets                                = local.database_subnets
  enable_dhcp_options                             = true
  enable_dns_hostnames                            = true
  enable_dns_support                              = true
  enable_flow_log                                 = true
  enable_nat_gateway                              = true
  flow_log_cloudwatch_log_group_retention_in_days = 7
  flow_log_max_aggregation_interval               = 60
  name                                            = var.environment
  one_nat_gateway_per_az                          = false
  private_subnet_suffix                           = "private"
  private_subnets                                 = local.private_subnets
  public_subnets                                  = local.public_subnets
  single_nat_gateway                              = true
  tags                                            = var.tags
}        

Building a DocumentDB Cluster

For this POC, we are using a single instance. You would want to have this spread across multiple AZs for a well-architected architecture.

resource "aws_docdb_cluster" "documentdb" {
  apply_immediately               = true
  backup_retention_period         = 1
  cluster_identifier              = "${local.environment}-cluster"
  db_cluster_parameter_group_name = aws_docdb_cluster_parameter_group.documentdb.name
  db_subnet_group_name            = aws_docdb_subnet_group.documentdb.name
  deletion_protection             = false
  enabled_cloudwatch_logs_exports = ["profiler"]
  master_password                 = random_string.password.result
  master_username                 = "administrator"
  preferred_backup_window         = "07:00-09:00"
  preferred_maintenance_window    = "Mon:22:00-Mon:23:00"
  skip_final_snapshot             = true
  storage_encrypted               = true
  tags                            = var.tags
  vpc_security_group_ids          = [aws_security_group.documentdb.id]
}

resource "aws_docdb_cluster_instance" "documentdb" {
  apply_immediately          = true
  auto_minor_version_upgrade = true
  cluster_identifier         = aws_docdb_cluster.documentdb.id
  identifier                 = aws_docdb_cluster.documentdb.cluster_identifier
  instance_class             = "db.t4g.medium"
  tags                       = var.tags

  depends_on = [aws_docdb_cluster.documentdb]
}

resource "aws_docdb_cluster_parameter_group" "documentdb" {
  name        = local.environment
  description = "${local.environment} DocumentDB cluster parameter group"
  family      = "docdb4.0"

  tags = var.tags
}

resource "aws_docdb_subnet_group" "documentdb" {
  name        = "documentdb_${local.environment}"
  description = "Allowed subnets for DocumentDB cluster instances"
  subnet_ids  = module.vpc.database_subnets
  tags        = var.tags
}

resource "aws_secretsmanager_secret" "documentdb" {
  name                    = "${local.environment}-credentials-${random_string.unique.result}"
  description             = "${local.environment} DocumentDB credentials"
  recovery_window_in_days = "7"
  tags                    = var.tags

  depends_on = [aws_docdb_cluster.documentdb]
}

resource "aws_secretsmanager_secret_version" "password" {
  secret_id = aws_secretsmanager_secret.documentdb.id
  secret_string = jsonencode(
    {
      username = aws_docdb_cluster.documentdb.master_username
      password = aws_docdb_cluster.documentdb.master_password
    }
  )

  lifecycle {
    ignore_changes = [secret_string]
  }

  depends_on = [aws_secretsmanager_secret.documentdb]
}        

Configuring the Lambda Function for CRUD operations

Build the Lambda with Python 3.9 inside the VPC (so it can talk to DocumentDB.)

resource "aws_lambda_function" "backend" {
  description   = "Backend Lambda function to communicate between the API gateway and DocumentDB for the ${var.environment} environment."
  filename      = "backend.zip"
  function_name = "${var.environment}_backend"
  handler       = "backend.handler"
  role          = aws_iam_role.lambda_execution_role.arn
  runtime       = "python3.9"
  timeout       = 30

  vpc_config {
    subnet_ids         = module.vpc.private_subnets
    security_group_ids = [aws_security_group.backend.id]
  }

  environment {
    variables = {
      DOCDB_HOST  = aws_docdb_cluster.documentdb.endpoint
      SECRET_NAME = aws_secretsmanager_secret.documentdb.name
    }
  }

  source_code_hash = data.archive_file.backend.output_base64sha256

  tags = var.tags
}        

We'll write Python code to handle CRUD (Create, Read, Update, Delete) operations.

1. Create: Our function should be able to accept a JSON object and insert it into our DocumentDB collection.
2. Read: It should be able to fetch documents based on provided criteria.
3. Update: It should be able to modify an existing document.
4. Delete: Finally, it should be able to delete a document.

Each operation corresponds to an HTTP method: POST for create, GET for read, PUT for update, and DELETE for delete. So that you know – the specific function implementation will depend on your application requirements.

import os
import json
import boto3
from pymongo import MongoClient
from bson.json_util import dumps
from bson import ObjectId

# DocumentDB configuration
HOST = os.environ['DOCDB_HOST']
PORT = 27017
DB_NAME = os.environ.get('DB_NAME', 'blog')

# Create a Secrets Manager client
session = boto3.session.Session()
client = session.client(service_name='secretsmanager', region_name=os.environ["AWS_REGION"])

def get_secret():
    try:
        get_secret_value_response = client.get_secret_value(SecretId=os.environ['SECRET_NAME'])
    except Exception as e:
        raise Exception("Error while retrieving the secret: " + str(e))
    else:
        if 'SecretString' in get_secret_value_response:
            secret = get_secret_value_response['SecretString']
            return json.loads(secret)
        else:
            raise Exception("Could not retrieve the secret string")

secret = get_secret()
USERNAME = secret['username']
PASSWORD = secret['password']

# Create a MongoDB client
client = MongoClient(f'mongodb://{USERNAME}:{PASSWORD}@{HOST}:{PORT}/?ssl=true&retryWrites=false')
db = client[DB_NAME]

def handler(event, context):
    operation = event['httpMethod']

    path_parameters = event.get('pathParameters')
    if path_parameters:
        proxy = path_parameters.get('proxy', '')
        proxy_parts = proxy.split('/')
        if len(proxy_parts) > 1:
            event['id'] = proxy_parts[1]

    if operation == 'GET':
        return handle_get(event)
    elif operation == 'POST':
        return handle_post(event)
    elif operation == 'PUT':
        return handle_put(event)
    elif operation == 'DELETE':
        return handle_delete(event)
    else:
        return {
            'statusCode': 400,
            'headers': {
                'Access-Control-Allow-Origin': '*',
                'Access-Control-Allow-Credentials': 'true'
            },
            'body': 'Invalid operation'
        }

def handle_get(event):
    # If an ID is provided, retrieve just that post
    path_parameters = event.get('pathParameters')
    if path_parameters and 'id' in path_parameters:
        post_id = path_parameters['id']
        result = db.posts.find_one({'_id': post_id})

        return {
            'statusCode': 200,
            'headers': {
                'Access-Control-Allow-Origin': '*',
                'Access-Control-Allow-Credentials': 'true'
            },
            'body': dumps(result)
        }
    else:
        # Otherwise, retrieve all posts
        result = db.posts.find()

        return {
            'statusCode': 200,
            'headers': {
                'Access-Control-Allow-Origin': '*',
                'Access-Control-Allow-Credentials': 'true'
            },
            'body': dumps(list(result))
        }


def handle_post(event):
    body = json.loads(event['body'])
    result = db.posts.insert_one(body)

    return {
        'statusCode': 201,
        'headers': {
            'Access-Control-Allow-Origin': '*',
            'Access-Control-Allow-Credentials': 'true'
        },
        'body': dumps({'_id': result.inserted_id})
    }

def handle_put(event):
    post_id = event['id']
    body = json.loads(event['body'])

    result = db.posts.update_one({'_id': ObjectId(post_id)}, {'$set': body})

    return {
        'statusCode': 200,
        'headers': {
            'Access-Control-Allow-Origin': '*',
            'Access-Control-Allow-Credentials': 'true'
        },
        'body': dumps({'matched_count': result.matched_count, 'modified_count': result.modified_count})
    }

def handle_delete(event):
    path_parameters = event.get('pathParameters')
    if path_parameters:
        proxy = path_parameters.get('proxy', '')
        proxy_parts = proxy.split('/')
        if len(proxy_parts) > 1:
            post_id = proxy_parts[1]
        else:
            post_id = path_parameters.get('id', None)
    else:
        post_id = None

    if post_id is None:
        return {
            'statusCode': 400,
            'headers': {
                'Access-Control-Allow-Origin': '*',
                'Access-Control-Allow-Credentials': 'true'
            },
            'body': 'No ID provided'
        }

    result = db.posts.delete_one({'_id': ObjectId(post_id)})

    return {
        'statusCode': 200,
        'headers': {
            'Access-Control-Allow-Origin': '*',
            'Access-Control-Allow-Credentials': 'true'
        },
        'body': dumps({'deleted_count': result.deleted_count})
    }        

It's important to note a few things about this code:

1. One of the libraries will need to be installed locally and zipped with the Python code for this to work correctly. Please have a look at the README for more information.
2. Lambda has permissions and grabs the credentials dynamically from the AWS Secrets manager, so the credentials are never in plain text.

Conclusion

Building a serverless backend using AWS Lambda and DocumentDB provides a scalable, maintenance-free solution for your applications. This serverless approach allows you to focus on writing code that delivers value to your users rather than spending time and resources on server management.

This tutorial has covered the basics of creating a VPC, setting up a DocumentDB cluster, and configuring a Lambda function for CRUD operations. Depending on what you need, you may need to adjust certain steps. However, the principles remain the same: leveraging serverless technologies to deliver efficient, scalable, and reliable applications.

Stay tuned for deploying the frontend!

Visit my website here.