AWS Lambda on Digi Hunch

AWS serverless services and developer tools

Wed, 09 Nov 2022 12:19:00 -0400

As discussed, serverless simply means cloud services that delegate autoscaling management to cloud platform. In my mind, the word “serverless” translates into “managed autoscaling”. As long as a service’s capacity is managed automatically, we can consider it as serverless. Given that capacity scaling accounts for a good amount of work in IT operation, moving to serverless significantly reduces operation overhead.

Unlike containers, the serverless ecosystem lacks standard. Since AWS is leading the charge in this field, let’s take a look at its offerings.

Lambda Function

Lambda functions have triggers. We can configure a trigger from an AWS service, or even a non-AWS service which supports AWS event bridge, to invoke Lambda function. Based on the trigger (e.g. SQS, S3, DynamoDB), you then specify event source mapping. For example, if trigger is SQS, you need to specify the queue name, batch size and batch window in event source mapping. If trigger is S3, the event source mapping needs to specify bucket, S3 action, etc. Event source mappings vary significantly among trigger types.

Depending on the trigger, a Lambda function may be invoked in one of the two ways:

	Synchronous invocation	Asynchronous invocation
Summary	Requestor fires request and waits until it receives response before closing connection.	Lambda places triggering event in a queue and immediately returns a success code (202). Then a separate process reads events off the queue and sends them to your Lambda function. When the function returns a success response or exits without throwing an error, Lambda sends a record of the invocation to an EventBridge event bus.
Pros	Requestor get invocation result as soon as function run is complete	The queue decouples the request and invocation.
Cons	When load is high the function requires higher capacity for concurrency	Involves more parties at play and can be quite complex to troubleshoot.

Here is a chart with the triggers that support each invocation mode.

Real-life applications often need to include libraries and dependencies that requires language-specific steps. For example, NodeJS applications need bundling (often using Webpack or esbuild). We can use Lambda layers to simplify the management. The additional libraries often need to be packaged in S3 bucket or as container image.

Lambda functions are subject to cold start to cope with. When a lot of invocations occur about the same time, the order of executions might be different than the order of upstream events that invokes the function.

API Gateway

AWS API Gateway is an API gateway implementation for REST and WebSocket APIs. It couples with Lambda in the classic multi-tier serverless architecture pattern.

In this classic pattern Lambda function often need to work with a database (e.g. DynamoDB etc), to perform CRUD operations and other custom business logics. The CRUD operations are so commonplace that it makes sense to use mapping template to configure CRUD operation instead of writing similar set of functions for each new data model. API Gateway supports such a mapping template called Velocity Template Language (VTL), a technology from Apache Velocity Project.

API Gateway integrate with many other AWS services. Here are the available integration types:

Integration Type	Description	Integration Mode	How it works
AWS integration	connects gateway to an AWS service action as the end point.	AWS_PROXY	this mode only supports only one action with one service: the function invoking action for Lambda service. Therefore it is available only for Lambda integration and no other AWS services. For that reason, it is also known as Lambda proxy integration. This mode is recommended for Lambda integration and is the default mode for LambdaIntegrationOption CDK construct. It connects a method (PUT, GET, etc) to a Lambda function and pass along the request on the way in, and the response on the way out. You do not set integration request or integration response. Even if you do, there’s no effect. This “pass-along” mode is easier to understand and configure.
AWS integration	connects gateway to an AWS service action as the end point.	AWS	this mode connects an API method to a broad range of supported AWS service action. Function invoking for Lambda service is a common example but not the only service action supported in this mode. When used in Lambda integration, it is also referred to as “Lambda custom integration”, or “normal (request/response mapping) integration”. This mode is good for advanced use cases (e.g. header modification) but involves more management effort. You have to control the mapping between method request and integration request, and between integration response and method response.
HTTP integration	for generic HTTP service endpoint	HTTP_PROXY	the pass-along mode for upstream HTTP endpoint
HTTP integration	for generic HTTP service endpoint	HTTP	the request/response mapping mode for upstream HTTP endpoint
MOCK integration	the API gateway itself serves as the endpoint.	MOCK	the API gateway itself acts endpoint without an upstream. One example use case is to return CORS-related headers upon a pre-flight OPTIONS query.

The table above is a summary of API Gateway integration types as covered here. In the AWS context, proxy mode suggests that the request is not being morphed (transformed). In non-proxy mode, request or response may be modified on their ways in or out.

For API gateway to invoke lambda function, synchronous invocation is used by default. You can also configure API gateway to invoke Lambda function asynchronously by using headers.

As with other API gateway implementations, AWS API Gateway can also connects to an authorizer (either another Lambda function or Cognito service) in order to authorize the incoming request.

AppSync

In a previous post, I discussed GraphQL as a modern and efficient alternative to REST API. AppSync to GraphQL is the same as API Gateway to REST API. One of the advantages that GraphQL has over REST API is more information in the response. Oftentimes, we use a proxy that supports GraphQL in front of REST API service. AppSync can act as such proxy. I think of it as a managed Apollo since they play the the role in the architecture.

Another benefit of GraphQL is the support of subscription, obviating WebSocket configuration. AppSync supports pub/sub API for real-time experience. Client application can get near real-time update as the data on the server is changed.

We can configure AppSync to connect to different data sources to formulate GraphQL response. The data source can be an HTTP endpoint, a Lambda function, a database (Relational or DynamoDB), or OpenSearch.

App Sync

Between data source and the request, AppSync uses resolver to convert GraphQL payload to the underlying protocols and executes if the caller is authorized to invoke it. Resolvers are comprised of request and response mapping templates, which contain transformation and execution logic. AppSync also uses VTL as the mapping template for resolvers.

Messaging Services

Messaging services have three patterns: queues, pub/sub, and event buses. In AWS, the corresponding services are SQS, SNS and EventBridge. Here is a good post about how to choose among them.

Queues are temporary storage to decouple the source and destination systems. The expectation is that the actions in response to the message can be delayed. If that is not the case and the response needs to be immediate, that is by definition an event-driven pattern. A common solution is to have the AWS service invoke Lambda function. If that is not supported, we can use SNS as an intermediary.

SNS is for simple event-driven pattern. In a more complicated event-driven architecture, we often need an event bus to route events in certain ways. We also need to support various event sources by different software providers. Moreover, we want the capability to register our own event schema. These are the scenarios where EventBridge can help.

Developer tools

In application development with serverless stack, the line between application and infrastructure is somewhat blurred. Developers often find themselves making repeated configuration on cloud resources while writing application code. For example, to test Python code (application), developer has to upload the code to S3, create Lambda function referencing the code, etc. This would require too much work with AWS CLI, or CloudFormation. The AWS SAM (AWS Serverless Application Model) is a better utility for serverless development workflow.

SAM comes with its own CLI and developers can feed it with template that interacts with a number of serverless resources (e.g. API, SimpleTable, Function, etc). I view SAM as one layer of abstraction on top of CloudFormation that handles some resources in serverless stack. It saves developers from re-writing resources in CloudFormation templates and keeping them consistent, which would have been tedious. Once you deployed your application using SAM, it will appear as a Lambda Application in the console.

SAM templates are also declarative. It is simple but limited in feature. AWS CDK is also a powerful utility that works with general purpose programming language for IaC and interacts with all AWS resources. It is very powerful for building serverless applications.

AWS Amplify

We have CDK, SAM and CloudFormation but we still have to ensure integration between those resources and our applications. For example, when creating a S3 buckets, they have to get the endpoint and reference it in the application code. So is Cognito. As a result, developers still have to spend time on resource integration. They still can’t focus on business logic. We need a tool that can make opinionated configuration of cloud resources and automatically reference them from the application code. This is the purpose of AWS Amplify.

Amplify not only help application developers create backend resources with opinionated configurations. It also provides libraries for the application to use and connect to those resources seamlessly. Developers will need to include Amplify libraries in the application code. To create cloud resources, they can use Amplify CLI or Studio (a web portal from AWS console).

Amplify natively supports a number of serverless resources such as API (using API gateway or AppSync), Storage (S3 and CloudFront), Lambda function, Cognito, etc. Developer may use Amplify CLI command to create such supported resources. Amplify will prompt some guiding questions in order to configure them correctly. In addition to the natively supported resources, developers can also create custom resources. They have to declare those custom resources with CloudFormation or CDK. I came across this project as a good illustration of how Amplify works.

The Amplify Studio can save developers from using Amplify CLI commands. It also can integrate with Figma, providing developers with a framework for frontend development. However, in my experience, it is still glitchy. For now I stick to Amplify CLI.

Amplify also supports hosting capability, providing users with opinionated and customizable CI/CD pipeline configuration.

How about “clientless”

Lambda function can also run client-side logics. System administrators have to create lots of client-side scripting. While Lambda functions can encapsulate those logics, we’d still need a script orchestrator to invoke those functions. AWS Step Function comes to rescue. It was even regarded as the most under-utilized service. As the author states, a state machine (as design pattern) is simply a flow of actions with decision making logics. AWS Step function helps you groom the logic flow of existing actions with Amazon States Language. There are standard and express workflows. Each step is a state. A state can be of several different types, such as Choice, Task, Succeed, Fail, End, Map, Wait, Parallel. The task can be a Lambda function, and even AWS API calls.

AWS Step function integrate with Lambda functions. It is typically used for patterns with long process and the need to orchestrate the execution of several Lambda functions. For specific use cases, look at these sample projects. For example, a network professional needs to run a lot of connectivity testing, reusing the same Python script but run it from different subnets. We need to create a Lambda function for commands like “nc -vz” then invoke the lambda function from VPC, multiple times from different VPCs. We should use step function to drive this. It works like a Makefile on Linux, without requiring your own computer to run.

AWS Step function also integrates with other AWS services, such as SNS, SQS, Dynamo, Batch, Glue, EMR, EKS, API gateway, event bridge.

Whenever we need to create a script, involving custom actions, or AWS API calls, we should consider using AWS step function to organize the actions. The benefits are: it saves you a laptop or bastion host (“client-less”), many ways to invoke them (not just cron”).

Summary

In this post, we started with some serverless services in AWS. For server-side application, we can use API Gateway to invoke Lambda function. For client-side, we can use step function to invoke Lambda function. We then discussed some tools to speed up application development in serverless pattern.

AWS categorizes both Amplify and AppSync under Front-end Mobile. Amplify does not represent any computing resources in AWS cloud. It is a library and CLI tools to enhance developer experience. AppSync on the other hand is a cloud computing resource acting as GraphQL API for mobile or web application.

Computing services: from PaaS to Serverless

Fri, 21 Oct 2022 19:31:00 -0400

Silicon Valley startups in mid-2000s likely do not run their own IT operations (i.e. renting their own data centre spaces, purchasing their own rack-mounted servers). Since the launch of EC2, AWS has been renting extra computing capacity to those startups, in the IaaS model. The leased infrastructure requires maintenance work, and AWS realized that many of these customers cannot afford specialized database admins, network admins, storage admins, or even server admins. As a result, they created a handful of managed services aiming to cut out admin overhead and let their customer focus on coding. This is how Platform-as-a-service (PaaS) came about.

Let’s take a look at what are exactly operation activities.

Ops activities

IT operation team manages server provisioning, installation of operating system, tuning performance, configuring auto scaling and load balancing, configure networking and storage systems, etc. Networking can be so complex that many infrastructure teams have a dedicated Network Operation Center (NOC), who along with security team, manages key aspects of networking, such as segmentation, router configuration, load balancing, firewall configuration.

For client-server application, the client-side code will make outgoing connections, utilizing the TCP/IP stack on the host through an ephemeral port. The server-side code has to be wrapped as a service. A daemon ensures the process running this service stays up and listens to a TCP port in order to respond to request by invoking the functions. Application team usually assumes these activities.

If database is involved, then the patching, upgrade, replication, data protection are all Ops problems. If storage is involved, then Ops has to manage mass data accumulated over years, the integration between storage and database and applications, performance, replication, etc. Some larger organizations have full-time database administrator and storage administrators.

Then comes container. Containers have their benefits but it increases the operation overhead by an order of magnitude. Running container application at scale warrants its own platform, most likely a Kubernetes platform, to address all of the problems above again at the cluster level. Some organization created platform team to manage container and VM platforms.

It is the Ops, that turns functional code into a running business. It is also the Ops, that becomes a pain point as a startup scales. With IaaS and PaaS models, AWS managed to convince many small businesses to delegate their IT operations to AWS. This is the humble start of cloud computing.

Elastic Beanstalk

At first, I wasn’t too impressed with Elastic Beanstalk, since it abstracts away too many details. However, I later realized that it has been surprisingly popular in the developer community, especially with individual developers and SMBs. It simplifies deployment to the point that their users don’t need to know other AWS services, allowing them to focus on coding application logic.

You configure Applications and Environments (one application may have multiple environments). In the Environment layer, you can specify code platform (e.g. Python 3.8 on 64bit Amazon Linux 2, Java, Go, PHP, Ruby) and even container platform (Docker on EC2 or ECS). Behind the scene, Elastic Beanstalk configures EC2 instances, Elastic Load Balancers, etc on the selected VPC and integrate with logging and monitoring services. In the console, Elastic Beanstalk exposes a list of configurations options (e.g. AMI, instance type). This centralized configuration page is dummied down for those who don’t want to deal with Ops.

The downside of Elastic Beanstalk is it takes away a lot of flexibility. Many developers find Elastic Beanstalk limit their choices of deployment, as their applications scale. Elastic Beanstalk does not suit for applications that demand extensive operation efforts. Its niche market is individual developers and SMB. Few enterprise applications run on Elastic Beanstalk.

Containerization with ECS and EKS

When we containerize an application, we build container images. Then we run these images with container runtimes, is a core feature of container platform. Container platform also provides orchestration engine since we frequently take containers up and down. In addition, container platform provides mechanisms for container networking and storage.

AWS has a couple options for container platform. ECS (Elastic Container Service) came out earlier. It organizes a group of EC2 instances as a cluster. You can manage autoscaling, networking, and persistent storage (EFS, FSx etc) on ECS. EKS (Elastic Kubernetes Service) is the managed Kubernetes service by AWS. Just like AKS, it provides a managed control plane along with computing nodes.

I see ECS as a proprietary and simplified container platform, and Kubernetes as an open-source standard for full-fledged container platform with an entire ecosystem. EKS includes an upstream-certified Kubernetes distribution with a set of tools specific to AWS. Since Kubernetes is the de-facto standard container platform, I prefer EKS by default, unless I can justify the use of ECS. In fact, ECS and Kubernetes have many concepts in common. For example, a “Task” in ECS is equivalent to a Pod in Kubernetes.

Whether it is ECS or EKS, right-sizing the computing node group is always challenging especially when the application traffic load is irregular. AWS Fargate is a technology that provides on-demand, right-sized compute capacities. It works with ECS and EKS. When integrated with EKS, we delegate the node management (e.g. scaling) to Fargate and forget about sizing the node pool.

Using ECS and Fargate involves quite a bit of configurations. To simplify that, we can use App Runner, which builds ECS cluster and uses Fargate to execute the container behind the scenes. App Runner helps client in a way similar to Elastic Beanstalk, but concentrate on Container workload.

Serverless with Lambda and API Gateway

The services above have their limitations when it comes to scaling capability. First, they cannot scale to zero. You still pay for idling resources. Also, it is not easy to find the optimal autoscaling setting. Lambda and API Gateway together solves these challenges. AWS refers to it as serverless, which has since become a buzzword. To understand what it is, let’s examine two concepts:

Function as a Service: service with the ability to execute code on demand. Users only pay for code execution time and do not care where the underlying runtime is
Backend as a Service: service with the ability to listen to a port and respond to web request

Lambda itself is a function as a service. Triggered by events, it only incurs a charge when it’s invoked. It does not stay up and listening to a TCP port for incoming web request, as does a backend service. In order to act as a backend, Lambda needs to pair up with API gateway. In this configuration, API gateway listens to a web request, and it fires an event to trigger the execution of Lambda function. Lambda and API gateway together makes a backend as a service. In AWS, the coupling of API gateway and Lambda function ensures an idle service does not incur computing cost.

Since Lambda supports many types of events as trigger, it is also used in event-driven architecture, either standalone or from a VPC. Under the hood, Lambda runs code in a container (with a quick startup time relative to a VM).

Developers can release Lambda code by uploading zip package to S3 bucket, or just packaging code into container image. For deployment, apart from AWS console and CLI, one can leverage CloudFormation, SAM (serverless application model), or CDK.

Lambda vs Fargate

Both Lambda and Fargate are serverless capabilities, at least from a marketing perspective. Both can be used to back web service but there are differences. They provision computing resource at different granularity.

In a web service, the execution duration of a Lambda function is the response duration to an API request, in terms of seconds. While the server is waiting for a request, there is no usage of the computing resource so you’re not paying for waiting for a request. However, this also creates the delay of cold-start, especially when the code size is large. There are several ways to optimize the cold start (e.g. SnapStart for Java), but none of those can completely get rid of the cold-start delay with a once-after-a-while request. A light GET call could take 5 seconds with cold start.

On the Fargate side, the resource provisioning is based on container lifecycle, instead of request lifecycle. As a result, you’re still paying for wait time, and it is not per-request billing. Since the container remains up, your request is not going to experience the cold-start if it’s been idle for a while. Although, Fargate saves you from the effort to right-sizing the computing nodes for container execution, it is not quite the idea of “scale-to-zero when idle” by itself.

Serverless Architecture

In the white paper AWS Serverless Multi-Tier Architectures with Amazon API Gateway and AWS Lambda, AWS advocates the serverless architecture as a modern alternative to the traditional widely adopted three-tier architecture (presentation, logic and data tiers). In the three tier architecture, the scalability of three tier are managed separately. The modern serverless architecture that AWS whitepaper proposes uses API Gateway and Lambda function in place of Load Balancer and EC2 instances (e.g. in an Auto Scaling Group), as illustrated below:

Both API Gateway and Lambda scale automatically to support the need of application workload. It assumes the role of logic tier in three-tier architecture but requires minimal maintenance work. For presentation tier, AWS has serverless alternatives such as CloudFront, S3. For data tier, AWS has serverless alternatives such as Amazon Aurora for relational database and DynamoDB for NoSQL. However, the “no request, no pay” model for Lambda does not apply to the data tier in serverless architecture.

Whilst this paradigm benefits small shop IT who wants to minimize infrastructure cost, it has downsides. There is no ability for infrastructure optimization. Since you do not manage where the code runs, client may have concerns over security (e.g. multi-tenant runtime). As business grows, keep using Lambda can result in technology lock-in. Also, a less used application usually requires warm-up time. A code start (downloading the code and preparing the environment behind the scene) can take 100ms to over a second.

Conclusion

PaaS attempts to help startups simplify the “grunt work” of IT operation. Serverless takes it even further. The semantics of serverless computing is confusing and the Wikipedia page acknowledges it as a misnomer. The nature of serverless model, is the cloud users delegate server capacity management to cloud platforms. The users don’t need to manage servers, VMs, instances, containers, etc on their own. In a previous post, I discussed the ability to scale to zero, which is just one of the many enabling technologies of serverless. Also, “no request, no pay” is neither an inherent nature of serverless model. Serverless service may involve storage (e.g. data service, S3, Aurora serverless) which incurs storage cost. There is a whitepaper on choosing the right AWS service to deploy your website or web application, with a decision tree.

AWS pioneered serverless with Lambda release in 2014 but competitor follows. In the Azure landscape, there is an entire suite of computing services from virtual machine to serverless (also with a decision tree in documentation). Azure’s counterpart for serverless architecture is Azure Function (released in 2016 for GA) with API Management. As for GCP, the serverless suite includes the event-driven Cloud Function (introduced in 2017) and Knative-based FaaS Cloud Run (introduced in 2019).

There are voices in advocacy of standardization of serverless model, and CNCF had since made minuscule efforts such as CloudEvents. The status quo, unfortunately, is anything but standardized.

Lastly, here is a table that summarizes the pros and cons of each computing service model.

Computing Service Model	Pro	Con
EC2	– Most straightforward and widespread legacy model – Legacy	– Ops tasks can be heavy (e.g. patch and vulnerability management of OS) – Utilization can be low
ECS	– Container orchestration is managed – Convenient to scale – Well integrated with other AWS services	– Limited advanced features – Vendor lock-in
EKS	– Highly scalable and flexible – Advanced, platform-neutral deployment tools available (e.g. Helm, ArgoCD, etc) – Custom configurations (e.g. operators)	– Significant operation overhead – Steep learning curve (especially for teams)
Lambda	– automatically scale – low ops overhead – pay per use	– limited choices of runtime – subject to latency due to cold start; yet warm start incurs cost – not suitable for long running tasks (batch processing jobs etc)

In summary, the Lambda-based serverless model is good for stateless server-side workload with short response time (<15s), tolerance of cold-start, no need for portability across platforms, and no complex package dependency.