With over two years experience developing software on serverless cloud platforms — much of that time spent continuously deploying changes to production — now seems as good a time as any to share some tips and tricks when it comes to maintaining serverless software in production.
Working exclusively on Amazon Web Services (AWS), this experience spans from Lambda functions, to Step Functions, to Aurora Serverless relational DB clusters, and more! There’s plenty lessons learned which cover all these technologies.
Starting with throughput. “Serverless is more!”, I hear you cheer. It certainly is. The ease with which serverless technology scales is…
Cloud computing involves a myriad of tools to build your software and provision hardware. These tools ultimately depend on the programming languages chosen and the underlying technology adopted. With Amazon Web Services (AWS), you may find yourself using the AWS Command Line Interface (CLI) or the AWS Serverless Application Model (SAM) CLI in addition to traditional apparatus like Bash and Node.js.
With any large scale software project, there are multiple avenues to consider when it comes to deployment: whether to ship your new software all at once, roll out the project in phases, or release a canary into the coal mines.
With all-at-once delivery, your new product will be presented to users in a single, “big bang” release. This would be the quickest approach, but yields large room for error which may come back to bite the development team further down the line. …
A recent addition to API Gateway, HTTP APIs provide a new avenue to define RESTful APIs in AWS. Similar to their REST API counterpart, HTTP APIs integrate seamlessly with HTTP endpoints and Lambda functions to serve requests, whilst aiming to operate at lower cost with lower latency.
Working with the latest version of AWS Cloud Development Kit (CDK) — 1.42.0 at the time of writing — let’s run through a short demo of HTTP API. We’ll explore how to set up the API, configure paths, and handle requests sent to the API for processing.
CDK constructs for HTTP API remain…
Since their first launch in November 2016, Step Functions have proved to be a valuable resource for the orchestration of both simple and complex processes in AWS. They offer an abundance of great features — supporting synchronous tasks, asynchronous callbacks, and activities on EC2 and ECS instances.
Permitting over 2,000 execution starts per second, Step Functions sufficed for the majority of use cases. However, you may have wondered, how could we manage workflows for greater volumes of data? Regardless of our approach to this, how could we minimise costs if we’re processing so many events?
Ranging from hand written number recognition, to facial identification, to speech analysis, machine learning proves invaluable across an abundance of fields. With a growing community joining Amazon Web Services, the demand for classification algorithms in the cloud continues to grow. So what does AWS offer that we can apply to our applications?
Announced early in 2019, Amazon DocumentDB delivers MongoDB compatibility to Amazon Web Services (AWS). Arriving only a few months after MongoDB challenged cloud providers with a new open-source license, DocumentDB is built from the ground up as a document database service, taking advantage of MongoDB’s open-source functionality.
Starting with a clean slate on the AWS console and a fresh stack initialised by the AWS Cloud Development Kit (CDK), let’s provision and configure a DocumentDB instance both by hand and programmatically. An AWS account is required (free tier available). Beginning with the console, let’s visit the DocumentDB page.
Presented with a…
When you define your infrastructure as code, you’ll often want to build and deploy your service with a pipeline — this forms the foundations for Continuous Integration and / or Continuous Delivery (CI / CD) further down the line. If you’re looking for a tool to achieve precisely that, then look no further, for declarative Jenkins pipelines will serve you well. Released in February 2017 as part of the Jenkins Pipeline Plugin, declarative pipelines are designed to automate the build and deployment process.
Working with a fresh AWS Cloud Development Kit (CDK) project, let’s implement a declarative Jenkins pipeline that…
Working with Amazon Web Services, Step Functions prove an invaluable resource to define your system’s workflows. If you find yourself working with asynchronous steps in these flows, you may be wondering how best to integrate them into your Step Functions, especially if these steps take hours, days, or even months to finish.
Fear not, for AWS announced callbacks for Step Functions in late May 2019. Offered at no extra cost, callbacks make it easier to define asynchronous tasks and configure the Step Functions to wait for these tasks to finish before proceeding to the next step.
Using the AWS Cloud…
Within a Kinesis data stream we have a collection of shards. Records put into the stream are allocated a shard based on their partition key, configured by the user. Each of these shards supports read throughput of 2MB per second, which all stream consumers have to share.
Announced in August 2018, Enhanced Fan-Out aims to improve this — delivering a dedicated throughput of 2MB per second per shard to each registered Kinesis consumer. When enabled, records are pushed immediately to the consumer.
Software engineer designing and building AWS cloud microservices. Working primarily with Java, Python, and TypeScript.