Sign up for our newsletter and get the latest big data news and analysis.

The Future Outlook of Serverless

In this special guest feature, Emrah Samdan, Vice President of Products for Thundra, takes a look at the future outlook of serverless technology. Thundra is a tool to provide serverless observability for AWS Lambda environments. Emrah is obsessed with helping the serverless community with their debugging and monitoring effort both in production and during development.

A new decade brings new challenges and opportunities, and although 2020 has been filled with a unique set of challenges, it’s also provided an unexpected amount of extra time. Let’s take advantage of this opportunity to dive deep in the past, present and future of serverless.

Serverless Emerges, Increases Agility

I recently had the great honor of sitting down with the “father of serverless”, Tim Wagner and the Chief Architect of DaySmart, Tom Kowalski to discuss the creation of AWS Lambda and how serverless began.

In its simplest form, serverless enables agile development of applications, which allows developers to shift their attention to scaling products instead using their time to manage and operate servers or runtime in the cloud or on-premises.

AWS Lambda launched over five years ago and instantly won the developer world’s attention. Wagner explains that AWS Lambda was not born out of EC2 to make computing more flexible with scaling from zero to infinity, like many thought. Instead, the idea of event-driven computing units was actually developed from the S3 team and its customers. After several long interviews with hundreds of customers, Wagner and the AWS team discovered what S3 was lacking and that’s how Lambda was created.

In the original development of Lambda, the AWS team did not envision serverless as a new computing paradigm. Initially, the team thought the functions would be only for simple jobs that should only take a minute or less with limited memory usage. Lambda 1.0 (if you will) was not built for full-fledged backend applications or machine learning applications. Wagner admits that he’s fascinated by all the use cases that can be done through Lambda functions.

The AWS team originally believed that users would adopt developing, testing, deploying and maintaining functions easily. Unfortunately, that was not the case and it impacted the adoption of serverless for the first several years. Wagner explained that the serverless community pitched in by creating tools like Stackery and Thundra to make the lives of application teams easier, and to make serverless enterprise-friendly.

The AWS team was confident in the efficacy of introducing almost all of the AWS services as an event source to Lambda — even when they only had S3, DynamoDB and Kinesis as a trigger when AWS Lambda was launched. By stepping back and observing the many variations of firing a Lambda function now and the ways events can be exchanged with Eventbridge, we can see how accurately they executed their vision throughout the years.

The State of Serverless in 2020

Kowalski shares his experience from DaySmart on how serverless has evolved in the last five years and what is underway in 2020. His team now spends a minimal amount of time spinning up the production architecture to onboard a new engineer, to do a PoC on a new idea, or to try new features that have been rolled out by AWS. This agility and flexibility with managed services has allowed DaySmart to focus on what really matters to them: the value that they provide to their users. 

In addition to agility, Wagner notes that the result of leaning on the serverless paradigm has led to VCs’ surprise of how mature and operable the Vendia product is with a small team. Today’s serverless also enables companies to re-shape their capital expenses for talent, they can now afford to invest in more reliable, resilient and scalable applications. This in turn means the companies are focused on added talent that can bring more value to the application.

At this stage in serverless, there’s still room for improvement. For instance there is not a clear path to migrating existing applications as they are or with minimal modifications to serverless.

The Future Outlook for Serverless

Due to the pandemic, it’s unlikely that we’ll have the re:Invent this year, but this won’t stop the developer world from asking for new features. Wagner and Kowalski shared their AWS serverless wish list:

  • Stateful serverless: Currently, Kowalski uses DynamoDB to manage the state when it’s required. This adds a layer of complexity to the addition of another product to the apps. By sharing the state across Lambda invocations and running custom business logic between them, simplicity and speed would be underway in DaySmart applications.
  • More granular cost structure: Wagner believes that 1ms granularity of cost will boost adoption and generate even more success stories with serverless from a cost perspective.
  • gRPC support: API Gateway and EventBridge use allows AWS to interact in different ways with the outside world, but there are still numerous use cases where the support for a common remote procedure is handled by gRPC.
  • EFS attached to Lambda: DynamoDB, S3, etc. are great with datastore systems, but there are use cases where we need to work with a real file system structure. For this reason, we need to have the possibility of attaching EFS to a Lambda function. AWS releases updates faster than our wishes. They just released the EFS support for AWS Lambda last month.
  • More lifecycle hooks for the container: More lifecycle hooks would open up endless possibilities, including the ability to run and align with Lambda functions in a more secure, efficient, and fast way would be ideal.

The future of serverless and the production readiness of serverless for many use cases will continue to improve and the potential to cover many others is arriving. The industry expects serverless to be the default computing platform by 2025.

Sign up for the free insideBIGDATA newsletter.

Leave a Comment

*

Resource Links: