It's good to be back!

Last week, several of us from the Etleap team participated in AWS re:Invent in Las Vegas... in person! After a nearly two year pause from large events, the energy and enthusiasm were high. It was a busy week for us, highlighted by Moderna’s Carlos Peralta speaking about their analytics journey with Etleap, multiple new Etleap feature announcements and AWS integrations, and countless (masked) face-to-face conversations about ETL, data lakes, and warehouses.

With serverless at the heart of many of AWS’ announcements, there was a notable focus on simplifying adoption for analytics users. That is music to Etleap’s ears as accelerating time to value and minimizing extensive coding requirements are among our core goals.

Here’s a roundup of the major Etleap activities at AWS re:Invent 2021.

Amazon Redshift Streaming Ingestion

Many customers rely on Etleap to land various data sources into an Amazon Redshift data warehouse. Historically, all Redshift customers would have to ingest data streams into S3 and then copy them into Redshift from there. At re:Invent, AWS announced that this additional step is no longer required. Redshift’s native Streaming Ingestion service now enables ingesting data directly from AWS Kinesis Data Streams into Redshift.

Thanks to our close partnership with AWS, Etleap customers will be able to ingest data into Redshift directly from an Etleap-managed Kinesis stream, as part of the AWS Streaming Ingestion Preview Program. This enables organizations to ingest hundreds of megabytes of data per second and query it at exceptionally low latency - in many cases only 10 seconds after entering the Kinesis stream.

We displayed a live demo dashboard in our booth to illustrate the new capabilities. The dashboard showed the largest stock gains of the day, but rather than querying a static data set, Etleap streamed the live NASDAQ data. Minute-by-minute, visitors could see the underlying data set grew from over 1 billion records and the top gainers update accordingly.

Moderna and Etleap

Moderna’s data engineering leader Carlos Peralta delivered a talk at re:Invent sharing how the biotech firm handled rapid data growth over the past two years. “When COVID hit and we realized our vaccine would lead to company hypergrowth, we knew we had to accelerate building our new analytics platform.”

AWS re:Invent registrants can now watch the full 15-minute recording here.

Key among Moderna’s requirements were:

• Handling many data sources

• Operating in a secure environment, and

• Minimizing data engineering staffing needs.