CLOUD LABS
Building a Logs Processing Pipeline with Amazon Kinesis
In this Cloud Lab, you’ll learn to create a data processing pipeline using Kinesis Data Streams and Lambda, which processes the log files and logs the results in the CloudWatch.
beginner
Certificate of Completion
Learning Objectives
In this Cloud Lab, you’ll learn to collect data from a log file using the Kinesis agent installed on an EC2 instance. The Kinesis agent sends this data to a Kinesis stream, which then triggers a Lambda function. The Lambda function processes the Kinesis stream and logs the event in CloudWatch. This technology is widely used in modern cloud-based systems to manage large amounts of data and process it in real-time.
This Cloud Lab will guide you on setting up and configuring the Kinesis agent on an EC2 instance. You’ll then create a Kinesis stream and set up a Lambda function to process the stream data. Moreover, you’ll learn to use CloudWatch to monitor the events processed by the Lambda function.
By the end of the Cloud Lab, you’ll have a working serverless data processing pipeline that can collect and store large amounts of data in real-time, have hands-on experience with AWS services like Kinesis data stream, Kinesis agent, and Lambda, and will be able to apply this knowledge to build similar data processing pipelines in your own projects. This Cloud Lab will equip you with the skills and knowledge needed to design and implement serverless data processing pipelines, advancing your career as a cloud developer.
The following is the high-level architecture diagram of the infrastructure you’ll create in this Cloud Lab:
Trusted by 1.4 million developers working at companies including
Your method is simple, straight to the point and I can practice with it everywhere, even from my phone, that's something I have never had in other learning platforms.
I highly recommend Educative. The courses are well organized and easy to understand.
I prefer Educative courses because they have a nice mix of text & images. I find that with full video courses, it can often be too easy to go into passive learning mode.
I prefer Educative courses because they have a nice mix of text & images. I find that with full video courses, it can often be too easy to go into passive learning mode.
Your method is simple, straight to the point and I can practice with it everywhere, even from my phone, that's something I have never had in other learning platforms.
I highly recommend Educative. The courses are well organized and easy to understand.
I prefer Educative courses because they have a nice mix of text & images. I find that with full video courses, it can often be too easy to go into passive learning mode.
Get access to Educative Cloud Labs
