Master AWS Certified Developer – Associate (DVA-C02) Exam/

...

/

Publishing Custom Metrics and Logs in CloudWatch

Publishing Custom Metrics and Logs in CloudWatch

Learn how to instrument applications using structured logging and custom metrics in CloudWatch.

We'll cover the following...

Let’s say we deploy a Lambda function that processes customer orders. AWS will automatically give us some default metrics, such as invocations, errors, and duration. But what if our business needs to track the number of premium orders processed per hour or log payment gateway failures? That kind of context doesn’t come built-in.

Press + to interact

Instrumentation is adding telemetry into our application logic so that CloudWatch can help us answer meaningful questions about performance, reliability, and behavior under real-world conditions.

Publishing custom metrics in applications

By default, AWS automatically tracks system metrics like CPU usage, disk I/O, and network traffic for EC2. But suppose we want to track application-specific data such as the number of user sign-ups, items added to a cart, or the processing time of a specific business function. In that case, we need to publish custom metrics.

There are three main methods to publish custom metrics. Let’s understand these methods.

Method 1: Using the PutMetricData API

The most straightforward way to publish custom metrics in an application is to use the PutMetricData API via an SDK or CLI. We simply construct a metric object, including Namespace, MetricName, Dimensions, and Value, and send it to CloudWatch.

The code snippet below shows how custom metrics can be published via the PutMetricData API in Python SDK (Boto3):

Press + to interact
import boto3
cloudwatch = boto3.client('cloudwatch')
response = cloudwatch.put_metric_data(
    Namespace='MyApp/Users',
    MetricData=[
        {
            'MetricName': 'SuccessfulSignups',
            'Dimensions': [
                {'Name': 'PlanType', 'Value': 'Premium'}
            ],
            'Value': 1,
            'Unit': 'Count'
        },
    ]
)

This method, however, has one downside. Making frequent API calls can add latency and complexity to the application.

Method 2: Embedded metric format (EMF)

Embedded metric format (EMF) is the modern, recommended approach for applications that also generate logs, especially in serverless and containerized environments. It is a special JSON layout that includes our metric name, value, unit, dimensions, and namespace. In this method, we write the JSON object to our logs, and ...