API Logging
Explore how to enable and configure API Gateway logging integrated with AWS CloudWatch to collect detailed logs for serverless APIs. Understand the importance of balanced logging to control costs and how to create the necessary IAM roles. Gain practical knowledge of deploying an API with logging and viewing its logs in CloudWatch.
We'll cover the following...
Importance of logging
Comments and logs are a critical part of any application development. Unfortunately, most coders feel it’s an unnecessary overhead. Still, to help whoever maintains the code, you should invest every effort to ensure that the comments are accurate and intuitive.
That’s sufficient when we’re working on monoliths. When we come to the cloud and the serverless world, though, we need a lot more. For example, application logs get scattered when we have multiple independent API requests. It takes a rock-solid framework to collate the logs for each of these into one single repository.
No marks for guessing. We’re talking about the AWS CloudWatch service, the single service that manages all logs for all services in AWS. The API Gateway can connect with CloudWatch to enable detailed records of every API invocation.
Much caution is necessary here. Logging is essential for any application, but we must remember that too many logs can be an unwanted load on the system. When we work on the cloud, such a load translates into high costs because we have to pay for each byte of data we store.
We must ensure that we use the optimal log settings. As we see, API Gateway allows granular control on what should log and what shouldn’t. We should use this feature with due consideration and evaluate the information we require in the logs.
Now, let’s look at how we can configure the API Gateway for logging.
Connecting to CloudWatch
Before we start using API Gateway logs, we need some initial configuration. First, open the IAM console in another tab to create a new role for the API Gateway.
Note: You must give
ApiGatewayLoggingRolename to the CloudWatch role. This role is required in some of the next lessons, if you find an error during stack creation that refers to the CloudWatch log role, make sure to create this role and provide its ARN in the API's settings as shown below.
Note: Please save the ARN of the recently created role.
Create this role for the API Gateway using the steps above and then navigate to the API Gateway settings page. Here, we have to provide the IAM role that the API Gateway can use to add logs to the CloudWatch.
With this in place, we can create an API that can add logs to the API Gateway.
Example
Let’s continue with our old Echo API that invokes a Lambda function that echoes the input as it is. We’ll make changes to the API to create logs as well.
/**
* This code is deployed as the Lambda function in AWS.
* It just returns the input event as is
*/
export const handler = async (event, context) => {
return event;
};Click "Run" to deploy and test the API. As the script runs, it deploys the new API and tests it with a simple request. Once the script completes successfully, open the CloudWatch console to view the API logs.
We can also see the detailed access logs for the API.
Understanding the code
Let’s explore the code above. We’ve seen the familiar code of the Echo API several times before. This time, we added one additional resource to the template, namely the deployment.
Check lines 30–35 of the template.yml file. They define the log settings for the API. We provide the ARN of the CloudWatch logs group where we want the logs. We can also specify the format of the logs and the extent of logging required (Info or Error, for example).