Adding Metrics to the Application
Explore how to instrument your Golang application using Prometheus to add metrics for message publishing and handling, track processing durations with histograms, and monitor customer registrations. Understand how to use middleware and application wrappers to integrate metrics without altering core application code, enhancing observability with minimal impact.
We will be using Prometheus to instrument our application to report metrics. Prometheus is quick to set up and just as quick to use.
Setting up the Prometheus endpoint for metrics collection
To begin, we need to set up an endpoint on the HTTP server so that Prometheus can fetch the metrics we will be publishing. Unlike OpenTelemetry, which uses a push model to send data to the collector, Prometheus uses a pull model and will need to be told where to look for metrics data.
To provide Prometheus with an endpoint to fetch the data, we need to import the promhttp package and then add the handler it provides to the HTTP server. We must modify the /internal/system/system.go file to add the endpoint:
Prometheus expects to find metrics at the /metrics path by default, but that can be changed when we configure Prometheus to fetch the data.
The Go client for Prometheus automatically sets up a bunch of metrics for our application. Hitting that endpoint will display a dizzying list of metrics that were set up for us for free. We can also set up custom metrics for our application; to demonstrate, we will start with the messaging system.
When publishing a message, we must use a counter to record a total count and a count for that specific message:
counter := promauto.NewCounterVec(prometheus.CounterOpts{Namespace: serviceName,Name: "sent_messages_count",Help: fmt.Sprintf("The number of messages sent by %s",serviceName,),},[]string{"message"},)
This is setting up a monotonically increasing counter that is broken up into partitions using a message ...