Measuring our Pipeline
The ultimate question of running a software is, “How do I know if it’s working?” Our data pipeline is no different. We need to be able to answer questions about the health of our pipeline so that we can fix any problems that occur. We’ll achieve this by adding measurements for how long our
Worker takes to process and how long it takes to broadcast our message.
We can use our stats knowledge to capture a timing event for our
Worker process. We’ll trigger a manual timing event to measure the time between item generation and push delivery. Let’s jump right into our
Replace our current
Worker.start_link function with this new timed one: