Building and supporting infrastructure in a large organization can be challenging due to the automation of all deployment steps (e.g., creating cloud environments for a department with a set of virtual machines that contain preinstalled software, utilities, and services that may have their own dependencies like database and storage, and connection to log analytics and other log services). Utilities and services may have external dependencies as well; therefore, you need to have a mechanism to check and ensure that required connections are established, dependencies are installed, and proper automation runbook is triggered. The mechanism should alert the administrator of error and perform a backup.
To build this mechanism, I will use the following Azure resources:
Event Grid is a key component that acts as the main event processor and contains:
The relation between publisher components, events, topic, and subscriber/endpoint is shown in the diagram below.
Event grid contains:
Queue Storage uses as the main event storage. When the event is generated via Event Grid, the final destination will be a queue.
Functions are used as microservices that contain logic to validate required resources and connections. Each function may also contain a database for storing configuration or state management data.
Now, after a rather high-level architecture description, I will provide more details below.
As you have noticed, an event grid is linked with a subscription listening on events related to new virtual machine creation. It is necessary to add filtering here; otherwise, the event grid will generate messages whenever any resource is created in a subscription or the target resource group. All events delivered are delivered in the storage queue. In my project, I’m using three queues:
A storage account is also linked to Azure Log Analytics to synchronize all logs and alerts. For example, log analytics will log this as an error and alert administrator if there are more messages in the retry queue than expected.
The Azure function app contains several Azure functions with validation, message processing, and logic to trigger the runbook.
The validation function is linked with the main event queue. When the event grid sends a message to the main queue, the function is automatically triggered.
Retry Function is based on a timer trigger and will be run constantly to check for failed messages intended to retry (in retry event queue).
API Function(HTTP trigger) is intended to trigger (re-run) the whole process from runbook, Admin UI, etc.
The whole functions workflow is described below:
The function app is written on PowerShell (Powershell Class) and uses the principles of OOP. This approach allows us to build modular, well-supported code and add or remove functions at any time.
In this article, I described how to build event-driven architecture to manage the virtual machine, related utilities, and components.
You can reuse the presented solution in the following scenarios:
This topic is quite useful and popular in the world of cloud and software architecture; moreover, these approaches solve a lot of business problems. With this in mind, I’ve decided to create a practical and interactive course. It is based on my experience building EDA architecture for enterprise customers.
You can play with the code, test architecture, and run pipelines directly in the course! Click the link below to enroll in the course:
View all Courses