How to append data in blob storage in Azure using Python
In Azure, you can store your data in various storage options provided by Azure like Blob, Table, CosmosDB, SQL-DB, etc. In this shot, we will learn how to append data to a text file stored in Azure Blob Storage.
“Azure Blob storage is Microsoft’s object storage solution for the cloud. Blob storage is optimized for storing massive amounts of unstructured data. Unstructured data is data that doesn’t adhere to a particular data model or definition, such as text or binary data.” - Azure Blob Storage documentation
Create an Azure Storage account
Let us first create an Azure Storage account. Follow the steps mentioned below:
- Search Storage Accounts in the Azure Portal.
- Click New to create a new storage account.
- Fill in all the details like the resource group, the subscription, the name of your storage account, region, etc.
- Now, after the successful creation of your storage account, open your storage account and click on Access Keys from the left navigation pane to get your storage account credentials. Copy them and save them for later use.
The blob that is appended with data is called an append blob. An append blob is made up of blocks and is optimized for append operations. Each block or data that you want to append in the blob can be of a different size, up to a maximum of 4 MB, and an append blob can include up to 50,000 blocks. Therefore, the maximum size of an append blob is slightly more than 195 GB (4 MB X 50,000 blocks).
Install the required package
Now, we are ready to connect this storage account using Python. Let us first install the required packages. Run the following command:
pip install azure
Code
Now, see the code to connect the Storage account with our Python script.
from azure.storage.blob import AppendBlobServicedef append_data_to_blob(data):service = AppendBlobService(account_name="<STORAGE_ACCOUNT_NAME>",account_key="<STORAGE_ACCOUNT_KEY>")try:service.append_blob_from_text(container_name="<CONTAINER_NAME>", blob_name="<FILE_NAME>", text = data)except:service.create_blob(container_name="<CONTAINER_NAME>", blob_name="<FILE_NAME>")service.append_blob_from_text(container_name="<CONTAINER_NAME>", blob_name="<FILE_NAME>", text = data)print('Data Appended to Blob Successfully.')append_data_to_blob('Hello Azure Blob Storage!')
Explanation
- In line 1, we import the required package.
- In line 3, we create a function
append_data_to_blob()that accepts the data we need to append to our blob. - In line 4, we create a connection to our storage account in Azure. Note that we pass the
<STORAGE_ACCOUNT_NAME>and<STORAGE_ACCOUNT_KEY>, which we have already saved after the creation of our storage account. - In line 7, we try to append the
datato a file<FILE_NAME>that is present inside our blob container<CONTAINER_NAME>. However, we haven’t created any containers yet; so, we need to keep this statement in atry-exceptblock. - In line 9, we create a container by using the
create_blob()function and passing the<CONTAINER_NAME>and<FILE_NAME>. You can specify any file name. For example, if you want to append yourdatato thelog.txtfile; then, you need to replace<FILE_NAME>withlog.txt. The<CONTAINER_NAME>could be any name unique to your storage account. - Then, in line 10, we append the
datato the file<FILE_NAME>. - In line 11, we print the statement.
- Finally, in line 13, we call our function and pass the
data.
So, in this way, we can append data generated at regular intervals and store them in the blob storage.
RELATED TAGS
CONTRIBUTOR
Harsh Jain