Event Source for Azure Blob Storage

This event source subscribes to blob events from an Azure Storage Account through an Event Grid subscription. Events are consumed from a dedicated Event Hubs instance, which is used as event destination in this setup.

Prerequisite(s)

• Storage Account
• Service Principal
• Event Hubs Namespace
• Event Hubs Instance (optional)

Storage Account

If you didn't already do so, create a Storage Account of one of the following supported types: General-purpose V2, BlockBlobStorage or BlobStorage. Take note of its resource ID, it is a required input to be able to run an instance of the Azure Blob Storage event source.

A resource ID for a Storage Account has the following format:

/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Storage/storageAccounts/{storageAccountName}

Resource IDs can also be obtained using the Azure CLI (az). The following command line uses values from the screenshots above:

$ az storage account show --resource-group blobstorage-source-dev --name eventsourcedev
{
  "id": "/subscriptions/15537daf-e607-4df8-b2ef-277248b205b3/resourceGroups/blobstorage-source-dev/providers/Microsoft.Storage/storageAccounts/eventsourcedev",
  "resourceGroup": "blobstorage-source-dev",
  "type": "Microsoft.Storage/storageAccounts",
  "kind": "BlobStorage",
  "name": "eventsourcedev",
  "location": "eastus",
  "provisioningState": "Succeeded",
  ...
}

Service Principal

A Service Principal is required in order to authenticate the event source against the Azure tenant that has authority over the Azure Subscription to monitor. You can create a Service Principal by following the instructions at How to: Use the portal to create an Azure AD application and service principal that can access resources.

The section called Assign a role to the application describes how to assign permissions to the Service Principal. Make sure you select a role which has at least the following permissions:

• Microsoft.EventGrid/eventSubscriptions/read
• Microsoft.EventGrid/eventSubscriptions/write
• Microsoft.EventGrid/eventSubscriptions/delete
• Microsoft.EventHub/namespaces/eventhubs/write

The following set of permissions is also required if you decide to delegate the management of the Event Hub to the event source. In case you prefer to use your own Event Hub, these can be safely be omitted. More details on that topic are provided in the Event Hubs Instance section below.

• Microsoft.EventHub/namespaces/eventhubs/read (optional)
• Microsoft.EventHub/namespaces/eventhubs/delete (optional)

Additionally, assign the built-in role Azure Event Hubs Data Receiver to the Service Principal to allow it to receive events from an Event Hubs instance.

In the example below, we create a custom IAM role that is dedicated to the TriggerMesh event source for Azure Blob Storage:

The corresponding role JSON is given as a reference which you can replicate to create a similar custom IAM role:

{
    "properties": {
        "roleName": "TriggerMesh Event Grid subscriptions manager",
        "description": "Allows the usage of TriggerMesh event sources for Azure Blob Storage.",
        "assignableScopes": [
            "/subscriptions/15537daf-e607-4df8-b2ef-277248b205b3"
        ],
        "permissions": [
            {
                "actions": [
                    "Microsoft.EventGrid/eventSubscriptions/read",
                    "Microsoft.EventGrid/eventSubscriptions/write",
                    "Microsoft.EventGrid/eventSubscriptions/delete",
                    "Microsoft.EventHub/namespaces/eventhubs/read",
                    "Microsoft.EventHub/namespaces/eventhubs/write",
                    "Microsoft.EventHub/namespaces/eventhubs/delete"
                ],
                "notActions": [],
                "dataActions": [],
                "notDataActions": []
            }
        ]
    }
}

After the Service Principal is created and assigned suitable roles, take note of the following information:

• Tenant ID and Client ID (see Get tenant and app ID values for signing in)
• Client secret (see Create a new application secret)

Event Hubs Namespace

Follow the instructions at Quickstart: Create an Event Hub using Azure portal, and create a new Event Hubs namespace. This namespace will contain an Event Hubs instance which will be configured by the event source as the destination of events originating from the Azure Storage Account.

Event Hubs Instance (optional)

This section can be skipped if you would like to let the event source manage its own Event Hub. In this case, please ensure you granted all necessary permissions to the Service Principal in the previous section.

If, however, you prefer to provide your own Event Hub for that purpose, follow the instructions at Quickstart: Create an Event Hub using Azure portal to create an Event Hubs instance. Take note of its resource ID, it is a required input to be able to run an instance of the Azure Blob Storage event source.

A resource ID for an Event Hub has the following format:

/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.EventHub/namespaces/{namespaceName}/eventHubs/{eventHubName}

Note

The resource ID of the corresponding Event Hubs namespace is obtained by simply omitting the /eventHubs/{eventHubName} part of the Event Hub's resource ID.

Resource IDs can also be obtained using the Azure CLI (az). The following command line uses values from the screenshots above:

$ az eventhubs eventhub show --resource-group blobstorage-source-dev --namespace-name eventsourcedev --name my-event-hub
{
  "id": "/subscriptions/15537daf-e607-4df8-b2ef-277248b205b3/resourceGroups/blobstorage-source-dev/providers/Microsoft.EventHub/namespaces/eventsourcedev/eventhubs/my-event-hub",
  "resourceGroup": "blobstorage-source-dev",
  "type": "Microsoft.EventHub/Namespaces/EventHubs",
  "name": "my-event-hub",
  "location": "East US",
  "status": "Active",
  ...
}

Deploying an Instance of the Source

Open the Bridge creation screen and add a source of type Azure Blob Storage.

In the Source creation form, give a name to the event source and add the following information:

• Secret: Service Principal authentication credentials, as described in the previous sections.
• Storage Account ID: Resource ID of the Storage Account.
• Event Hub ID: Resource ID of either
  • an Event Hubs namespace (Event Hub managed by the event source)
  • an Event Hubs instance (Event Hub managed by the user)
• Event types: (optional) List of event types to subscribe to. BlobCreated and BlobDeleted are enabled by default when no item is set.

After clicking the Save button, you will be taken back to the Bridge editor. Proceed to adding the remaining components to the Bridge, then submit it.

A ready status on the main Bridges page indicates that the event subscription was successfully created in the configured Storage Account, and that the event source is ready to consume events from Event Hubs.

This can be confirmed by navigating back to the Azure Portal and ensuring that:

• The Storage Account contains a new Event Subscription targeting Event Hubs.
• The Resource Group contains an Event Grid System Topic with an Event Subscription matching the one from the Storage Account.

Event Types

The TriggerMesh event source for Azure Blob Storage emits events of the following types:

• Microsoft.Storage.BlobCreated
• Microsoft.Storage.BlobDeleted
• Microsoft.Storage.BlobRenamed
• Microsoft.Storage.DirectoryCreated
• Microsoft.Storage.DirectoryDeleted
• Microsoft.Storage.DirectoryRenamed
• Microsoft.Storage.BlobTierChanged
• Microsoft.Storage.AsyncOperationInitiated
• Microsoft.Storage.BlobInventoryPolicyCompleted