Configuring an Azure Data Lake destination is quick and easy. This guide will walk you through creating a Gen2 storage account, a data lake container, and access credentials.
This process aims to generate three elements needed by Openbridge so we can securely establish a connection to your Azure Data Lake destination. These are:
Storage Account Name
Container Name
Connection string
Please be sure to take note of all three as you go through this process. Let's go...
Step 1: Create a v2 Storage Account
Log into your Azure portal: https://portal.azure.com/. To create a general-purpose v2 storage account in the Azure portal, follow these steps:
On the Azure portal menu, select All Services. In the list of resources, type Storage Accounts. As you begin typing, the list filters based on your input. Select Storage Accounts.
On the Storage Accounts window that appears, choose Add.
Select the subscription to create the storage account.
Under the Resource group field, select Create New. Enter a name for your new resource group, as shown in the following image.
Next, enter a name for your storage account. The name you choose must be unique across Azure. It must also be between 3 and 24 characters in length and can include numbers and lowercase letters only.
Select a location for your storage account, or use the default location.
Leave these fields set to their default values:
We will use Azure Data Lake Storage, which requires you to choose the Advanced tab and then set the Hierarchical namespace to Enabled.
Select Review + Create to review your storage account settings and create the account.
You just created a Gen2 storage account! Next, we need to create a data lake container.
Step 2: Create Your Data Lake Container
Locate your newly created storage account under
Storage accounts":
You should see your newly created v2 storage account listed:
Select the storage account you want Openbridge to use.
You will need to create a new container. Look for the "Containers" option in the Azure interface.
Select Containers, add (+) a new container, and enter a name reflecting Openbridge usage. For example,
openbridge-data-lake. You can use a name that best reflects a name you can identify as being used by Openbridge.Make sure access is
Private.
Once you are ready, then create.
Congrats! You now have a data lake storage account and container.
IMPORTANT: Account name vs. Container name
Don’t forget to note the different names. For example, the Storage account name below is openbridgelake . The Storage container's name is datalake. When registering your destination, you must supply the Storage container name, not the Storage account name. This can get confusing, but it should help you out.
Step 3: Access Authorization
The last step is getting access credentials so we can write to your new data lake.
Within your storage account, navigate to "Access Keys".
You will see the
Storage account nameand theConnection string. Make sure theStorage account nameis the one where your container is located.
You are all done! You now have a Storage Container Name and Connection string. Openbridge will use these to authenticate our applications when requesting your Azure storage account.
Log into your Openbridge account and configure your Azure Data Lake destination.
Note: As always, store your access keys securely - for example, do not share this information over email or any other insecure channel! Lastly, if you regenerate your access keys, you must update Openbridge resources and applications that access this storage account.