Databricks can be configured to query data in an external location, like Amazon S3 or Azure Data Lakes. This document covers the information needed to register a Databricks external configuration in Openbridge.

There are five configuration elements relating to external storage.

Be aware that this data will be found in a mix of locations. Some will be in the Azure UI, and others in the Databricks UI.

If you are unfamiliar with using external storage with Databricks, see these docs describing how to configure your Azure and Databricks environments;

The container name in Azure should match your registered external location in Databricks.

The connection string for the Azure Data Lake storage account of the container.

The name of the storage credentials used when you registered your external location in Databricks. This should be the same credential that appears below in the Data Lake External Location Storage URI. The credential will look like a long string: xxxxxx-xxxx-xxxx-xxxx-xxxxxxx-xxxx-xxxxxx-xxxx-xxxxxxxx

This is the URI for the external storage location you registered within Databricks.The URI will always begin with a abfss.
​
The URI also follows a standard format with your container name and storage account like this: abfss://<container-name>@<storage-account-name>.dfs.core.windows.net/

A path is optional, though we suggest creating a folder in your container called parquet to organize deliveries.
​
For example, if you have a directory in your container that you want to use for storing all your Openbridge data, you would first create that directory in your container and specify this directory in the setup screen on Openbridge.
​
If the directory were called, myexternaldatabricksdata you would set the path to myexternaldatabricksdata . If left empty, we would use the container's root to store data.