Azure Data Factory

Azure Data Factory is a cloud-based data integration and orchestration service by Microsoft Azure. See all your ETLs in decube and monitor the status of your jobs.

Supported Capabilities

Supported Capabilities

General

• Metadata — metadata extraction and display of asset information (tables, columns, schemas). Types collected: Schema, Virtual Table, Virtual Column, Data Job, Data Run, Data Task

Not Supported

Azure Data Factory can map lineage relationships to upstream and downstream objects from the following connectors:

• Upstream Connectors: postgresql, mysql, synapse, azure_server, databricks, sql_server, redshift, bigquery, adls

• Downstream Connectors: postgresql, mysql, synapse, azure_server, databricks, sql_server, redshift, bigquery, adls

Connection Requirements

From our Azure account, we will need the following information:

• Tenant ID

• Client ID

• Client Secret

• Subscription ID

• Resource Group Name

• Factory Name

• Data Source Name

Azure Data Factory

How to connect

1. On the Azure Home Page, go to Azure Active Directory.

1. Go to App registrations.

1. Click on New registration.

1. Click Register.

1. Save the Application (client) ID and Directory (tenant) ID.

2. Click Add a certificate or secret.

3. Go to Client secrets and click + New client secret.

1. Click Add.

1. Copy and save the Value.

1. Go to Data Factories and click the factory you wanna add.

1. Go to Access Control (IAM).

1. Click on + Add and select Add role assignment.

1. Select Data Factory Contributor.

1. Go to Members tab and Click on + Select members.

1. Search and select the service principal that was created in the previous step. Click on Select.

1. Go to Review + assign tab and Click Review + assign

1. Go to Data Factories and select the factory you wanna add, copy the Name and Resource group.

1. Copy the Subscription ID.

1. Fill all the required fields in the connection form, and click on Test this connection once connection is successful, give your database a name and connect the data source.