Azure Data Factory

Azure Data Factory is a cloud-based data integration and orchestration service by Microsoft Azure. See all your ETLs in decube and monitor the status of your jobs.

Supported Capabilities

Capability	Supported
Metadata Extraction	✅
Metadata Types Collected	Schema, Virtual Table, Virtual Column, Data Job, Data Run, Data Task
Data Profiling	❌
Data Preview	❌
Data Quality	❌
Configurable Collection	❌
External Table	❌
View Table	❌
Stored Procedure	❌

Supported Lineage Mapping

Azure Data Factory can map lineage relationships to upstream and downstream objects from the following connectors:

• Upstream Connectors: postgresql, mysql, synapse, azure_server, databricks, sql_server, redshift, bigquery, adls

• Downstream Connectors: postgresql, mysql, synapse, azure_server, databricks, sql_server, redshift, bigquery, adls

Connection Requirements

From our Azure account, we will need the following information:

• Tenant ID

• Client ID

• Client Secret

• Subscription ID

• Resource Group Name

• Factory Name

• Data Source Name

Azure Data Factory

How to connect

1. On the Azure Home Page, go to Azure Active Directory.

1. Go to App registrations.

1. Click on New registration.

1. Click Register.

1. Save the Application (client) ID and Directory (tenant) ID.

2. Click Add a certificate or secret.

3. Go to Client secrets and click + New client secret.

1. Click Add.

1. Copy and save the Value.

1. Go to Data Factories and click the factory you wanna add.

1. Go to Access Control (IAM).

1. Click on + Add and select Add role assignment.

1. Select Data Factory Contributor.

1. Go to Members tab and Click on + Select members.

1. Search and select the service principal that was created in the previous step. Click on Select.

1. Go to Review + assign tab and Click Review + assign

1. Go to Data Factories and select the factory you wanna add, copy the Name and Resource group.

1. Copy the Subscription ID.

1. Fill all the required fields in the connection form, and click on Test this connection once connection is successful, give your database a name and connect the data source.