Airflow

Adding Airflow to your decube connections helps your team to discover, document and monitor the quality of your pipelines.

Supported Capabilities

Data Quality
Capability

Freshness

Volume

Schema Drift

Field Health

Custom SQL

Job Failure

Catalog
Capability

Data Profiling

Data Preview

Minimum Requirement

  • Airflow Username

  • Airflow User Password

  • Airflow API host address

  • Airflow API enabled and set to Basic Auth. See Airflow Documentation for this.

  • Airflow Version 2.3.0 and above (Version >= 2.0.0 and < 2.3.0 may not work fully)

Connect to Airflow

Creating an Airflow User for Decube

  1. Go to Security > List Users

  1. Click on "+" to Add User

  1. Insert information for new decube user

    1. Username - suggested value: decube

    2. Role - Minimum required Op (which is a default role from Airflow)

    3. Password - Use a strong password

Airflow API is not Publicly Accessible

  • For decube to monitor your Airflow service, we will require that the Airflow API be publicly accessible or privately accessible to a SSH bastion host. Instruction on setting up a bastion host can be found here SSH Tunneling

Last updated