Decube
Try for free
  • 🚀Overview
    • Welcome to decube
    • Getting started
      • How to connect data sources
    • Security and Compliance
    • Data Policy
    • Changelog
    • Public Roadmap
  • Support
  • 🔌Data Warehouses
    • Snowflake
    • Redshift
    • Google Bigquery
    • Databricks
    • Azure Synapse
  • 🔌Relational Databases
    • PostgreSQL
    • MySQL
    • SingleStore
    • Microsoft SQL Server
    • Oracle
  • 🔌Transformation Tools
    • dbt (Cloud Version)
    • dbt Core
    • Fivetran
    • Airflow
    • AWS Glue
    • Azure Data Factory
    • Apache Spark
      • Apache Spark in Azure Synapse
    • OpenLineage (BETA)
    • Additional configurations
  • 🔌Business Intelligence
    • Tableau
    • Looker
    • PowerBI
  • 🔌Data Lake
    • AWS S3
    • Azure Data Lake Storage (ADLS)
      • Azure Function for Metadata
    • Google Cloud Storage (GCS)
  • 🔌Ticketing and Collaboration
    • ServiceNow
    • Jira
  • 🔒Security and Connectivity
    • Enabling VPC Access
    • IP Whitelisting
    • SSH Tunneling
    • AWS Identities
  • ✅Data Quality
    • Incidents Overview
    • Incident model feedback
    • Enable asset monitoring
    • Available Monitor Types
    • Available Monitor Modes
    • Catalog: Add/Modify Monitor
    • Set Up Freshness & Volume Monitors
    • Set Up Field Health Monitors
    • Set Up Custom SQL Monitors
    • Grouped-by Monitors
    • Modify Schema Drift Monitors
    • Modify Job Failure Monitors (Data Job)
    • Custom Scheduling For Monitors
    • Config Settings
  • 📖Catalog
    • Overview of Asset Types
    • Assets Catalog
    • Asset Overview
    • Automated Lineage
      • Lineage Relationship
      • Supported Data Sources and Lineage Types
    • Add lineage relationships manually
    • Add tags and classifications to fields
    • Field Statistcs
    • Preview sample data
  • 📚Glossary
    • Glossary, Category and Terms
    • Adding a new glossary
    • Adding Terms and Linked Assets
  • Moving Terms to Glossary/Category
  • AI Copilot
    • Copilot's Autocomplete
  • 🤝Collaboration
    • Ask Questions
    • Rate an asset
  • 🌐Data Mesh [BETA]
    • Overview on Data Mesh [BETA]
    • Creating and Managing Domains/Sub-domains
    • Adding members to Domain/Sub-domain
    • Linking Entities to Domains/Sub-domains
    • Adding Data Products to Domains/Subdomains
    • Creating a draft Data Asset
    • Adding a Data Contract - Default Settings
    • Adding a Data Contract - Freshness Test
    • Adding a Data Contract - Column Tests
    • Publishing the Data Asset
  • 🏛️Governance
    • Governance module
    • Classification Policies
    • Auto-classify data assets
  • ☑️Approval Workflow
    • What are Change Requests?
    • Initiate a change request
    • What are Access Requests?
    • Initiate an Access Request
  • 📋Reports
    • Overview of Reports
    • Supported sources for Reports
    • Asset Report: Data Quality Scorecard
  • 📊Dashboard
    • Dashboard Overview
    • Incidents
    • Quality
  • ⏰Alert Notifications
    • Get alerts on email
    • Connect your Slack channels
    • Connect to Microsoft Teams
    • Webhooks integration
  • 🏛️Manage Access
    • User Management - Overview
    • Invite users
    • Deactivate or re-activate users
    • Revoke a user invite
  • 🔐Group-based Access Controls
    • Groups Management - Overview
    • Create Groups & Assign Policies
    • Source-based Policies
    • Administrative-based Policies
    • Module-based Policies
    • What is the "Owners" group?
  • 🗄️Org Settings
    • Multi-factor authentication
    • Single Sign-On (SSO) with Microsoft
    • Single Sign-On (SSO) with JumpCloud
  • ❓Support
    • Supported Features by Integration
    • Frequently Asked Questions
    • Supported Browsers and System Requirements
  • Public API (BETA)
    • Overview
      • Data API
        • Glossary
        • Lineage
        • ACL
          • Group
      • Control API
        • Users
    • API Keys
Powered by GitBook
On this page
  • Table-level tests
  • Schema Drift
  • Freshness
  • Volume
  • Field-level tests
  • Null
  • Uniqueness
  • Average
  • Min and Max
  • Cardinality
  • String length
  • Is email
  • Is UUID
  • Matches Regex
  1. Data Quality

Available Monitor Types

Here's a list of all monitors that we support out of the box.

PreviousEnable asset monitoringNextAvailable Monitor Modes

Last updated 1 year ago

Table-level tests

Schema Drift

This is enabled automatically for all tables when you connect a data source. This detects any schema changes, including table or column addition, deletion, or changing data types. These changes may cause compatibility issues with the database and applications that use these tables.

Freshness

Our Freshness monitors track the time since the table was last inserted or updated. Our incident detection model learns how frequently your table is updated and raises an alert if it has been too long.

Volume

Our Volume monitors track the row count that was inserted into a table. Our incident detection model defines a threshold of the expected count of rows inserted based on previous scans, and if it detects an insert that was below or exceeds the threshold, an incident will be raised.

Field-level tests

The available tests depend on the data type of the field selected; eg. min/max is only supported on numeric columns.

Null

Checks for missing values within the table. You can set:

  • is Null test which checks for the count of rows with null values in a column. This test will return a fail when at least one column is null.

  • Null percentage test which calculates the % of how many values in a column are nulls. age

This test is helpful for columns where the column has dependencies on other columns or is a column frequently left blank by users.

Uniqueness

Measures the distinctness of values within the column and if there are any duplicates. You can set:

  • is Unique test which checks if there are any duplicates within a column.

  • Unique percentage test which calculates the % of how many values in a column are distinct.

This test is useful when the column used is a primary key or has unique constraints.

Average

Checks the average values within the column and compare them to the threshold values set.

Min and Max

Checks the minimum or maximum values within the column and compare them to the threshold values set.

Cardinality

Tests the number of distinct values in a table column relative to the number of rows in the table. A lot of distinct values can be known as high cardinality, and a lot of repeated values can be known as low cardinality.

When you enable this monitor, we will first check the cardinality of your columns and classify them as high, medium or low. An incident will be raised when the classification moves from one to another, eg. high to medium or low to high.

String length

Checks the minimum or maximum of the column's length which column type is string.

Is email

Checks if there are any values that do not conform to a valid email address format.

Is UUID

Checks if there are any values that do not conform to the UUID format.

Matches Regex

You can write a regex pattern that we can check against.

For Microsoft SQL Server and Azure SQL data sources, note that the input thresholds for matching will not be a regex pattern match but wildcard syntax matching. More info here:

✅
Microsoft Wildcard Characters