Available Monitor Types

Here's a list of all monitors that we support out of the box.

Table-level tests

Schema Drift

This is enabled automatically for all tables when you connect a data source. This detects any schema changes, including table or column addition, deletion, or changing data types. These changes may cause compatibility issues with the database and applications that use these tables.

Freshness

Our Freshness monitors track the time since the table was last inserted or updated. Our incident detection model learns how frequently your table is updated and raises an alert if it has been too long.

Volume

Our Volume monitors track the row count that was inserted into a table. Our incident detection model defines a threshold of the expected count of rows inserted based on previous scans, and if it detects an insert that was below or exceeds the threshold, an incident will be raised.

Field-level tests

The available tests depend on the data type of the field selected; eg. min/max is only supported on numeric columns.

Null

Checks for missing values within the table. You can set:

• is Null test which checks for the count of rows with null values in a column. This test will return a fail when at least one column is null.

• Null percentage test which calculates the % of how many values in a column are nulls. age

This test is helpful for columns where the column has dependencies on other columns or is a column frequently left blank by users.

Uniqueness

Measures the distinctness of values within the column and if there are any duplicates. You can set:

• is Unique test which checks if there are any duplicates within a column.

• Unique percentage test which calculates the % of how many values in a column are distinct.

This test is useful when the column used is a primary key or has unique constraints.

Average

Checks the average values within the column and compare them to the threshold values set.

Min and Max

Checks the minimum or maximum values within the column and compare them to the threshold values set.

Cardinality

Tests the number of distinct values in a table column relative to the number of rows in the table. A lot of distinct values can be known as high cardinality, and a lot of repeated values can be known as low cardinality.

When you enable this monitor, we will first check the cardinality of your columns and classify them as high, medium or low. An incident will be raised when the classification moves from one to another, eg. high to medium or low to high.

String length

Checks the minimum or maximum of the column's length which column type is string.

Is email

Checks if there are any values that do not conform to a valid email address format.

Is UUID

Checks if there are any values that do not conform to the UUID format.

Matches Regex

You can write a regex pattern that we can check against.

For Microsoft SQL Server and Azure SQL data sources, note that the input thresholds for matching will not be a regex pattern match but wildcard syntax matching. More info here: Microsoft Wildcard Characters