Available Monitor Types

The monitor types available in Decube's Data Quality module and what each one detects.

Decube provides six monitor types, each designed to detect a different category of data quality issue. Schema Drift and Job Failure monitors are enabled automatically when you connect a data source. All other types are created manually from the Config module.

---

Table-level monitors

Schema Drift

Schema Drift monitors detect structural changes to your tables — column additions, deletions, data type changes, and other schema modifications. They are enabled automatically on all tables when you connect a data source and require no additional setup.

Modify Schema Drift Monitors

Freshness

Freshness monitors detect when data stops arriving on its expected schedule. The scheduled Freshness monitor uses an ML model trained on historical arrival patterns — it learns when data normally shows up and raises an incident when the probability of arrival drops below 50%. This means it accounts for expected quiet periods (weekends, off-hours) without alerting unnecessarily.

Set Up Freshness Monitors

Volume

Volume monitors detect when the row-count increment added to a table per scan window falls outside the expected range. The monitor tracks the number of new rows added since the last scan — not the total row count — so it catches both missing data loads and unexpected data spikes.

Set Up Volume Monitors

---

Column-level monitors

Field Health

Field Health monitors run tests against individual columns — null rates, uniqueness, value ranges, string patterns, and more. Available test types include:

• Null — monitors null values (Absolute, Percentage, or Auto threshold)

• Unique — monitors duplicate values (Absolute, Percentage, or Auto threshold)

• Average / Min / Max — monitors statistical metrics (Range or Auto threshold)

• Cardinality — tracks distinct value counts using a 5-period rolling window (Scheduled only; no training period required)

• String Length — validates string lengths (Range or Auto threshold)

• Email / UUID / Regex Match — validates format compliance (Absolute, Percentage, or Auto threshold)

Available test types depend on the column's data type. For example, Average, Min, and Max apply to numeric columns only.

Set Up Field Health Monitors

---

Advanced monitors

Custom SQL

Custom SQL monitors let you define validation logic in SQL. You write a query that returns the failing rows; Decube counts those rows and opens an incident when the count is greater than zero. Custom SQL monitors attach to a data source connection rather than a specific table, so your query can reference any table in that source.

Custom SQL monitors do not support Smart Training. All thresholds are set manually.

Set Up Custom SQL Monitors

Job Failure

Job Failure monitors detect failed runs in connected ETL tools (dbt, Airflow, and others). They are auto-created when you connect an ETL-type data source and require no additional configuration.

Modify Job Failure Monitors (Data Job)

---

Grouped-By monitors

Any monitor type (except Volume and Freshness) can be configured with a Group By column, creating one sub-monitor per distinct value in that column. Each sub-monitor tracks its own metric and opens incidents independently, giving you per-segment visibility without creating monitors manually for each value.

Grouped-by Monitors