# Available Monitor Types

{% embed url="<https://youtu.be/k9fM4a8KICE>" %}

Decube offers comprehensive monitoring capabilities to ensure data quality across your entire data infrastructure. Choose the right monitor type based on your specific data quality requirements.

## Quick Selection Guide

| **Monitor Type** | **Best For**               | **Setup Time** | **Technical Level** |
| ---------------- | -------------------------- | -------------- | ------------------- |
| **Schema Drift** | Preventing pipeline breaks | Auto-enabled   | Beginner            |
| **Freshness**    | Data dependencies          | 5 minutes      | Beginner            |
| **Volume**       | Data load validation       | 5 minutes      | Beginner            |
| **Field Health** | Column-level data quality  | 10 minutes     | Intermediate        |
| **Custom SQL**   | Complex business rules     | 15 minutes     | Advanced            |
| **Job Failure**  | ETL pipeline monitoring    | Auto-enabled   | Beginner            |

***

## Table-Level Monitors

### Schema Drift ⚡ Auto-Enabled

**Purpose:** Automatically detects structural changes to your tables and columns **Value:** Prevents downstream application failures and pipeline breaks

This monitor is enabled automatically for all tables when you connect a data source. It detects:

* Table or column additions/deletions
* Data type changes
* Schema modifications that may cause compatibility issues

**Use Cases:**

* 🚨 Alert when critical table structures change
* 🛡️ Protect downstream applications from schema breaks
* 📊 Track data warehouse evolution over time

{% content-ref url="how-to-set-up-monitors/set-up-schema-drift-monitors" %}
[set-up-schema-drift-monitors](https://docs.decube.io/data-quality/how-to-set-up-monitors/set-up-schema-drift-monitors)
{% endcontent-ref %}

### Freshness 🕒

**Purpose:** Tracks when tables were last updated to ensure data timeliness **Value:** Ensures dashboards and reports have current data

Our ML-powered freshness monitors learn your table update patterns and alert when data becomes stale based on historical patterns.

**Use Cases:**

* ⏰ Critical for real-time dashboards
* 📈 Essential for daily business reports
* 🔄 Monitor ETL pipeline completion

{% content-ref url="how-to-set-up-monitors/set-up-freshness-and-volume-monitors" %}
[set-up-freshness-and-volume-monitors](https://docs.decube.io/data-quality/how-to-set-up-monitors/set-up-freshness-and-volume-monitors)
{% endcontent-ref %}

### Volume 📊

**Purpose:** Monitors row count changes and detects data load anomalies **Value:** Catches missing or incomplete data loads before they impact business

Volume monitors establish baselines for expected row counts and alert when insertions fall below or exceed normal ranges.

**Use Cases:**

* 📉 Detect missing data loads
* 📈 Identify unexpected data spikes
* 🔍 Monitor ETL pipeline data volumes

{% content-ref url="how-to-set-up-monitors/set-up-freshness-and-volume-monitors" %}
[set-up-freshness-and-volume-monitors](https://docs.decube.io/data-quality/how-to-set-up-monitors/set-up-freshness-and-volume-monitors)
{% endcontent-ref %}

***

## Column-Level Monitors

### Field Health 🔍

**Purpose:** Validates data quality at the column level with comprehensive tests **Value:** Ensures clean, consistent data for analytics and ML models

{% hint style="info" %}
Available tests depend on the **data type** of the field selected. For example, min/max tests are only supported on `numeric` columns.
{% endhint %}

**Available Field Health Tests:**

#### Data Completeness Tests

**Null Checks**

* `Null`: Monitors null values with configurable thresholds
  * **Absolute mode**: Set threshold by row count (e.g., "fail when > 100 null rows")
  * **Percentage mode**: Set threshold by percentage (e.g., "fail when > 5% nulls")
  * **Auto mode**: Let ML learn acceptable null patterns (scheduled only)

*Use Case:* Essential for mandatory fields and columns with dependencies

**Uniqueness Validation**

* `Unique`: Monitors duplicate values with configurable thresholds
  * **Absolute mode**: Set threshold by row count (e.g., "fail when > 10 duplicate rows")
  * **Percentage mode**: Set threshold by percentage (e.g., "fail when < 95% unique")
  * **Auto mode**: Let ML learn acceptable uniqueness patterns (scheduled only)

*Use Case:* Critical for primary keys and unique constraints

#### Data Range & Distribution Tests

**Statistical Validation**

* `Average`: Monitors average values against expected ranges
* `Min/Max`: Validates minimum and maximum value boundaries
* `Cardinality`: Tracks distinct value counts (high/medium/low classification)

*Use Case:* Detect outliers and validate business rule compliance

**String Validation**

* `String length`: Validates minimum/maximum string lengths using range thresholds
* `Email`: Monitors invalid email addresses with configurable thresholds
  * **Absolute mode**: Set threshold by row count (e.g., "fail when > 50 invalid emails")
  * **Percentage mode**: Set threshold by percentage (e.g., "fail when > 2% invalid")
  * **Auto mode**: Let ML learn acceptable patterns (scheduled only)
* `UUID`: Monitors invalid UUIDs with configurable thresholds (supports Absolute, Percentage, Auto modes)
* `Regex Match`: Custom pattern matching validation with configurable thresholds
  * Pattern is specified separately from threshold
  * Threshold controls how many non-matching values trigger an incident
  * Supports Absolute, Percentage, and Auto modes

{% hint style="info" %}
**Microsoft SQL Server Note:** For SQL Server and Azure SQL data sources, regex patterns use wildcard syntax instead of standard regex. See [Microsoft Wildcard Characters](https://learn.microsoft.com/en-us/sql/t-sql/language-elements/percent-character-wildcard-character-s-to-match-transact-sql?view=sql-server-ver16) for details.
{% endhint %}

{% content-ref url="how-to-set-up-monitors/set-up-field-tests" %}
[set-up-field-tests](https://docs.decube.io/data-quality/how-to-set-up-monitors/set-up-field-tests)
{% endcontent-ref %}

***

## Advanced Monitoring

### Custom SQL 🔧

**Purpose:** Create custom validation logic for complex business rules **Value:** Monitor sophisticated data relationships and business-specific requirements

Write custom SQL queries to validate complex business logic. Incidents trigger when your query returns any rows (row\_count > 0).

**Common Use Cases:**

* 💰 Revenue reconciliation between systems
* 🔗 Cross-table data consistency checks
* 📋 Complex business rule validation
* ⚖️ Data quality SLA monitoring

**Example Applications:**

```sql
-- Detect revenue discrepancies
SELECT * FROM sales
WHERE total_amount != (price * quantity * tax_rate)

-- Monitor data freshness SLAs
SELECT * FROM critical_tables
WHERE last_updated < NOW() - INTERVAL '2 hours'
```

{% content-ref url="how-to-set-up-monitors/custom-sql-monitors" %}
[custom-sql-monitors](https://docs.decube.io/data-quality/how-to-set-up-monitors/custom-sql-monitors)
{% endcontent-ref %}

### Job Failure Monitoring ⚙️ Auto-Enabled

**Purpose:** Automatically monitors ETL pipeline and data job execution **Value:** Ensures data transformation processes complete successfully

Job Failure monitors are automatically created when you connect ETL-type sources. They track failed jobs to identify processing issues.

**Key Features:**

* 🔄 Auto-configured for dbt, Airflow, and other ETL tools
* 📧 Customizable alert channels
* 📊 Historical failure tracking
* ⚡ Real-time failure detection

{% content-ref url="how-to-set-up-monitors/set-up-data-job-job-failure-monitors" %}
[set-up-data-job-job-failure-monitors](https://docs.decube.io/data-quality/how-to-set-up-monitors/set-up-data-job-job-failure-monitors)
{% endcontent-ref %}

***

## Enhanced Monitoring Features

### Grouped-By Monitoring 🎯

**Purpose:** Segment monitoring by dimension values for granular insights **Value:** Monitor data quality at the business logic level

Track data quality metrics segmented by specific column values (e.g., by region, customer type, or product category).

**Applications:**

* 📍 Monitor data quality by geographic region
* 👥 Track customer data completeness by segment
* 📦 Validate product data by category
* 🏢 Ensure compliance by business unit

{% content-ref url="how-to-set-up-monitors/set-up-grouped-by-monitors" %}
[set-up-grouped-by-monitors](https://docs.decube.io/data-quality/how-to-set-up-monitors/set-up-grouped-by-monitors)
{% endcontent-ref %}

***

## Getting Started Recommendations

### For Data Engineers 👨‍💻

1. Start with **Freshness & Volume** for critical tables
2. Add **Custom SQL** for complex validation logic
3. Use **Field Health** for data pipeline validation

### For Governance Teams 🛡️

1. Begin with **Schema Drift** monitoring (auto-enabled)
2. Implement **Field Health** for compliance requirements
3. Set up **Grouped-By** monitoring for business segments

### For Platform Admins 👤

1. Configure **Job Failure** monitoring for all ETL processes
2. Set up **Volume** monitoring for data load validation
3. Establish **Freshness** SLAs for business-critical data

***

**Next Steps:** Ready to get started? Visit our [Enable Asset Monitoring](https://docs.decube.io/data-quality/enable-asset-monitoring) guide to begin setting up your first monitors.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.decube.io/data-quality/available-monitor-types.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.