# Available Monitor Types

{% embed url="<https://youtu.be/k9fM4a8KICE>" %}

Decube offers comprehensive monitoring capabilities to ensure data quality across your entire data infrastructure. Choose the right monitor type based on your specific data quality requirements.

## Quick Selection Guide

| **Monitor Type** | **Best For**               | **Setup Time** | **Technical Level** |
| ---------------- | -------------------------- | -------------- | ------------------- |
| **Schema Drift** | Preventing pipeline breaks | Auto-enabled   | Beginner            |
| **Freshness**    | Data dependencies          | 5 minutes      | Beginner            |
| **Volume**       | Data load validation       | 5 minutes      | Beginner            |
| **Field Health** | Column-level data quality  | 10 minutes     | Intermediate        |
| **Custom SQL**   | Complex business rules     | 15 minutes     | Advanced            |
| **Job Failure**  | ETL pipeline monitoring    | Auto-enabled   | Beginner            |

***

## Table-Level Monitors

### Schema Drift ⚡ Auto-Enabled

**Purpose:** Automatically detects structural changes to your tables and columns **Value:** Prevents downstream application failures and pipeline breaks

This monitor is enabled automatically for all tables when you connect a data source. It detects:

* Table or column additions/deletions
* Data type changes
* Schema modifications that may cause compatibility issues

**Use Cases:**

* 🚨 Alert when critical table structures change
* 🛡️ Protect downstream applications from schema breaks
* 📊 Track data warehouse evolution over time

{% content-ref url="how-to-set-up-monitors/set-up-schema-drift-monitors" %}
[set-up-schema-drift-monitors](https://docs.decube.io/data-quality/how-to-set-up-monitors/set-up-schema-drift-monitors)
{% endcontent-ref %}

### Freshness 🕒

**Purpose:** Tracks when tables were last updated to ensure data timeliness **Value:** Ensures dashboards and reports have current data

Our ML-powered freshness monitors learn your table update patterns and alert when data becomes stale based on historical patterns.

**Use Cases:**

* ⏰ Critical for real-time dashboards
* 📈 Essential for daily business reports
* 🔄 Monitor ETL pipeline completion

{% content-ref url="how-to-set-up-monitors/set-up-freshness-and-volume-monitors" %}
[set-up-freshness-and-volume-monitors](https://docs.decube.io/data-quality/how-to-set-up-monitors/set-up-freshness-and-volume-monitors)
{% endcontent-ref %}

### Volume 📊

**Purpose:** Monitors row count changes and detects data load anomalies **Value:** Catches missing or incomplete data loads before they impact business

Volume monitors establish baselines for expected row counts and alert when insertions fall below or exceed normal ranges.

**Use Cases:**

* 📉 Detect missing data loads
* 📈 Identify unexpected data spikes
* 🔍 Monitor ETL pipeline data volumes

{% content-ref url="how-to-set-up-monitors/set-up-freshness-and-volume-monitors" %}
[set-up-freshness-and-volume-monitors](https://docs.decube.io/data-quality/how-to-set-up-monitors/set-up-freshness-and-volume-monitors)
{% endcontent-ref %}

***

## Column-Level Monitors

### Field Health 🔍

**Purpose:** Validates data quality at the column level with comprehensive tests **Value:** Ensures clean, consistent data for analytics and ML models

{% hint style="info" %}
Available tests depend on the **data type** of the field selected. For example, min/max tests are only supported on `numeric` columns.
{% endhint %}

**Available Field Health Tests:**

#### Data Completeness Tests

**Null Checks**

* `Null`: Monitors null values with configurable thresholds
  * **Absolute mode**: Set threshold by row count (e.g., "fail when > 100 null rows")
  * **Percentage mode**: Set threshold by percentage (e.g., "fail when > 5% nulls")
  * **Auto mode**: Let ML learn acceptable null patterns (scheduled only)

*Use Case:* Essential for mandatory fields and columns with dependencies

**Uniqueness Validation**

* `Unique`: Monitors duplicate values with configurable thresholds
  * **Absolute mode**: Set threshold by row count (e.g., "fail when > 10 duplicate rows")
  * **Percentage mode**: Set threshold by percentage (e.g., "fail when < 95% unique")
  * **Auto mode**: Let ML learn acceptable uniqueness patterns (scheduled only)

*Use Case:* Critical for primary keys and unique constraints

#### Data Range & Distribution Tests

**Statistical Validation**

* `Average`: Monitors average values against expected ranges
* `Min/Max`: Validates minimum and maximum value boundaries
* `Cardinality`: Tracks distinct value counts (high/medium/low classification)

*Use Case:* Detect outliers and validate business rule compliance

**String Validation**

* `String length`: Validates minimum/maximum string lengths using range thresholds
* `Email`: Monitors invalid email addresses with configurable thresholds
  * **Absolute mode**: Set threshold by row count (e.g., "fail when > 50 invalid emails")
  * **Percentage mode**: Set threshold by percentage (e.g., "fail when > 2% invalid")
  * **Auto mode**: Let ML learn acceptable patterns (scheduled only)
* `UUID`: Monitors invalid UUIDs with configurable thresholds (supports Absolute, Percentage, Auto modes)
* `Regex Match`: Custom pattern matching validation with configurable thresholds
  * Pattern is specified separately from threshold
  * Threshold controls how many non-matching values trigger an incident
  * Supports Absolute, Percentage, and Auto modes

{% hint style="info" %}
**Microsoft SQL Server Note:** For SQL Server and Azure SQL data sources, regex patterns use wildcard syntax instead of standard regex. See [Microsoft Wildcard Characters](https://learn.microsoft.com/en-us/sql/t-sql/language-elements/percent-character-wildcard-character-s-to-match-transact-sql?view=sql-server-ver16) for details.
{% endhint %}

{% content-ref url="how-to-set-up-monitors/set-up-field-tests" %}
[set-up-field-tests](https://docs.decube.io/data-quality/how-to-set-up-monitors/set-up-field-tests)
{% endcontent-ref %}

***

## Advanced Monitoring

### Custom SQL 🔧

**Purpose:** Create custom validation logic for complex business rules **Value:** Monitor sophisticated data relationships and business-specific requirements

Write custom SQL queries to validate complex business logic. Incidents trigger when your query returns any rows (row\_count > 0).

**Common Use Cases:**

* 💰 Revenue reconciliation between systems
* 🔗 Cross-table data consistency checks
* 📋 Complex business rule validation
* ⚖️ Data quality SLA monitoring

**Example Applications:**

```sql
-- Detect revenue discrepancies
SELECT * FROM sales
WHERE total_amount != (price * quantity * tax_rate)

-- Monitor data freshness SLAs
SELECT * FROM critical_tables
WHERE last_updated < NOW() - INTERVAL '2 hours'
```

{% content-ref url="how-to-set-up-monitors/custom-sql-monitors" %}
[custom-sql-monitors](https://docs.decube.io/data-quality/how-to-set-up-monitors/custom-sql-monitors)
{% endcontent-ref %}

### Job Failure Monitoring ⚙️ Auto-Enabled

**Purpose:** Automatically monitors ETL pipeline and data job execution **Value:** Ensures data transformation processes complete successfully

Job Failure monitors are automatically created when you connect ETL-type sources. They track failed jobs to identify processing issues.

**Key Features:**

* 🔄 Auto-configured for dbt, Airflow, and other ETL tools
* 📧 Customizable alert channels
* 📊 Historical failure tracking
* ⚡ Real-time failure detection

{% content-ref url="how-to-set-up-monitors/set-up-data-job-job-failure-monitors" %}
[set-up-data-job-job-failure-monitors](https://docs.decube.io/data-quality/how-to-set-up-monitors/set-up-data-job-job-failure-monitors)
{% endcontent-ref %}

***

## Enhanced Monitoring Features

### Grouped-By Monitoring 🎯

**Purpose:** Segment monitoring by dimension values for granular insights **Value:** Monitor data quality at the business logic level

Track data quality metrics segmented by specific column values (e.g., by region, customer type, or product category).

**Applications:**

* 📍 Monitor data quality by geographic region
* 👥 Track customer data completeness by segment
* 📦 Validate product data by category
* 🏢 Ensure compliance by business unit

{% content-ref url="how-to-set-up-monitors/set-up-grouped-by-monitors" %}
[set-up-grouped-by-monitors](https://docs.decube.io/data-quality/how-to-set-up-monitors/set-up-grouped-by-monitors)
{% endcontent-ref %}

***

## Getting Started Recommendations

### For Data Engineers 👨‍💻

1. Start with **Freshness & Volume** for critical tables
2. Add **Custom SQL** for complex validation logic
3. Use **Field Health** for data pipeline validation

### For Governance Teams 🛡️

1. Begin with **Schema Drift** monitoring (auto-enabled)
2. Implement **Field Health** for compliance requirements
3. Set up **Grouped-By** monitoring for business segments

### For Platform Admins 👤

1. Configure **Job Failure** monitoring for all ETL processes
2. Set up **Volume** monitoring for data load validation
3. Establish **Freshness** SLAs for business-critical data

***

**Next Steps:** Ready to get started? Visit our [Enable Asset Monitoring](https://docs.decube.io/data-quality/enable-asset-monitoring) guide to begin setting up your first monitors.