Available Monitor Types
Comprehensive guide to all monitor types available in Decube's data quality platform.
Decube offers comprehensive monitoring capabilities to ensure data quality across your entire data infrastructure. Choose the right monitor type based on your specific data quality requirements.
Quick Selection Guide
Monitor Type
Best For
Setup Time
Technical Level
Schema Drift
Preventing pipeline breaks
Auto-enabled
Beginner
Freshness
Data dependencies
5 minutes
Beginner
Volume
Data load validation
5 minutes
Beginner
Field Health
Column-level data quality
10 minutes
Intermediate
Custom SQL
Complex business rules
15 minutes
Advanced
Job Failure
ETL pipeline monitoring
Auto-enabled
Beginner
Table-Level Monitors
Schema Drift ⚡ Auto-Enabled
Purpose: Automatically detects structural changes to your tables and columns Value: Prevents downstream application failures and pipeline breaks
This monitor is enabled automatically for all tables when you connect a data source. It detects:
Table or column additions/deletions
Data type changes
Schema modifications that may cause compatibility issues
Use Cases:
🚨 Alert when critical table structures change
🛡️ Protect downstream applications from schema breaks
📊 Track data warehouse evolution over time
Freshness 🕒
Purpose: Tracks when tables were last updated to ensure data timeliness Value: Ensures dashboards and reports have current data
Our ML-powered freshness monitors learn your table update patterns and alert when data becomes stale based on historical patterns.
Use Cases:
⏰ Critical for real-time dashboards
📈 Essential for daily business reports
🔄 Monitor ETL pipeline completion
Volume 📊
Purpose: Monitors row count changes and detects data load anomalies Value: Catches missing or incomplete data loads before they impact business
Volume monitors establish baselines for expected row counts and alert when insertions fall below or exceed normal ranges.
Use Cases:
📉 Detect missing data loads
📈 Identify unexpected data spikes
🔍 Monitor ETL pipeline data volumes
Column-Level Monitors
Field Health 🔍
Purpose: Validates data quality at the column level with comprehensive tests Value: Ensures clean, consistent data for analytics and ML models
Available Field Health Tests:
Data Completeness Tests
Null Checks
is Null: Validates no null values exist (fails if any nulls found)Null percentage: Monitors percentage of null values in column
Use Case: Essential for mandatory fields and columns with dependencies
Uniqueness Validation
is Unique: Ensures all values are distinct (no duplicates)Unique percentage: Calculates percentage of distinct values
Use Case: Critical for primary keys and unique constraints
Data Range & Distribution Tests
Statistical Validation
Average: Monitors average values against expected rangesMin/Max: Validates minimum and maximum value boundariesCardinality: Tracks distinct value counts (high/medium/low classification)
Use Case: Detect outliers and validate business rule compliance
String Validation
String length: Validates minimum/maximum string lengthsIs email: Ensures proper email address formattingIs UUID: Validates UUID format complianceMatches Regex: Custom pattern matching validation
Advanced Monitoring
Custom SQL 🔧
Purpose: Create custom validation logic for complex business rules Value: Monitor sophisticated data relationships and business-specific requirements
Write custom SQL queries to validate complex business logic. Incidents trigger when your query returns any rows (row_count > 0).
Common Use Cases:
💰 Revenue reconciliation between systems
🔗 Cross-table data consistency checks
📋 Complex business rule validation
⚖️ Data quality SLA monitoring
Example Applications:
-- Detect revenue discrepancies
SELECT * FROM sales
WHERE total_amount != (price * quantity * tax_rate)
-- Monitor data freshness SLAs
SELECT * FROM critical_tables
WHERE last_updated < NOW() - INTERVAL '2 hours'Job Failure Monitoring ⚙️ Auto-Enabled
Purpose: Automatically monitors ETL pipeline and data job execution Value: Ensures data transformation processes complete successfully
Job Failure monitors are automatically created when you connect ETL-type sources. They track failed jobs to identify processing issues.
Key Features:
🔄 Auto-configured for dbt, Airflow, and other ETL tools
📧 Customizable alert channels
📊 Historical failure tracking
⚡ Real-time failure detection
Enhanced Monitoring Features
Grouped-By Monitoring 🎯
Purpose: Segment monitoring by dimension values for granular insights Value: Monitor data quality at the business logic level
Track data quality metrics segmented by specific column values (e.g., by region, customer type, or product category).
Applications:
📍 Monitor data quality by geographic region
👥 Track customer data completeness by segment
📦 Validate product data by category
🏢 Ensure compliance by business unit
Getting Started Recommendations
For Data Engineers 👨💻
Start with Freshness & Volume for critical tables
Add Custom SQL for complex validation logic
Use Field Health for data pipeline validation
For Governance Teams 🛡️
Begin with Schema Drift monitoring (auto-enabled)
Implement Field Health for compliance requirements
Set up Grouped-By monitoring for business segments
For Platform Admins 👤
Configure Job Failure monitoring for all ETL processes
Set up Volume monitoring for data load validation
Establish Freshness SLAs for business-critical data
Next Steps: Ready to get started? Visit our Enable Asset Monitoring guide to begin setting up your first monitors.
Last updated