Examples of Monitor Creation
Practical examples for creating data quality monitors using the Decube Public API. These examples demonstrate proper payload structure and follow all validation rules.
This page provides comprehensive examples for creating monitors using the Decube Public API. All examples follow the validation rules and constraints documented in the Create, Manage & Delete guide.
Prerequisites
Authentication
All API requests require authentication via the X-Decube-Api-Key header:
# Set your API key as an environment variable
export DECUBE_API_KEY="your-api-key-here"
# Include in curl requests
curl -H "X-Decube-Api-Key: $DECUBE_API_KEY" ...Base URL
Replace <REGION> with your Decube region (e.g., us, eu):
https://connect.<REGION>.decube.io/api/v1/data/monitorsBasic Examples
Example 1: Minimal Scheduled Volume Monitor
A basic scheduled monitor for dataset-level volume monitoring with hourly frequency.
curl -X POST \
"https://connect.<REGION>.decube.io/api/v1/data/monitors" \
-H "Content-Type: application/json" \
-H "X-Decube-Api-Key: $DECUBE_API_KEY" \
-d '{
"asset": { "type": "dataset", "id": 123 },
"test_type": "volume",
"mode": "scheduled",
"name": "hourly-volume-monitor",
"description": "Hourly volume monitoring for dataset 123",
"notify": true,
"incident_level": "warning",
"frequency": { "frequency": "1H" },
"row_creation": {
"filter_mode": "all_records"
}
}'Expected Response:
{ "monitor_id": 1 }Key Points:
row_creationis required for volume testsHourly frequency doesn't require timezone or time_of_day
all_recordsrow creation mode processes the entire dataset
Example 2: Daily Freshness Monitor with Timezone
A scheduled freshness monitor running daily at a specific time with timestamp-based row creation.
curl -X POST \
"https://connect.<REGION>.decube.io/api/v1/data/monitors" \
-H "Content-Type: application/json" \
-H "X-Decube-Api-Key: $DECUBE_API_KEY" \
-d '{
"asset": { "type": "dataset", "id": 456 },
"test_type": "freshness",
"mode": "scheduled",
"name": "daily-freshness-monitor",
"description": "Daily freshness monitor running at 3:00 UTC",
"notify": true,
"custom_alerts": {
"email": ["[email protected]"],
"slack": [],
"teams_webhook": []
},
"incident_level": "critical",
"frequency": {
"frequency": "1D",
"time_of_day": 3,
"timezone": "UTC"
},
"row_creation": {
"filter_mode": "timestamp",
"metric_column_id": 789,
"enable_smart_training": true
}
}'Key Points:
Daily frequency requires
time_of_day(0-23) andtimezonemetric_column_idmust reference an existing timestamp/datetime columnenable_smart_trainingenables automatic threshold learning
Example 3: On-Demand Field Health Monitor
An on-demand monitor for validating email format using regex matching.
curl -X POST \
"https://connect.<REGION>.decube.io/api/v1/data/monitors" \
-H "Content-Type: application/json" \
-H "X-Decube-Api-Key: $DECUBE_API_KEY" \
-d '{
"asset": { "type": "property", "id": 987 },
"test_type": "regex_match",
"mode": "on_demand",
"name": "email-format-validation",
"description": "Validate email field format using regex",
"notify": false,
"incident_level": "info",
"row_creation": {
"filter_mode": "all_records"
},
"threshold": {
"type": "value",
"value": "^[\\w.-]+@[\\w.-]+\\.[a-zA-Z]{2,}$"
},
"lookback_period": 7
}'Key Points:
frequencyis omitted for on-demand monitorsregex_matchusesvaluethreshold type with regex patternlookback_periodis required for on-demand monitors with all_records row creation
Advanced Examples
Example 4: Custom SQL Monitor
A scheduled monitor using custom SQL that defines its own pass/fail logic.
curl -X POST \
"https://connect.<REGION>.decube.io/api/v1/data/monitors" \
-H "Content-Type: application/json" \
-H "X-Decube-Api-Key: $DECUBE_API_KEY" \
-d '{
"asset": { "type": "source", "id": 1 },
"test_type": "custom_sql",
"mode": "scheduled",
"name": "custom-sql-quality-check",
"description": "Custom SQL monitor for data quality validation",
"notify": true,
"custom_alerts": {
"email": ["[email protected]"],
"slack": [],
"teams_webhook": []
},
"incident_level": "warning",
"frequency": {
"frequency": "1D",
"time_of_day": 2,
"timezone": "UTC"
},
"custom_sql": "SELECT CASE WHEN COUNT(*) > 100 THEN 1 ELSE 0 END as has_issues FROM {{table}} WHERE status NOT IN ('"'"'ok'"'"','"'"'pending'"'"')"
}'Key Points:
Custom SQL monitors don't use
row_creationorthresholdThe query itself contains the validation logic and thresholds
When submitting the custom_sql payload, ensure that the SQL is valid. If the SQL is invalid, the API will return an error.
Common Errors and Solutions
Missing Required Fields
❌ Incorrect: Missing row_creation for volume test
{
"asset": { "type": "dataset", "id": 123 },
"test_type": "volume",
"mode": "scheduled",
"name": "invalid-monitor",
"notify": true,
"incident_level": "warning",
"frequency": { "frequency": "1H" }
}✅ Correct: Include required row_creation
{
"asset": { "type": "dataset", "id": 123 },
"test_type": "volume",
"mode": "scheduled",
"name": "valid-monitor",
"notify": true,
"incident_level": "warning",
"frequency": { "frequency": "1H" },
"row_creation": { "filter_mode": "all_records" }
}Incorrect Frequency Format
❌ Incorrect: Daily frequency missing required fields
{
"frequency": { "frequency": "1D" }
}✅ Correct: Include time_of_day and timezone
{
"frequency": {
"frequency": "1D",
"time_of_day": 9,
"timezone": "UTC"
}
}Invalid Threshold Types
❌ Incorrect: Wrong threshold type for range-based test
{
"test_type": "average",
"threshold": { "type": "value", "value": "100" }
}✅ Correct: Use range threshold for numeric tests
{
"test_type": "average",
"threshold": { "type": "range", "min": 50, "max": 150 }
}Quick Reference
Required Fields for All Monitors
asset(with valid type and id)test_typemode("scheduled"or"on_demand")name(max 100 characters)notify(boolean)incident_level
Additional Required Fields by Mode
Scheduled:
frequencyOn-demand:
lookback_period(for timestamp/sql_expression row creation)
Row Creation Requirement by Test Type
Required: All field health tests, volume, freshness
Not applicable: Custom SQL, schema drift, job failure
Frequency Requirements
Hourly: Just
frequencyfieldDaily:
frequency,time_of_day,timezoneWeekly:
frequency,time_of_day,day_of_week,timezoneMonthly:
frequency,time_of_day,day_of_month,timezone
For more detailed validation rules, see the Monitor Validation & Constraints section.
Last updated