Examples of Monitor Creation
Practical examples for creating data quality monitors using the Decube Public API. These examples demonstrate proper payload structure and follow all validation rules.
This page provides comprehensive examples for creating monitors using the Decube Public API. All examples follow the validation rules and constraints documented in the Create, Manage & Delete guide.
Prerequisites
Authentication
All API requests require authentication via the X-Decube-Api-Key
header:
# Set your API key as an environment variable
export DECUBE_API_KEY="your-api-key-here"
# Include in curl requests
curl -H "X-Decube-Api-Key: $DECUBE_API_KEY" ...
Base URL
Replace <REGION>
with your Decube region (e.g., us
, eu
):
https://connect.<REGION>.decube.io/api/v1/data/monitors
Basic Examples
Example 1: Minimal Scheduled Volume Monitor
A basic scheduled monitor for dataset-level volume monitoring with hourly frequency.
curl -X POST \
"https://connect.<REGION>.decube.io/api/v1/data/monitors" \
-H "Content-Type: application/json" \
-H "X-Decube-Api-Key: $DECUBE_API_KEY" \
-d '{
"asset": { "type": "dataset", "id": 123 },
"test_type": "volume",
"mode": "scheduled",
"name": "hourly-volume-monitor",
"description": "Hourly volume monitoring for dataset 123",
"notify": true,
"incident_level": "warning",
"frequency": { "frequency": "1H" },
"row_creation": {
"filter_mode": "all_records"
}
}'
Expected Response:
{ "monitor_id": 1 }
Key Points:
row_creation
is required for volume testsHourly frequency doesn't require timezone or time_of_day
all_records
row creation mode processes the entire dataset
Example 2: Daily Freshness Monitor with Timezone
A scheduled freshness monitor running daily at a specific time with timestamp-based row creation.
curl -X POST \
"https://connect.<REGION>.decube.io/api/v1/data/monitors" \
-H "Content-Type: application/json" \
-H "X-Decube-Api-Key: $DECUBE_API_KEY" \
-d '{
"asset": { "type": "dataset", "id": 456 },
"test_type": "freshness",
"mode": "scheduled",
"name": "daily-freshness-monitor",
"description": "Daily freshness monitor running at 3:00 UTC",
"notify": true,
"custom_alerts": {
"email": ["[email protected]"],
"slack": [],
"teams_webhook": []
},
"incident_level": "critical",
"frequency": {
"frequency": "1D",
"time_of_day": 3,
"timezone": "UTC"
},
"row_creation": {
"filter_mode": "timestamp",
"metric_column_id": 789,
"enable_smart_training": true
}
}'
Key Points:
Daily frequency requires
time_of_day
(0-23) andtimezone
metric_column_id
must reference an existing timestamp/datetime columnenable_smart_training
enables automatic threshold learning
Example 3: On-Demand Field Health Monitor
An on-demand monitor for validating email format using regex matching.
curl -X POST \
"https://connect.<REGION>.decube.io/api/v1/data/monitors" \
-H "Content-Type: application/json" \
-H "X-Decube-Api-Key: $DECUBE_API_KEY" \
-d '{
"asset": { "type": "property", "id": 987 },
"test_type": "regex_match",
"mode": "on_demand",
"name": "email-format-validation",
"description": "Validate email field format using regex",
"notify": false,
"incident_level": "info",
"row_creation": {
"filter_mode": "all_records"
},
"threshold": {
"type": "value",
"value": "^[\\w.-]+@[\\w.-]+\\.[a-zA-Z]{2,}$"
},
"lookback_period": 7
}'
Key Points:
frequency
is omitted for on-demand monitorsregex_match
usesvalue
threshold type with regex patternlookback_period
is required for on-demand monitors with all_records row creation
Advanced Examples
Example 4: Custom SQL Monitor
A scheduled monitor using custom SQL that defines its own pass/fail logic.
curl -X POST \
"https://connect.<REGION>.decube.io/api/v1/data/monitors" \
-H "Content-Type: application/json" \
-H "X-Decube-Api-Key: $DECUBE_API_KEY" \
-d '{
"asset": { "type": "source", "id": 1 },
"test_type": "custom_sql",
"mode": "scheduled",
"name": "custom-sql-quality-check",
"description": "Custom SQL monitor for data quality validation",
"notify": true,
"custom_alerts": {
"email": ["[email protected]"],
"slack": [],
"teams_webhook": []
},
"incident_level": "warning",
"frequency": {
"frequency": "1D",
"time_of_day": 2,
"timezone": "UTC"
},
"custom_sql": "SELECT CASE WHEN COUNT(*) > 100 THEN 1 ELSE 0 END as has_issues FROM {{table}} WHERE status NOT IN ('"'"'ok'"'"','"'"'pending'"'"')"
}'
Key Points:
Custom SQL monitors don't use
row_creation
orthreshold
The query itself contains the validation logic and thresholds
When submitting the custom_sql payload, ensure that the SQL is valid. If the SQL is invalid, the API will return an error.
Common Errors and Solutions
Missing Required Fields
❌ Incorrect: Missing row_creation
for volume test
{
"asset": { "type": "dataset", "id": 123 },
"test_type": "volume",
"mode": "scheduled",
"name": "invalid-monitor",
"notify": true,
"incident_level": "warning",
"frequency": { "frequency": "1H" }
}
✅ Correct: Include required row_creation
{
"asset": { "type": "dataset", "id": 123 },
"test_type": "volume",
"mode": "scheduled",
"name": "valid-monitor",
"notify": true,
"incident_level": "warning",
"frequency": { "frequency": "1H" },
"row_creation": { "filter_mode": "all_records" }
}
Incorrect Frequency Format
❌ Incorrect: Daily frequency missing required fields
{
"frequency": { "frequency": "1D" }
}
✅ Correct: Include time_of_day
and timezone
{
"frequency": {
"frequency": "1D",
"time_of_day": 9,
"timezone": "UTC"
}
}
Invalid Threshold Types
❌ Incorrect: Wrong threshold type for range-based test
{
"test_type": "average",
"threshold": { "type": "value", "value": "100" }
}
✅ Correct: Use range threshold for numeric tests
{
"test_type": "average",
"threshold": { "type": "range", "min": 50, "max": 150 }
}
Quick Reference
Required Fields for All Monitors
asset
(with valid type and id)test_type
mode
("scheduled"
or"on_demand"
)name
(max 100 characters)notify
(boolean)incident_level
Additional Required Fields by Mode
Scheduled:
frequency
On-demand:
lookback_period
(for timestamp/sql_expression row creation)
Row Creation Requirement by Test Type
Required: All field health tests, volume, freshness
Not applicable: Custom SQL, schema drift, job failure
Frequency Requirements
Hourly: Just
frequency
fieldDaily:
frequency
,time_of_day
,timezone
Weekly:
frequency
,time_of_day
,day_of_week
,timezone
Monthly:
frequency
,time_of_day
,day_of_month
,timezone
For more detailed validation rules, see the Monitor Validation & Constraints section.
Last updated