Create, Manage & Delete
Programmatically manage data quality monitors for your assets so you can monitor various aspects of data quality including freshness, volume, field health and custom rules.
Creates a new monitor with the provided configuration. Returns the created monitor ID.
Successful Response
Validation Error
POST /api/v1/data/monitors HTTP/1.1
Host: connect.<REGION>.decube.io
Content-Type: application/json
Accept: */*
Content-Length: 504
{
"asset": {
"type": "text",
"id": 1
},
"test_type": "volume",
"mode": "scheduled",
"name": "text",
"description": "text",
"notify": true,
"custom_alerts": {
"email": [],
"slack": [
"text"
],
"teams_webhook": []
},
"incident_level": "info",
"dimension": "accuracy",
"row_creation": {
"filter_mode": "text",
"sql_expression": "text",
"enable_smart_training": true
},
"frequency": {
"frequency": "1H"
},
"lookback_period": 1,
"custom_sql": "text",
"threshold": {
"type": "text",
"min": 1,
"max": 1
},
"grouping": {
"group_by": {
"type": "text",
"id": 1
},
"groups": [
"text"
]
}
}{
"monitor_id": 1
}Updates an existing monitor (excluding Schema Drift and Job Failure monitors). Requires full monitor configuration in the request body. Returns the updated monitor ID.
Successful Response
Validation Error
PUT /api/v1/data/monitors HTTP/1.1
Host: connect.<REGION>.decube.io
Content-Type: application/json
Accept: */*
Content-Length: 393
{
"id": 1,
"name": "text",
"description": "text",
"notify": true,
"custom_alerts": {
"email": [],
"slack": [
"text"
],
"teams_webhook": []
},
"incident_level": "info",
"dimension": "accuracy",
"row_creation": {
"filter_mode": "text",
"sql_expression": "text",
"enable_smart_training": true
},
"frequency": {
"frequency": "1H"
},
"lookback_period": 1,
"custom_sql": "text",
"threshold": {
"type": "text",
"min": 1,
"max": 1
},
"groups": [
"text"
]
}{
"monitor_id": 1
}Delete a monitor by ID.
Unique identifier of the monitor
Successful Response
Validation Error
DELETE /api/v1/data/monitors?monitor_id=1 HTTP/1.1
Host: connect.<REGION>.decube.io
Accept: */*
No content
Monitor Validation & Constraints
This section documents the request model, validation rules, and conditional requirements for creating and updating monitors. It is important to understand these rules to ensure that the monitors are created and updated correctly.
Monitorable assets
Allowed asset types for monitors are restricted to: Source, Dataset/Data Asset, Collection, Property/Data Asset Property, Data Job.
The
assetmust exist and be active (not deleted) in your organization.
Asset types by test
The following table maps monitor test types to the primary asset types they support. Use this as a quick reference when creating or updating monitors.
Freshness
dataset
yes
Volume
dataset
yes
Field Health
property
yes
Custom SQL
source
yes
Schema Drift
collection
no
Job Failure
data_job
no
Note: "create_supported" indicates whether the API supports creating a monitor of this test type. "no" means there is no create option via the monitors create endpoint for that test type; these tests are already created upon source connection and only can be updated.
Supported test types by monitor mode
The following table shows which test types are available for each monitor mode:
Freshness
✅
✅
Dataset-level monitoring
Volume
✅
✅
Dataset-level monitoring
Field Health Tests
Average
✅
✅
Numeric columns only
Min
✅
✅
Numeric columns only
Max
✅
✅
Numeric columns only
Sum
✅
✅
Numeric columns only
Range
✅
✅
Numeric columns only
Not Null
✅
✅
All data types
Null %
✅
✅
All data types
Is Unique
✅
✅
All data types
Unique %
✅
✅
All data types
String Length
✅
✅
String columns only
Is Email
✅
✅
String columns only
Is UUID
✅
✅
String columns only
Value is
✅
✅
String columns only
Value in
✅
✅
String columns only
Regex Match
✅
✅
String columns only
Date in Past
✅
✅
Date/datetime columns only
Date in Future
✅
✅
Date/datetime columns only
%Negative
✅
✅
Numeric columns only
%Zero
✅
✅
Numeric columns only
Cardinality
❌
✅
Special restrictions apply
Other Test Types
Custom SQL
✅
✅
Custom query validation required
Schema Drift
❌
✅
Table/Column changes only
Job Failure
❌
✅
Data job monitoring only
Test type capabilities and features
The following table shows which features are supported for each test type across different monitor modes:
Dataset-Level Tests
Freshness
✅ (Scheduled only)
✅
❌
✅
Volume
✅ (Scheduled only)
✅
❌
✅
Field Health Tests
Average
✅ (Scheduled only)
✅
❌
✅
Min
✅ (Scheduled only)
✅
❌
✅
Max
✅ (Scheduled only)
✅
❌
✅
Sum
✅ (Scheduled only)
✅
❌
✅
Range
✅ (Scheduled only)
✅
❌
✅
Not Null
✅ (Scheduled only)
❌
✅
✅
Null %
✅ (Scheduled only)
✅
❌
✅
Is Unique
✅ (Scheduled only)
❌
✅
✅
Unique %
✅ (Scheduled only)
✅
❌
✅
String Length
✅ (Scheduled only)
✅
❌
✅
Is Email
✅ (Scheduled only)
❌
✅
✅
Is UUID
✅ (Scheduled only)
❌
✅
✅
Value is
✅ (Scheduled only)
❌
✅
✅
Value in
✅ (Scheduled only)
❌
✅
✅
Regex Match
✅ (Scheduled only)
❌
✅
✅
Date in Past
✅ (Scheduled only)
❌
✅
✅
Date in Future
✅ (Scheduled only)
❌
✅
✅
%Negative
✅ (Scheduled only)
✅
❌
✅
%Zero
✅ (Scheduled only)
✅
❌
✅
Cardinality
❌
❌
❌
✅
Other Test Types
Custom SQL
✅
❌
❌
❌
Schema Drift
❌
❌
❌
❌
Job Failure
❌
❌
❌
❌
Mode and scheduling rules
modedetermines whether a monitor is scheduled or on-demand.modedetermines whether a monitor is scheduled or on-demand. Use the exact API token values for this field:scheduledoron_demand.Scheduled monitors:
frequencyis required.
On-demand monitors:
frequencymust be omitted.Only a subset of
test_typevalues are allowed (see “On-demand compatible tests”).
Frequency shapes and constraints
This controls how often a scheduled monitor runs. The available frequency options depend on the selected row creation mode.
Frequency availability by row creation mode:
Timestamp (datetime column)
✅
✅
✅
✅
Timestamp (date column)
❌
✅
✅
✅
SQL Expression
✅
✅
✅
✅
All Records
✅
✅
✅
✅
Frequency object shapes:
Hourly Frequency
{
"frequency": "1H" | "3H" | "6H" | "12H"
}Must NOT include timezone or time-of-day fields
Not available when using date columns (only datetime)
Daily Frequency
{
"frequency": "1D",
"time_of_day": <integer>,
"timezone": "<string>"
}Requires
time_of_day(hour of day, 0-23) andtimezone
Weekly Frequency
{
"frequency": "7D",
"time_of_day": <integer>,
"day_of_week": <integer>,
"timezone": "<string>"
}Requires
time_of_day,day_of_week(0=Sunday, 6=Saturday), andtimezone
Monthly Frequency
{
"frequency": "MS",
"time_of_day": <integer>,
"day_of_month": <integer>,
"timezone": "<string>"
}Requires
time_of_day,day_of_month, andtimezoneday_of_month: provide integer values 1-28 for monitor to be run on these dates of the month.For monitor to run on every last day of the month, use
"day_of_month": 31
Additional validation rules:
timezonemust be a valid IANA time zone (e.g.,UTC,America/New_York)day_of_weekis only valid for weekly frequency; providing it for other frequencies is rejectedday_of_monthis only valid for monthly frequency; providing it for other frequencies is rejectedtime_of_dayhour range is validated by the server (0-23)
Row creation modes and constraints
row_creation controls how rows are selected for computing metrics/tests. This setting is required for most test types and has different availability based on the monitor mode and test type.
Row creation mode availability
Dataset-Level Tests
Freshness
On-demand
✅
✅
❌
Requires time-based filtering
Freshness
Scheduled
✅
✅
✅
All modes supported
Volume
On-demand
✅
✅
✅
All modes supported
Volume
Scheduled
✅
✅
✅
All modes supported
Field Health Tests
All field health
On-demand
✅
✅
✅
All modes supported
All field health
Scheduled
✅
✅
✅
All modes supported
Other Test Types
Custom SQL
On-demand
❌
❌
❌
Row creation not applicable
Custom SQL
Scheduled
❌
❌
❌
Row creation not applicable
Schema Drift
Scheduled
❌
❌
❌
Row creation not applicable
Job Failure
Scheduled
❌
❌
❌
Row creation not applicable
Row creation mode specifications
AllRecordsMode
{
"filter_mode": "all_records"
}Use all rows in the dataset
No additional fields required
Smart training is not available for this mode.
TimestampMode
{
"filter_mode": "timestamp",
"metric_column_id": <number>,
"enable_smart_training": <boolean> // optional
}Requires
metric_column_idreferencing a timestamp/datetime columnColumn must exist on the target asset and be timestamp-like data type
Optionally supports
enable_smart_training
SqlExpressionMode
{
"filter_mode": "sql_expression",
"sql_expression": "<string>",
"enable_smart_training": <boolean> // optional
}Requires
sql_expression(a boolean filter SQL expression)Expression is validated for syntax against the underlying source via connector
Invalid SQL will be rejected with timeout/error if source cannot be reached
Optionally supports
enable_smart_training
Lookback period constraints
Timestamp
✅
Must be > 0
SQL Expression
✅
Must be > 0
All Records
❌
-
For on-demand monitors, lookback period behavior depends on the row creation mode:
Note: row_creation enum values are case-sensitive; use lowercase values as shown above.
Grouping and group mapping
This allows monitors to be run on specific segments within a table (e.g., by values in a dimension column). See Set Up Grouped-by Monitors for UI setup details.
is_group_by = trueenables grouping; otherwise grouping-related fields are ignored.group_byidentifies the asset to group by (must be compatible with the target asset and monitor setting).groupsis an optional allowlist of specific group values to monitor. If provided, it must not exceed the server’s limit (current max is 100 distinct values).
Custom SQL grouping
This allows
group_mappingis optional and allows mapping the raw grouping values to business labels:Fields:
mapping_column_id(column containing raw values),group_by_field_name(target label field name).
Use the
GET /monitors/distinct_groups?column_id=...helper to discover distinct group values for a column.
Alerts and notifications
notifycontrols whether alerts are sent. Ifnotify = false, anycustom_alertsare ignored.custom_alertssupports channels:email: list of addressesslack: list of channel IDs or webhook URLs (org-config dependent)teams_webhook: list of webhook URLs
If
notify = trueandcustom_alertsis provided, channel values must follow the organization’s connector configuration and be valid.
Thresholds: validation rules and types by test type
Thresholds define the acceptable range or values for a test to pass. The shape and validation rules of the threshold depend on the test_type. "auto_threshold" can be used where supported to let the monitor automatically determine thresholds based on historical data.
Threshold types and validation rules
No Threshold Required
Not Null
N/A
No threshold accepted
Automatically fails when null values found
Is Unique
N/A
No threshold accepted
Automatically fails when duplicates found
Is Email
N/A
No threshold accepted
Validates email format
Is UUID
N/A
No threshold accepted
Validates UUID format
Date in Past
N/A
No threshold accepted
Validates dates are in the past
Date in Future
N/A
No threshold accepted
Validates dates are in the future
Cardinality
N/A
No threshold accepted
Counts distinct values
Freshness
N/A
Auto-learned
Uses historical patterns
Volume (Scheduled)
N/A
Auto-learned
Uses historical patterns
Custom SQL
N/A
Query-dependent
Depends on SQL logic
Schema Drift
N/A
No threshold accepted
Detects schema changes
Job Failure
N/A
No threshold accepted
Monitors job status
Range Thresholds
Average
RangeThreshold
-∞ ≤ min ≤ max ≤ ∞ max > min At least one bound required
Accepts any numeric value
Min
RangeThreshold
-∞ ≤ min ≤ max ≤ ∞ max > min At least one bound required
Accepts any numeric value
Max
RangeThreshold
-∞ ≤ min ≤ max ≤ ∞ max > min At least one bound required
Accepts any numeric value
Sum
RangeThreshold
-∞ ≤ min ≤ max ≤ ∞ max > min At least one bound required
Accepts any numeric value
Range
RangeThreshold
-∞ ≤ min ≤ max ≤ ∞ max > min At least one bound required
Accepts any numeric value
String Length
RangeThreshold
0 ≤ min ≤ max ≤ ∞ max > min At least one bound required
Only positive integers
Volume (On-demand)
RangeThreshold
0 ≤ min ≤ max ≤ ∞ max > min At least one bound required
Only positive integers
Percentage Thresholds
Null %
RangeThreshold
0 ≤ min ≤ max ≤ 100 max > min At least one bound required
Integer values only
Unique %
RangeThreshold
0 ≤ min ≤ max ≤ 100 max > min At least one bound required
Integer values only
%Negative
RangeThreshold
0 ≤ min ≤ max ≤ 100 max > min At least one bound required
Integer values only
%Zero
RangeThreshold
0 ≤ min ≤ max ≤ 100 max > min At least one bound required
Integer values only
Value Thresholds
Value is
ValueThreshold
Single exact value
String validation
Regex Match
ValueThreshold
Valid regex pattern
Regex syntax validated
Values Thresholds
Value in
ValuesThreshold
Non-empty list of values
Multiple string values
Threshold object shapes
The API accepts four discriminated threshold object shapes. Each shape includes a required type field whose value is a literal constant:
RangeThreshold
{
"type": "range",
"min": <integer|null>,
"max": <integer|null>
}minandmaxmay benullto indicate an open boundUse
nullfor one-sided bounds (e.g.,{ "min": 10, "max": null })
ValueThreshold
{
"type": "value",
"value": "<string>"
}Used for exact-value checks and regex patterns
For
regex_match,valuecontains the regex string
ValuesThreshold
{
"type": "values",
"values": ["a", "b", "c"]
}Used for
value_intests where multiple accepted values are required
AutoThreshold
{
"type": "auto"
}Available for tests that support machine learning
Server auto-learns thresholds from historical data
Test-type specific notes
Schema Drift tests (
table_addition,table_deletion,column_addition,column_deletion,column_type_change) and Job Failure tests have their own specialized behavior and do not use thresholds, grouping, or row creation like field-health testsFreshness and Volume monitors (dataset-level) use auto-learned thresholds in scheduled mode and do not accept manual thresholds
Custom SQL monitors require query validation before submission
Names and descriptions
name: required, max 100 characters.description: optional, max 1000 characters.Name uniqueness is enforced within an asset/test-type/mode combination; if violated, requests will be rejected.
Time zones
timezonemust be a valid IANA time zone. Invalid zones are rejected.
Appendix: Quick Field reference
Frequency unions:
Hourly: {"frequency": "1H|3H|6H|12H"}
Daily: {"frequency": "1D", "time_of_day": int, "timezone": string}
Weekly: {"frequency": "7D", "time_of_day": int, "day_of_week": int, "timezone": string}
Monthly: {"frequency": "MS", "time_of_day": int, "day_of_month": int, "timezone": string}
acceptable values for
day_of_month: 1-28, 31
Row creation unions (literal
filter_modevalues):All: { "filter_mode": "all_records" }
Timestamp: { "filter_mode": "timestamp", "metric_column_id": number, "enable_smart_training"?: boolean }
SQL expression: { "filter_mode": "sql_expression", "sql_expression": string, "enable_smart_training"?: boolean }
Thresholds:
RangeThreshold:
{ "type": "range", "min": <integer|null>, "max": <integer|null> }ValueThreshold:
{ "type": "value", "value": "<string>" }ValuesThreshold:
{ "type": "values", "values": ["<string>"] }AutoThreshold:
{ "type": "auto" }
Last updated