Monitor Configuration Reference

A reference for every setting available when creating or editing a monitor, including which settings are locked after creation.

This page describes every setting available in the monitor creation and edit forms. Use it to understand what each setting does, what values are valid, and which settings you cannot change after a monitor is created.

Settings locked after creation

The following settings cannot be changed on an existing monitor. To change them, delete the monitor and create a new one.

Setting
Why it's locked

Asset (Source, Schema, Dataset)

The monitor is bound to a specific table. Changing the asset would invalidate all historical data.

Test type

Each test type measures a different metric. Changing it would make historical comparisons meaningless.

Monitor mode (Scheduled / On-Demand)

The training approach and data collection method differ fundamentally between modes.

Group by

Group-by configuration determines how the monitor partitions its data at creation time.


All settings

Asset

The table the monitor runs against. You select a Source, optionally filter by Schema, then select a Dataset (table). This setting is locked after creation.


Test type

The metric the monitor measures (for example, Null, Freshness, Volume, Cardinality). Each test type has a fixed set of compatible threshold types and modes. Test type is locked after creation.

See Set Up Field Health Monitors for the full list of Field Health test types and their compatibility.


Monitor mode

Whether the monitor runs on a schedule (Scheduled) or only when manually triggered (On-Demand). Monitor mode is locked after creation.

Mode
When to use

Scheduled

Production monitoring, continuous coverage, Smart Training

On-Demand

Ad-hoc validation, development and testing, manual runs

On-Demand mode does not support Smart Training or Group By. See Available Monitor Modes for the full comparison.


Row filtering

Controls which rows Decube considers when calculating the monitored metric. Three options are available:

Option
Behaviour

Timestamp

Decube uses the selected timestamp column to identify rows added since the last scan. This is the most accurate option for incremental tables.

SQL Expression

You provide an expression (in your data source's SQL dialect) that produces a timestamp. Use this when timestamps are stored in non-standard formats (strings, Unix epoch, split date/time columns). Validating the expression before saving is required.

All Records

Decube scans the entire table on every run. No timestamp column is needed, but this mode cannot track increments reliably on tables where rows are deleted.

Changing the row filtering mode (or the timestamp column / SQL expression) after creation triggers a retrain and deletes all historical data. See Retraining Monitors.


Smart Training

When enabled, Decube trains an ML model on historical data to establish a dynamic confidence interval. Smart Training requires:

  • A timestamp-based row filtering mode (Timestamp or SQL Expression)

  • Scheduled monitor mode (not On-Demand)

When Smart Training is enabled, the Lookback Period becomes configurable and a training period begins. See How Anomaly Detection Works for details on the training process, historical lookback windows, and sparse-data behaviour.

Toggling Smart Training on after creation triggers a retrain.


Scan frequency

How often a Scheduled monitor runs. Options range from hourly to monthly. The frequency you choose affects how much historical data the ML model collects during training.

See Custom Scheduling for Monitors for all available frequency options.


Threshold type

The method used to evaluate whether a scan result is anomalous. Four types are available:

Threshold type
How it works
Compatible test types

Absolute

Compares the raw row count of failing records against a min/max bound

Null, Unique, Email, UUID, Regex Match

Percentage

Compares the percentage of failing records against a min/max bound (0–100)

Null, Unique, Email, UUID, Regex Match

Positive Range

Compares a numeric metric against a min/max bound (values ≥ 0)

Average, Min, Max, String Length

Any Range

Compares a numeric metric against a min/max bound (negative values allowed)

Average, Min, Max

Auto

ML model sets the bounds dynamically based on historical data. Scheduled mode only.

Null, Unique, Email, UUID, Regex Match, Average, Min, Max, String Length

At least one bound (min or max) is required for manual threshold types. Leave the other bound empty to create a one-sided check (e.g., max only = "alert if the value exceeds X").

Threshold type is safe to change after creation — it does not trigger a retrain.


Threshold bounds

The min and/or max values that define the acceptable range for the monitored metric. Bounds are always interpreted relative to the selected threshold type:

  • For Absolute: non-negative integers (row counts)

  • For Percentage: integers between 0 and 100

  • For Positive Range: non-negative numeric values

  • For Any Range: any numeric value including negatives

Threshold bounds are safe to change after creation — they do not trigger a retrain.


Group by

Splits the monitor into one sub-monitor per distinct value in the selected column. Each sub-monitor tracks its own metric and opens incidents independently.

Group by is locked after creation and is not available in On-Demand mode.


Lookback period

Available on On-Demand monitors only. Defines how far back in time the monitor scans when it runs. For example, a lookback period of 7 days checks for qualifying rows from the past 7 days.

On Scheduled monitors with Smart Training enabled, the lookback period is set automatically based on scan frequency — see the historical lookback table.


Incident level

The severity assigned to incidents this monitor opens. Options are typically Critical, High, Medium, and Low. Incident level affects notification routing if you have level-based alert rules configured.

Incident level is safe to change after creation — it does not trigger a retrain.


Notifications

Controls which channels receive alerts when this monitor opens an incident. Toggle Notify default channel to enable custom routing, then add email addresses or Slack channel names.

If left off, incidents are still created and visible in the Data Quality module but no external notification is sent.

Notification settings are safe to change after creation — they do not trigger a retrain.


Quality Dimension

An optional classification that maps the monitor to a data quality dimension (for example, Completeness, Validity, Timeliness). Dimension affects how incidents are grouped in DQ score reporting.

Quality Dimension is safe to change after creation.


Name and Description

A human-readable name and optional description for the monitor. You can create multiple monitors of the same test type on the same column — use names to distinguish them.

Both fields are safe to change after creation — they do not trigger a retrain.

Last updated