Retraining Monitors
Which monitor configuration changes trigger a retrain, what data gets deleted when a retrain runs, and how to reconfigure monitors safely.
When you modify certain settings on an existing monitor, Decube discards the monitor's historical data and retrains the ML model from scratch. Understanding which changes trigger this — and what gets deleted — helps you avoid unintended data loss.
A retrain deletes all historical test results, metrics, cases, and incident history for the monitor, including any open incidents. Note open incidents before reconfiguring a monitor if you need to preserve that information.
What triggers a retrain
The following configuration changes trigger an automatic retrain:
Row creation mode (Timestamp / SQL Expression / All Records)
Changing mode resets the data collection baseline
Timestamp column
Changing the column changes what the model is trained on
SQL Expression value
Any change to the expression text
Smart Training toggled on
Enabling Smart Training after creation starts a fresh training run
Scan frequency
Changing frequency changes the lookback window (see How Anomaly Detection Works)
Timezone
Changes the alignment of scan windows
Time of day
Changes when scans run
Day of week
Applies to weekly-frequency monitors
Day of month
Applies to monthly-frequency monitors
What gets deleted during a retrain
When a retrain is triggered, Decube permanently deletes the following for that monitor:
All historical test results and metric values
All cases associated with past scan runs
All incident history
All open incidents — these are closed and deleted, not resolved
There is no recovery path for deleted incident history. If you need to retain a record of open incidents, resolve or export them before making any of the changes listed above.
What does not trigger a retrain
The following changes are safe to make without triggering a retrain or losing any data:
Monitor name
Yes
Monitor description
Yes
Notification channels
Yes
Incident level
Yes
Threshold bounds (Absolute, Percentage, Positive Range, Any Range)
Yes
Operational guidance
Before reconfiguring a monitor that uses Smart Training:
Go to All Monitors and filter to the monitor you plan to change.
Review any open incidents and take note of them, or resolve them if they are addressed.
Make your configuration change. Decube begins retraining automatically.
The monitor enters a training period during which no new incidents are generated. Check the training duration for your scan frequency to know when to expect incidents to resume.
If you need to change a setting that triggers a retrain on a high-volume production table, consider the timing carefully. The monitor will be inactive during the training period, which can range from hours to days depending on the scan frequency.
Settings that cannot be changed after creation
Some settings are locked after a monitor is created. To change them, you must delete the monitor and create a new one:
Asset (source, schema, dataset)
Test type (for example, you cannot change a Freshness monitor into a Volume monitor)
Monitor mode (Scheduled vs On-Demand)
Group by configuration
Last updated