Retraining Monitors

Which monitor configuration changes trigger a retrain, what data gets deleted when a retrain runs, and how to reconfigure monitors safely.

When you modify certain settings on an existing monitor, Decube discards the monitor's historical data and retrains the ML model from scratch. Understanding which changes trigger this — and what gets deleted — helps you avoid unintended data loss.

What triggers a retrain

The following configuration changes trigger an automatic retrain:

Setting changed
Notes

Row creation mode (Timestamp / SQL Expression / All Records)

Changing mode resets the data collection baseline

Timestamp column

Changing the column changes what the model is trained on

SQL Expression value

Any change to the expression text

Smart Training toggled on

Enabling Smart Training after creation starts a fresh training run

Scan frequency

Changing frequency changes the lookback window (see How Anomaly Detection Works)

Timezone

Changes the alignment of scan windows

Time of day

Changes when scans run

Day of week

Applies to weekly-frequency monitors

Day of month

Applies to monthly-frequency monitors

What gets deleted during a retrain

When a retrain is triggered, Decube permanently deletes the following for that monitor:

  • All historical test results and metric values

  • All cases associated with past scan runs

  • All incident history

  • All open incidents — these are closed and deleted, not resolved

There is no recovery path for deleted incident history. If you need to retain a record of open incidents, resolve or export them before making any of the changes listed above.

What does not trigger a retrain

The following changes are safe to make without triggering a retrain or losing any data:

Setting
Safe to change

Monitor name

Yes

Monitor description

Yes

Notification channels

Yes

Incident level

Yes

Threshold bounds (Absolute, Percentage, Positive Range, Any Range)

Yes

Operational guidance

Before reconfiguring a monitor that uses Smart Training:

  1. Go to All Monitors and filter to the monitor you plan to change.

  2. Review any open incidents and take note of them, or resolve them if they are addressed.

  3. Make your configuration change. Decube begins retraining automatically.

  4. The monitor enters a training period during which no new incidents are generated. Check the training duration for your scan frequency to know when to expect incidents to resume.

If you need to change a setting that triggers a retrain on a high-volume production table, consider the timing carefully. The monitor will be inactive during the training period, which can range from hours to days depending on the scan frequency.

Settings that cannot be changed after creation

Some settings are locked after a monitor is created. To change them, you must delete the monitor and create a new one:

  • Asset (source, schema, dataset)

  • Test type (for example, you cannot change a Freshness monitor into a Volume monitor)

  • Monitor mode (Scheduled vs On-Demand)

  • Group by configuration

Last updated