Freshness (SLA and Staleness)

Freshness monitors check whether a table's data is as up to date as expected.

There are two variants — SLA and Staleness.

• Freshness – SLA — did the data arrive by a specific deadline? Typically tied to a scheduled job that's expected to finish by a certain time.

• Freshness – Staleness — has the data continued to update regularly, without going stale? Typically a recurring health check on an ongoing data flow.

Roadmap

• Staleness is planned to get automatic thresholding.

• SLA is planned to get the ability to exclude certain days like weekends.

When to use it

• SLA — A nightly ETL job is expected to finish loading the daily_sales reporting table by 2:00 AM. A team sets an SLA check to run at 2:30 AM, confirming the table was updated within the last 30 minutes — so if the load ran late or failed, the data team is alerted before business hours.

• Staleness — A clickstream_events table is streamed into continuously throughout the day and should never go quiet. A team runs an hourly staleness check to confirm the most recent record is no more than an hour old, catching stalled ingestion before downstream dashboards go stale.

What's configurable

• Schedule

  • SLA must run on a fixed recurring schedule since it needs a fixed time to measure whether the data was ready by a specific point.

  • Staleness can run on a fixed recurring schedule, or be set to Manual (triggered on demand through the UI or the API).

• Detection Method — how the monitor figures out when the data was last written:

  • Metadata-based — uses the data source's own record of when the table was last modified. Not available for every source or table type (some views don't expose this).

  • Column-based — you pick a timestamp column; the monitor uses the maximum value in that column as the last-write time. If the column doesn't store timezone information (for example, Snowflake TIMESTAMP_NTZ), you also set the UTC offset its values are written in, so Decube can compute the age correctly. See Column timezone.

• Threshold — a Duration (a number and unit: minutes, hours, or days). This is the maximum acceptable age of the data. The check fails if the computed age is greater than this duration.

Worked example — Freshness SLA

Example of incident creation

• Asset: daily_sales table (Snowflake)

• Detection Method: Column-based, timestamp column = updated_at

• Schedule: Daily, 2:30 AM (Asia/Kuala_Lumpur)

• Threshold: Duration — 30 minutes

Resulting rule (shown to you in-product): "An incident will be created when the data age is above 30 minutes."

What happens when it runs: at 2:30 AM, the monitor checks the maximum value of updated_at. If the nightly load finished at 2:05 AM, the data is 25 minutes old — passes. If the load stalled and the last row was written at 1:45 AM, the data is 45 minutes old — fails, and Decube raises an incident.

Worked example — Freshness Staleness

• Asset: clickstream_events table

• Detection Method: Metadata-based (uses the source's own last-modified time)

• Schedule: Every hour

• Threshold: Duration — 1 hour

Resulting rule: "An incident will be created when the data age is above 1 hour."

Notes

• If the monitor computes a negative age — this can happen because the check actually executes slightly after its scheduled time, and new data arrived in between — the age is simply treated as 0. This is expected behavior, not a failure, so don't be alarmed if you see it in evidence or logs.

• Changing the schedule or detection method later is a destructive change (it creates a new monitor version). Adjusting only the threshold value is not.