> For the complete documentation index, see [llms.txt](https://docs.decube.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.decube.io/data-quality-v2/monitors/building-blocks.md). # Building Blocks A handful of concepts reappear across almost every monitor type in Decube's Data Quality module. This page explains each one once, in plain language, so you don't need it re-explained on every monitor type page that follows. {% embed url="" %} ### Schedule **How often does this monitor run?**

Every monitor runs on a schedule. You can choose from these frequencies: every 30 minutes, every hour, every 3 hours, every 6 hours, every 12 hours, daily, weekly, first day of the month, last day of the month, monthly, or **Manual**. Pick the frequency to match how often the underlying data actually changes. * A team monitoring a continuously updating `clickstream_events` table might run a check every 30 minutes to catch a stalled pipeline fast. * A team monitoring a table that only closes once a month, like a finance close table, only needs the last day of the month schedule — anything more frequent just adds noise. For any schedule other than Manual, you also set a start time and timezone for daily, weekly, and monthly-style schedules. Monitors run in fixed 30-minute slots — for example 09:00, 09:30, 10:00. #### Named timezone vs. fixed offset

Timezone type selection to choose between a named timezone or UTC offset

* Choosing a named timezone (for example, Asia/Singapore) means the monitor always runs at the same local clock time, and **automatically adjusts for daylight saving** where applicable. * Choosing a fixed UTC offset (for example, +08:00) means the monitor always runs at the same point relative to UTC, regardless of any local daylight saving changes. For example: * A governance lead in Kuala Lumpur checking that a daily sales report loaded before the business day starts should pick the named timezone Asia/Kuala\_Lumpur, so the check always runs at 8:00 AM local time. * A compliance team that needs a check to run at exactly 00:00 UTC every day, regardless of any region's daylight saving changes, should choose a fixed offset instead. Most users should pick their local named timezone — the fixed-offset option covers edge cases like this one. #### Manual schedule Manual means the monitor doesn't run on any recurring schedule — it only runs when you click **Run now** or trigger it through the API. A manual run behaves exactly like a scheduled run in every other way: Decube still evaluates it, it can still raise an incident, and it still triggers alerts. Not every monitor type supports Manual scheduling: | Monitor type | Manual schedule available? | | ----------------------- | -------------------------------------------------------------------------- | | Freshness – Staleness | Yes | | Freshness – SLA | No — needs a fixed recurring schedule | | Volume | No — needs a fixed recurring schedule, to compare against the prior period | | All Field Health checks | Yes | | Custom SQL | Yes | Manual is useful while you're still tuning a monitor — for example: * a data steward testing a new Null check on a staging table might run it on demand until they're confident in the configuration, then switch it to a recurring schedule once it's live. * It's also a good fit for a table that's only refreshed occasionally by an ad hoc process, like a one-time data migration. If you set a Field Health monitor's schedule to Manual, it must also scan the full table each time it runs, rather than only a recent slice of it — see Scan Method below. ### Detection Method **How does the monitor know when the data last changed, or how many rows there are?**

Detection Method applies to Freshness and Volume monitors. * **Metadata-based detection (Reported by source)** — the monitor asks the data source itself for this information, such as the warehouse's own record of when a table was last modified, or its own row count. You don't need to pick a column. This isn't available for every table — some data sources or table types, like views, don't expose this metadata, so Decube won't offer metadata-based detection as an option for those. * **Column-based detection** — you pick a specific timestamp column on the table, and the monitor works out freshness or growth from the values in that column, such as the most recent timestamp, or how many rows fall in a given time window. This requires selecting the column. If the column doesn't store timezone information, you also set its timezone — see [Column timezone](#column-timezone). #### Deciding between detection methods * Metadata-based detection suits a straightforward case: you just want to know when a Snowflake table was last written to, and its own last-modified timestamp is good enough to catch a stalled nightly load. * Column-based detection suits cases where the warehouse's technical write time doesn't reflect what you actually care about — for example, an `orders` table loaded through CDC might show a recent write time even though the underlying order itself is old, so you'd pick the `order_date` column instead to track business freshness rather than load freshness. * Volume has a third option specific to it: **full table count**, which simply runs a count on the table each time. Decube compares this against the previous run's count to work out growth. This is the fallback when a source doesn't expose reliable row-count metadata — for example, some file-based or cloud storage sources — so Decube counts the rows directly instead. #### Column timezone Decube calculates every freshness age and volume window in UTC, so it needs to know exactly which moment each value in your timestamp column represents. Whether you see the timezone field depends on the column's data type: * **Columns that store timezone information** — Decube reads the offset from each value and converts it to UTC. The timezone field doesn't appear. * **Columns that don't store timezone information** — for example, Snowflake `TIMESTAMP_NTZ` — the timezone field appears and is required. Set it to the UTC offset your pipeline writes the values in. A value such as `2026-10-15 08:00:00` with no offset could mean 08:00 UTC or 08:00 in Singapore, eight hours apart. Decube doesn't guess, because a wrong assumption shifts every calculation by the size of the error: * **Freshness** reports the data as older or newer than it really is, which raises false incidents or hides a real delay. * **Volume** counts rows in the wrong time window, so growth is measured against the wrong period. The column timezone is always a fixed UTC offset, such as `+08:00`. It describes how the stored values are written, and it's separate from the schedule timezone, which only controls when the monitor runs. {% hint style="success" %} If your pipeline writes timestamps in UTC, select `+00:00`. Decube then reads the values exactly as stored, with no conversion. {% endhint %} ### Scan method (Records to check) **How much of the table does the monitor look at each time it runs?**

Scan method applies to Field Health checks: Average, Minimum, Maximum, Sum, Numerical Range, String Length, Regex Match, Regex Not Match, Value In, Value Not In, Null, Unique, Zero, Is Before, and Is After. * **Full Table Scan** — every run scans the entire table. * **Incremental Scan** — every run only scans the slice of the table added since the last run, based on either a timestamp column or a custom filter expression you provide. If you scan by a timestamp column that doesn't store timezone information, you also set its [column timezone](#column-timezone). This is efficient for large, continuously growing tables where you only care about newly arrived data. #### Deciding between scan methods * Full Table Scan works well on a small reference table — for example, a `countries` lookup table — where scanning everything each run is cheap. * Incremental Scan is built for a large, continuously growing table — for example, a `clickstream_events` table with billions of rows — where re-scanning the whole table every run would be slow and expensive when you only care about validating what's new. A Manual-frequency Field Health monitor must use Full Table Scan — there's no "since last run" window to compare against when there's no fixed schedule. ### Threshold **What is the acceptable boundary for the run?**

A threshold applies to monitors that produce a single computed value each run — these are **aggregate monitors**: Freshness (data age), Volume (row count change), and the Field Health numeric checks Average, Minimum, Maximum, and Sum. A threshold defines the acceptable boundary for that one value. If the value falls outside the threshold, the run fails and Decube raises an incident. Threshold shapes, depending on monitor type: * **Duration** (Freshness) — a value and unit (minutes, hours, or days). The check fails when the data's age is greater than this duration. * For example, a nightly ETL job expected to finish by 2:00 AM might use a 30-minute duration threshold, checked at 2:30 AM. * **Numeric range** (Average, Minimum, Maximum, Sum, and the Numerical Range check) — a minimum and/or maximum. You must set at least one bound; if you set both, the minimum must be less than the maximum. Bounds are inclusive — a value exactly at the boundary passes. For example, a business expecting average order value to stay between $20 and $200 would set both bounds on an Average monitor. * **Count delta** (Volume) — a minimum and/or maximum change in row count between runs, for example "between 5,000 and 20,000 more rows" for an hourly-ingesting events table. * **Percentage delta** (Volume, with metadata-based or full-table-count detection only) — a minimum and/or maximum percentage growth or shrinkage, for example "no less than -10% growth" for a daily-appended orders table. There's no upper cap — a table can grow by more than 100% in a period. ### Tolerance

Tolerance is a separate, distinct concept from threshold — it doesn't apply to the same monitors. Tolerance applies to **row-level monitors**: monitors that check every row in the scanned data and count how many fail a rule, such as a range, a pattern, a value set, or a condition like "is this null." These are: Numerical Range, String Length, Null, Unique, Zero, Is Before, Is After, Regex Match, Regex Not Match, Value In, Value Not In, and Custom SQL. Tolerance defines how many failing rows are allowed before the monitor raises an incident: * **Absolute** — a fixed number of rows, for example "allow up to 5 failing rows." * **Percentage** — a share of the rows scanned, for example "allow up to 10% of scanned rows to fail." #### Deciding between tolerances * Absolute tolerance suits a small, high-stakes table — for example, allowing at most 5 duplicate rows in an `orders` table, where every row matters and the volume is low enough that a fixed number makes sense. * Percentage tolerance suits a large table with some expected noise — for example, allowing up to 10% of rows in a `phone_number` column to be null, since phone number is known to be optional at signup. Tolerance is exclusive — the check only fails once the failing count goes above the tolerance, not at or below it. If you don't set a tolerance, it behaves as if tolerance were zero: even a single failing row triggers an incident. ### Custom attributes

Every monitor can carry **custom attributes** — tags or key/value pairs you define that are meaningful to your organization, such as severity, business domain, or team owner. These attributes show up on the monitor itself and on every incident and case it produces, and you can use them to route alerts — for example, notifying the Finance Slack channel whenever a monitor tagged `domain: finance` fails. You can already create custom attributes in the Org Settings in the My Account page - just make sure to add `Monitors` as an Applicable object for the Custom Attribute. ### DQ dimension

You can also assign every monitor a **DQ dimension** — one of Accuracy, Completeness, Uniqueness, Validity, Timeliness, Consistency, Granularity, or Others. Decube uses this purely for reporting, on the Data Quality Scorecard, to show which quality dimension a monitor's score contributes to. It doesn't affect how the monitor runs or evaluates. For example, a Null check on a required `email` column might be tagged Completeness, while a Unique check on `order_id` might be tagged Uniqueness — so the Scorecard can show you, at a glance, that your Completeness score dipped this week without you having to trace it back to individual monitors. ### Destructive changes and monitor versions Most changes to a monitor's configuration are **destructive** — they create a new, separate version of the monitor rather than editing it in place. This keeps run history meaningful: if you change what a monitor is actually checking, its past results shouldn't blend in with results measured against the new definition. {% embed url="" %} A small set of changes are not destructive — they update the current version in place: adjusting the threshold value, adjusting the tolerance value, and names, descriptions of monitors. For example, if a governance lead changes a Freshness monitor's Detection Method from metadata-based to a specific column, or changes an Average monitor's Scan Method from Full Table Scan to Incremental Scan, Decube creates a new monitor version — so results from before and after the change aren't blended together. But if they only loosen a Freshness monitor's threshold from 30 minutes to 45 minutes, that's not destructive — it just updates the existing version in place. Changing the pattern on a Regex Match or Regex Not Match monitor, or changing the list of values on a Value In or Value Not In monitor, is treated as destructive — even though it might feel like a threshold tweak. It redefines what a passing row looks like, so it creates a new version. When you make a destructive change, Decube warns you before saving that this will create a new version of the monitor.