> For the complete documentation index, see [llms.txt](https://docs.decube.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.decube.io/data-quality-v2/frequently-asked-questions-monitors.md). # Frequently Asked Questions (Monitors) Answers to questions that come up once you start configuring monitors, grouped by the building block they relate to. ## Threshold vs. Tolerance ### What's the difference between Threshold and Tolerance? Threshold and Tolerance answer two different questions, and on some monitor types, both apply at once. Threshold defines what counts as a failing value — the acceptable boundary a single value must fall within. Tolerance defines how many failing rows are allowed across a scan before Decube raises an incident. * **Aggregate monitors** — Freshness, Volume, and the Field Health checks Average, Minimum, Maximum, and Sum — produce exactly one computed value per run, so Threshold alone decides pass/fail. There's no "how many" to allow, so these monitors don't use Tolerance. * **Numerical Range and String Length** use both: a Threshold defines what makes an individual row fail (a numeric or character-length range), and a Tolerance separately defines how many such failing rows are allowed before an incident. * **Null, Unique, Zero, Is Before, Is After, Regex Match, Regex Not Match, Value In, and Value Not In** don't use a Threshold at all — what counts as a failing row is fixed by the check itself, a pattern, or a value set instead. Tolerance still applies to decide how many failures to allow. * **Custom SQL** has no Threshold field either, but Tolerance (Absolute, or Percentage if you provide a Search Space Query) decides how many rows the query can return before it's an incident. ### If I don't set a Tolerance, what happens? Decube treats it as if the tolerance were zero. Any single failing row raises an incident — there's no buffer. ### Does a value exactly at the Threshold or Tolerance boundary count as a pass or a fail? The two behave differently. Threshold bounds are inclusive: a value exactly at the minimum or maximum passes. For example, an Average monitor with a threshold of min 20, max 200 treats an average of exactly 20 or exactly 200 as passing. Tolerance is exclusive: the check only fails once the failing count goes above the tolerance, not at or below it. A tolerance of 5 allows exactly 5 failing rows without raising an incident — the 6th row is what triggers it. ## Schedule & Scan Method ### Why does Manual scheduling require Full Table Scan? Incremental Scan works by comparing against "since the last run" — but a Manual monitor has no fixed run cadence, so there's no reliable window to define what counts as new. Full Table Scan sidesteps that by reading the whole table every time, regardless of when the previous run happened. ### Why can't Freshness – SLA or Volume run on a Manual schedule? Both depend on comparing against a fixed cadence: * **Freshness – SLA** measures whether data arrived by a specific expected time, which only means something if the check runs on a predictable schedule. * **Volume** compares the current run's row count against the prior period's, so it needs that prior period to exist on a regular cycle. Freshness – Staleness and the Field Health checks don't have this dependency, so they do support Manual. ### When should I use Incremental Scan instead of Full Table Scan? Incremental Scan is built for large, continuously growing tables — for example, a `clickstream_events` table with billions of rows — where re-scanning everything each run would be slow and expensive, and you only care about validating newly arrived data. Full Table Scan works well on small reference tables, like a `countries` lookup table, where scanning everything each run is cheap. ### What's the difference between a named timezone and a fixed UTC offset, and which should I pick? * A **named timezone** (for example, Asia/Singapore) keeps the monitor running at the same local clock time and automatically adjusts for daylight saving. * A **fixed UTC offset** (for example, +08:00) keeps the monitor running at the same point relative to UTC, regardless of daylight saving changes anywhere. Most teams should pick their local named timezone — a fixed offset is for edge cases, like a compliance check that must run at exactly 00:00 UTC every day. ## Detection Method ### When should I use column-based detection instead of metadata-based? Use column-based detection when the warehouse's technical write time doesn't reflect what you actually care about. For example, an `orders` table loaded through CDC might show a recent write time even though the underlying order itself is old — picking the `order_date` column tracks business freshness instead of load freshness. ### Why would Decube not offer metadata-based detection for a table? Some data sources or table types — views, for instance — don't expose last-modified or row-count metadata. Decube only offers metadata-based detection where that metadata is actually available; otherwise, use column-based detection, or for Volume, full table count. ### Why do I have to set a timezone for my timestamp column? Decube only asks for a column timezone when the column doesn't store timezone information, such as Snowflake `TIMESTAMP_NTZ`. Decube calculates freshness age and volume windows in UTC, and a value like `2026-10-15 08:00:00` with no offset could be any of several moments, hours apart. Without the offset, a Freshness monitor could report the wrong data age and a Volume monitor could count rows in the wrong window. If your values are already written in UTC, select `+00:00`. For more detail, see [Column timezone](/data-quality-v2/monitors/building-blocks.md#column-timezone). ## Custom SQL specifics ### Why do I have to run Test Value before I can save a Custom SQL monitor? Test Value is a required validation step. It confirms the SQL you've written is valid, and gives you a way to sanity-check the result against what you'd expect from running the same query directly against the warehouse — so you catch a logic mistake before committing to the monitor. ### Does re-running Test Value later count as a monitor run? No. You can re-run Test Value from an existing monitor at any time as a spot-check, independent of the monitor's regular schedule. It doesn't affect run history, incidents, or the DQ score. ### What happens if I don't associate any assets with a Custom SQL monitor? The monitor falls back to being associated with the data source as a whole, rather than a specific table. ### Why can't I set a percentage Tolerance on my Custom SQL monitor? Percentage tolerance requires a Search Space Query — a second query defining the total population being checked, so Decube can express the failing count as a share of that total. Without a Search Space Query, Decube only knows the failing count, not what it's a percentage of, so only Absolute tolerance is available. ## Versioning & destructive changes ### Why does changing a monitor's schedule or scan method create a new version instead of just updating it? Most configuration changes are destructive because they change what the monitor is actually checking — blending results measured against two different definitions of "pass" into one run history would make that history meaningless. Changing the schedule or scan method changes how or how often the data gets evaluated, so Decube creates a new monitor version rather than editing the current one in place. ### Is changing the pattern on a Regex check or the value list on a Value In/Value Not In check destructive? Yes, even though it can feel like a minor tweak. Changing the pattern or the value list redefines what a passing row looks like, so Decube treats it the same as any other destructive change and creates a new version. ### What changes are safe to make without creating a new monitor version? These update the current version in place, without creating a new one: * Adjusting the Threshold value * Adjusting the Tolerance value * Updating a monitor's name or description or DQ dimension or custom attributes None of these redefine what the monitor is checking — they only change the acceptable range, the allowed failure count, or metadata used for organization and reporting. ## Choosing between monitor types ### What's the difference between the Field Health numeric checks (Average, Maximum, etc.) and Numerical Range? Average, Minimum, Maximum, and Sum each compute a single aggregate statistic across the whole scan and check that one number against a Threshold. Numerical Range looks at every individual row and checks each value against the range independently. * An **Average** monitor on `orders.order_value` with a threshold of min 20, max 200 only fails if the average order value across the whole run drifts outside that band — a handful of unusually cheap or expensive orders won't trigger it on their own, as long as the average holds. * A **Numerical Range** monitor on `users.age` with a range of 18–65 fails a specific row the moment its value falls outside that range, regardless of what the rest of the table looks like — a single row with age -5 or age 999 fails immediately, even if every other row is fine. Use the numeric checks when you care about the aggregate picture. Use Numerical Range when every individual row needs to fall within bounds. ### Regex Match, Value In, and Condition checks all seem to validate column values — how do I pick the right one? They differ in what kind of rule defines a failing row. * Regex Match (and Regex Not Match) test a value against a pattern — use this when valid values follow a structure you can express as a regular expression, like requiring `users.email` to match an email format. * Value In (and Value Not In) test a value against a fixed, finite list — use this when the valid values are a known, enumerable set, like restricting `users.country` to US, CA, or MX. * Null, Unique, and Zero don't take a pattern or a value set at all — each has one fixed, built-in rule (a row fails if it's blank, a duplicate, or zero), so Tolerance is the only thing you configure. If your valid values follow a shape, use Regex. If they're a specific list, use Value In or Value Not In. If the rule is simply "not blank," "not a duplicate," or "not zero," use Null, Unique, or Zero directly, rather than trying to express the same rule as a pattern or value set. ### Why can't I add a monitor to a virtual table? A virtual table, such as a dbt model or a Tableau data source, is a logical model that Decube catalogs from a transformation or BI connector. It has no physical data in a warehouse for a monitor to query, so it is not an applicable asset for any monitor type. Add the monitor to the table or view the model materializes in your warehouse instead. For the asset each monitor type applies to, see [Applicable assets](/data-quality-v2/monitor-types.md#applicable-assets). ## Custom attributes & DQ dimension ### Where do I create custom attributes for a monitor? Create custom attributes in Org Settings, from the My Account page. When you create one, add Monitors as an Applicable object for that custom attribute — otherwise it won't be available to assign to your monitors.