Changelog
Here's where we let you know what's new on decube.
21 March 2024
Release 1.9.6 Updates
Linked Glossary/Business terms Column on Asset Details > Schema Tab:
Within the Asset Details > Schema tab, a new column titled "Linked Assets" has been added.
Users can now easily see the linked glossary/business term associated with the asset.
By selecting the linked term pill, users are directed straight to the corresponding glossary/business term, providing quick access to related information.
Config Settings - Slack Notification configuration enhancement.
User are able to set up config settings just by submitting their Slack Channel ID or Slack channel name.
Custom SQL - Grouped-by option is now supported !
This feature empowers users to organize their custom SQL queries by grouping desired columns and selecting distinct values according to their specific requirements.
22 February 2024
Documentation
Introducing our Documentation Feature, this feature allows users to add a knowledge base to an asset. Users can add notes directly to each data asset under asset details, facilitating seamless collaboration, knowledge sharing, and troubleshooting.
Custom Frequency
Our new Custom Frequency Option gives you complete control over how often your data is monitored. users are now allowed to set custom frequency based on the following frequencies:
Daily: Users can select any hour of the day and a preferred time zone.
Weekly: Users can select a day of the week, along with a preferable time zone and time.
Monthly: Users can select between a monitor running on the last day of every month, or on a specified day of their choice, along with a time and time zone.
Simply select your frequency, set your credentials, and enjoy personalized data monitoring that fits your needs perfectly.
Model Feedback
Introducing our Model Feedback update, a user-friendly feature designed to fine-tune the level of sensitivity when providing feedback on incidents within your data monitoring platform. This tool helps users indicate how serious or urgent an incident is, making it easier for administrators to prioritize and respond accurately.
Adjustable Sensitivity Slider: Upon rating an incident as "bad", using the thumbs down icon, users can easily slide the sensitivity scale to indicate the severity or urgency of the incident they're reporting.
On-Demand Monitoring
Welcome to our On-Demand Monitoring System, where you get instant insights into your data infrastructure whenever you need them. Here are the monitors that offer On-Demand Monitoring:
Freshness
Volume
Custom SQL
Field Health (except cardinality)
This feature allows users to create monitors to meet specific, one-off requirements.
29 January 2024
What's New: Microsoft Teams Integration
We are excited to announce the deployment of our latest update, introducing seamless integration with Microsoft Teams. Users can now utilise Microsoft Teams as their primary alerts medium and effortlessly configure custom alerts for their monitors through Decube.
Key Features:
Microsoft Teams Alerts: Users can now choose Microsoft Teams as their preferred channel to receive alerts, enhancing real-time communication within teams. This can be configured at 'Config Alerts' under 'Config Settings' tab.
Custom Alert Configuration: Tailor alerts for your configured monitors according to your specific monitor configuration needs.
18 January 2024
What's New
Release 1.8.0 New Features:
Enhanced Asset Management:
Simplify permissions with the new "Request Access" feature, enabling users to conveniently request and grant permissions directly through the Catalog. This streamlined process ensures precise access for designated individuals.
Designate ownership for connected data sources, bringing clarity and accountability to your system. Now, it's clear who is responsible for each connected data source.
Flexibility in ownership extends to data sources, allowing seamless adaptation and reassignment of responsibilities within teams, fostering improved collaboration and resource management.
Empower your teams with the ability to transfer ownership of schema and table assets, providing greater flexibility in adapting and reallocating asset responsibilities.
Catalog Module Improvements:
Experience a more streamlined user interface with Catalog module enhancements, now including Schema assets and Data source assets. This update offers a comprehensive overview of tables within a schema and data sources, with the added ability to explore individual levels of selected data source assets.
Asset Overview Update:
The updated Asset Overview in the Schemas and Data Sources catalog provides a more comprehensive summary for your schemas and data sources, including
counts of schemas,schemas without owners,total incidents,incident percentage,number of tables, andtables with incidents.
Access Request Management:
Enhance communication and streamline access requests with our updated "Request" tab within the Catalog module. Reviewers/Data Source/Schema/Data Source owners now receive a "Request Access" email, facilitating efficient management of access requests.
Centralised Permissions Overview:
Simplify user access management with the centralized permissions overview in User Info and Group Management. Access a comprehensive overview of granted permissions on table level/group level requests, making the process more user-friendly.
Lineage Revamp:
Explore our revamped Lineage feature, offering a dynamic and user-friendly interface to showcase your data flow. Experience dynamic upstream/downstream relationship lines and efficient column-level search functionality within the lineage table.
Bug Fixes
Patched issues with the Redshift Connector Form.
Patched issues with the Power BI Connector Form.
6 December 2023
What's new
Single Sign-On (SSO)
We are thrilled to announce the latest enhancement to our application, Single Sign-On (SSO) integration with Microsoft. This new feature is designed to streamline security and simplify the login process, providing a more seamless and secure experience for our users.
Reports Module
Introducing the latest "Reports" module on our platform. This powerful addition empowers users with valuable insights into dataset access and queries by user, enhancing transparency, accountability, and efficiency in managing data resources.
Users can now generate detailed reports on dataset access, providing a comprehensive view of who has accessed specific datasets within the platform.
The "Reports" module includes the ability to generate reports on dataset queries, showcasing the queries executed by individual users.
31 October 2023
Release 1.7.3
Added schema selection for Redshift and Synapse so users can only add the schemas that they want to add into decube.
30 October 2023
Release 1.7.2
Introducing the new Feed Feature in our Data Catalog module! This system on our platform allows users to interact, post threads, mention others, reply, and even edit their responses.
Users will now get email notifications when they are mentioned, receive replies, or when comments are made on their data asset.
Users are also able to subscribe or unsubscribe threads to stop/receive email notifications.
Data assets now include a "Rating" feature, allowing users to evaluate them by giving ratings out of 5.
A ratings filter was added to the Data Catalog, allowing users to sort data assets by their given star ratings.
We also implemented new connectors!
We added a new connector:
AWS S3 Datalake.
2 October 2023
Release 1.7.1
Multifactor Authentication with authenticator apps is now supported.
MFA can now be enforced in your organization by an Owner or a user with
Manage users for access management.Users can set up MFA on their account to ensure security for the account.
26 September 2023
Release 1.7.0
New module - Data Governance. One place to manage classification policies in your organization. Users are able to store all governance policies in the platform to be managed as well as link actual data assets from the Catalog to the policy.
Classifications are now customisable - users can now create custom classifications by adding a custom policy in the Classification Policies tab.
Users can now use our auto-classification workflow to set a rule in connected data sources to automatically tag columns that match a keyword or regex input. Example: classify all first_name columns with PII.
All classified assets under a policy (eg. list of columns classified as PII) shows up in the Managed assets of the policy. Users can export the entire list in csv format.
Module-based policy in Group Management has been updated with new selection in dropdown that is Data Governance. Users will need to give access to the Data governance module by adding the required access through the module-based policy form.Revamped Config - Centralised control to create, update all monitors. Our config module landing page simplifies the process of setting up various monitoring methods for your assets. You can navigate here by the Config panel in the sidebar.
All monitors in Config now shows all available monitors that your organization has added, for easier modification and enable/disable. Job Failures, Schema Drift, Custom SQL monitors all are now shown in one page based on your selected data source.
Incident Levels (Info / Warning / Critical): You can now set individual incident details
For Field Health, bulk monitoring is now supported. Instead of adding a single test to each column at a time, this has been improved to add tests to multiple columns at once.
You can control Schema Drift monitoring in a more granular manner by controlling which schema change type to monitor, and which schema to be monitored.
Smart training & Lookback period: You can now shorten the period of training for each monitor by enabling smart training. Adding a lookback period also allows you to backfill your monitor with historical data, so you can see any anomalous data from before the time the monitor was set up.
Custom alert channels: You can now set up specific alert channels for each monitor. For example, link Airflow Job failures monitoring to a specific Slack channel #job-failures. This setting will override the default alert setting in your organization.
With the addition of custom alerting, we have upgraded the notification system to send out individual alerts instead of grouping the alerts based on test type. With this change to individual alerts, each alerts will now contain more information for each incident, and clicking on the provided link will send you straight to the specific incident details page instead of landing on the Data Quality page, saving you time.
Row Creation selection for monitoring: “Metric Time” has now been renamed to “Row Creation” for more clarity. There are several important changes here:
All Records: Adding a monitor to scan the entire table is now supported.
SQL Expression: If there’s no datetime/date column in your table to be selected as a timestamp column (eg. you store it as a string type column), you can now convert it by adding a valid SQL expression to be used as a timestamp column on our platform.
9 August 2023
What’s new
Incidents Details
More informative graphs for each incident type, with new chart types and added axis titles.
The ability to assign the incident to one of the users.
See the owners for the affected asset in the incident details on the side panel.
An incident history where user actions of changing incident status of muting, closing the incident is tracked.
For Field Health incidents, a sample of the affected rows can be found directly in the Preview tab, where dynamic masking of PII information is applied.
See all scans from your monitors with datetime and actual with expected values in the History tab. Users can click into any failed incidents to see the specific incident description.
See all assets impacted downstream of the asset with incident in the Impacted Areas tab, users can also download the list as a csv.
8 August 2023
What’s new
We added a new connector: AWS Glue. You can now see your transformations as a Data Job in the Catalog.
2nd August 2023
Bug Fixes
UPS error ("invalid asset") is resolved during group creation without any sourced-based policy added.
1st August 2023
Bug Fixes
Missing data sources during Sourced-Based Policy creation now appears.
Clicking on back button on policy forms sends user back to Policy selection form instead of closing the modal.
Source Modal Policies creation using template (Use as template) fixed.
When user wants to add more than 3 tags, there is now an error prompt that is shown. Users will also be shown an error when attempting to create a tag with special characters.
"Database" word removed from forms with non-database connectors.
July 2023
31 July 2023
What’s new
Search in our Catalog is now powered by Elasticsearch, allowing for a better search experience with natural language search. Searching by path has also been enabled, such as searching
schema_name.table_name.Clicking on specific search suggestions from your search input now leads you directly to the asset. Pressing on “Enter” or clicking on the Search button searches your Catalog with your search input.
Narrow down your search by quickly filtering assets by type, schema, tags, classifications, owners, and count of incidents, ensuring you find exactly what you need with minimal effort.
Introduced a sort option to sort by relevance, or alphabetical order.
Glossary items such as Glossary, Categories and Terms are also shown in search results. Clicking on these will bring you to the location of it in the Glossary directly.
Bug Fixes
Classification not showing in the Glossary Terms has been fixed.
13 July 2023
Bug Fixes
Page crash experienced on Asset Details > Preview for Table assets has been fixed.
11 July 2023
Bug Fixes
Added patches for security vulnerabilities identified.
10 July 2023
Bug Fixes
Fixed the issue of the password field on the organization invite form not allowing special characters. Added a list of special characters in the UI.
The issue of schema selection in BigQuery form not being able to be submitted has been fixed.
June 2023
21 June 2023
What's new
Our navbar became thinner! This gives you more space for your Lineage, Incident Details to take up more real estate on your screen.
Our lineage also now can collapse once you've expanded it, by clicking on the "-" sign.
Bug fix
Some of our tooltips were hard to read because of the sizing, so it is not fixed to fit perfect on each component.
16 June 2023
What's new
We've got a new connector - Azure Data Factory. Check out how to get connected here.
14 June 2023
What's new
Group Management: Manage user access by creating groups and granting these groups permissions. Check out the summary here.
If you're an existing customer, we will migrate you seamlessly to the new system. Admins are now in the Owners group, and Members will be transitioned to the Members (Legacy) group. All data sources access will be remained. Read more.
New customers who is the first user sign up will be added to the Owners group. Read more.
A guide has been made for create groups and assigning policies here.
Approval Workflow: Now you need to go through an approval process to make changes to the Catalog or Glossary. Check it out here.
2 June 2023
What's new
New connector: Azure Synapse Analytics! We are continuously adding integrations to Azure products. Check out our Public Roadmap here.
Made our Schema Drift incidents to show description of the event that happened, eg.
addition,table, ortype_change.
May 2023
26 May 2023
What's new
New alert type: Webhook integration so that you can set up your own custom integrations.
Upgraded our Lineage with our SQL parsing engine, you may see some new lineage relationships shown in the Lineage tab of your Asset details.
You can now disconnect your Slack organization on the Configure Alerts page.
Bug fixes
Issue where the decube Catalog does not show the right assets from your added Databricks Catalog is now fixed.
19 May 2023
What's new
Data Recon UI has been completely refreshed! This includes:
Adding new recon is now a more focused experience, taking up a whole page instead of being a sidebar.
You can compare datasets from two separate data sources.
Set up conditions to filter out rows to be checked in the recon. This functions as a WHERE clause. Read more here.
Selection of datetime field is now optional.
You can now export your completed recon in .csv. This downloads the unmatched rows found during the data recon.
Preview tab has been added for Asset Details.
We show a sample of 10 rows from your table.
If there are columns that are marked as PII or Sensitive, this will show up masked in the Preview.
Bug fixes
UI clashing issue with the sidebar on small window sizes has been fixed.
Fonts in graphs and charts have been updated to Plus Jakarta Sans.
Notify toggle on Modify monitor modal has been fixed. It now shows the correct state, whether it's enabled or disabled.
11 May 2023
What's new
New look on our app! We've updated our font for better readability across all our pages.
Bug fixes
Lineage graph not re-appearing when navigating tabs has been fixed.
When viewed on small screen sizes, the UI on Table Overview had clashing issues, which has been fixed. Screen sizes down to 720p on normal scaling are now supported without issues.
April 2023
27 April 2023
Bug fixes
On Safari 15.6, page crash when viewing the field health monitors tab has been fixed. (Safari 16.1 and other browsers are not affected by this bug).
Job failure monitors: Toggle for Notification on Data Job monitors now work as intended.
20 April 2023
What's new
You can now see the lineage from your transformations in your Lineage now! Just go to the Asset Details and head to the Lineage tab. You'll need to map your Lineage via the Additional Config tab in the Modify Data Source under My Account.
Support for Azure SQL has been added.
Bug fixes
Documentation editor icons have been fixed. The editor also looks better on wider screens now.
Data Recon now shows the empty state when there are no new data recons added yet.
19 April 2023
Bug fix
Clicking on a table in Dashboard was sending users to a broken link. It's not been fixed and should lead to the Asset Details now.
13 April 2023
What's new
Added new integration: SQL Server! You can now connect your SQL Server to decube to monitor your data health and see all your tables in the catalog. Read more about it here.
12 April 2023
What's new
Added a new integration: Singlestore! We're also have a partnership now. Read more about it here.
5 April 2023
What's new
Business glossary: Introducing a new feature where users can add their own metrics, add owners and documentation. You can also link them to data assets. The business glossary will allow users to create a common language for metrics and data assets, improving communication and collaboration across teams.
New integration with Fivetran
New integration with Databricks
March 2023
21 March 2023
What's new
New integration with dbt, supporting new asset types such as data jobs, data runs, virtual tables etc.
New incident type, job failures has been introduced.
Schema tab added, users can now add tags and classifications to fields.
Field stats added, users can now query their datasets and find out some statistics on each field. Also integrated the data quality incidents into this view.
A new look to the Data Quality and Dashboard page, with new asset icons.
Bug fixes
Fixed a bug where Intercom is still recognizing the user when user is logged out automatically.
User is now directly correctly to the filtered incident types in the Data Quality page from notifications.
Incident graph in Data Quality main page are now bar charts!
9 March 2023
What's new
For Google Big Query and Snowflake connections, you can now add Volume and Freshness monitoring without a metric time selection.
For all sources, you can now use
datecolumn as a metric time column. Adding adatetype column would restrict your scan frequency selection to24 hoursonly.Monitoring for schema changes is now enabled on all tables by default. You will just need to let us know if you'd like to receive notifications by toggling the notification within the Asset Details.
UI improvements on the Bulk Table Config and Asset Table Config for better user experience, such as updated icons for metric time selection and descriptive placeholder texts. Also added clearer tooltip for users to be aware of the pre-requisite to select the metric time before a field monitor can be added.
Bug fixes
Lots of bug fixes and improvements made on the Data Quality engine for better reliablity and flexibility.
6 March 2023
Security
If user becomes inactive, after some time the cookie will expire and user will be redirected to the log-in page. Logged in user who is actively using the application will have user session automatically extended.
2 March 2023
What’s New
Sign-up and sign-in pages have been updated to be more responsive and make the sign-in process smoother.
We are now supporting your organization via Intercom! Drop us a message via the bubble on the bottom right if you need any help.
The UI for adding and modifying monitors has been updated.
Bug Fix
Fixed new data recon: Scheduled run period not submitting when user did not interact with the dropdown.
February 2023
23 Feb 2023
What’s New
For Google Big Query connections, you can now specify which schemas are to be scanned into decube.
If you previously had connected GBQ, you can now go to My Account and modify the connection and unselect the schemas that you would not like to include. These schemas will be removed on our next collector run.
You can also opt to not automatically add new schemas in the future when they are detected by our collector by selecting your preferred option.
Don’t like the name that you used while signing up? You can now update it via the Update Profile tab on My Account.
You are now able to add Owners (data owner and business owner) to an asset. For now, tags on tables will be unavailable as we are improving this functionality to add the tags directly on the field level such as PII, Sensitive tagging which is coming very soon.
You can now save time by updating your table-level tests (Freshness, Volume, Schema drift) from the Data Quality page directly.
The incident details graph has been improved to show the min/max of the Expected vs the Actual for Volume graphs.
Our docs has been updated for easier navigation to specific connections and features.
Bug Fixes
Fixed the count of data sources on filter selection when data sources were disabled.
Fixed behavior of custom monitors lists so that it does not show lists of other tables.
8 Feb 2023
What’s New
User will need to accept the disclaimer on the Table Overview (which is an opt-in only feature) before the profiler is ran.
Bug Fixes
Bug fixes on the UI on the Dashboard and the bulk selection.
Terms and Privacy Policy on the sign up page have been updated with new links.
1 Feb 2023
Bug Fixes
There is now a sorting logic on the Bulk Table Configuration so that monitored tables appear on the top of the list followed by selections with metric time. This helps you to be able to see configure your tables easily.
January 2023
31 Jan 2023
What’s New
Plans & Billings page under My Account now shows the usage limit according to the plan the organization is subscribed to.
Bug Fixes
UI fixes across the app such as truncating long asset names and rephrasing for a better user experience.
30 Jan 2023
Bug Fix
Creating a new data recon now shows loader when operation is running. If a recon job is not successful, a failure state will be shown.
25 Jan 2023
What’s New
A new connector to Snowflake has been added. Users can now add Snowflake as a new data source under the My Account page.
Bug Fixes
When creating a new Data Recon, you can now search for your tables that you would like to use.
When there are no incidents yet to be shown on the dashboard, we now have a more descriptive text on the UI.
20 Jan 2023
What’s New
The Lineage UI has been enhanced with a cleaner look and new icons.
Bug Fixes
Disabled data sources now do not show inside data sources selection.
Incidents list within Table details have been fixed.
19 Jan 2023
Bug fixes
Fixed issue in Bulk Table Configuration where selections were not able to be viewed when there are many metric time selections.
16 Jan 2023
What’s New
Catalog has been enhanced with new icons and filtering for search functionality.
User can now add new monitor directly from the Data Quality page.
Get support via live chat on the bottom right of our page.
Custom SQL monitors can now be added:
User will be able to add Custom SQL scripts to write specific tests cases as monitors.
Monitors page within Table details has been updated to support custom monitors which has user-defined names and descriptions.
Bug fixes
Fixed issue where more tables and incidents were not loaded when scrolling to the bottom of the page in the Dashboard and Data Quality.
Fixed issues on Data Recon page.
10 Jan 2023
What’s New
User are now able to use Bulk Table config feature to do bulk selection on multiple metric time columns. This new feature can be accessed via the Catalog.
Users will be required to configure their tables’ metric time columns and monitoring before enabling table-level and field-level tests. Switching on monitoring for a table enables all table-level tests to be on by default.
Schema/dataset name has been added to new data recon selection.
Bug fixes
Incident debug messages have been improved.
Adding tags to assets, schema drift and table config overlay issue with metric time selection have been fixed.
Removed error logs on console to prevent info leaks on logs.
9 Jan 2023
Bug fixes
Ensure users are not able to log in after deactivation of account.
6 Jan 2023
What’s New
User management
Admin able to change user roles of other users (
membertoadminand vice versa).Admin able to revoke invites of users who are
pending sign up.Admin able to deactivate/re-activate other users.
5 Jan 2023
Bug fixes
Safari users facing issues with UI freezes on certain pages with loading functionality which was fixed with his hotfix.
Last updated