Google Bigquery
Adding Google Big Query to your decube connections helps your team to find relevant datasets, understand their quality via incident monitoring and apply governance policies via our data catalog.
Last updated
Adding Google Big Query to your decube connections helps your team to find relevant datasets, understand their quality via incident monitoring and apply governance policies via our data catalog.
Last updated
Freshness
Volume
Schema Drift
Field Health
Custom SQL
Job Failure
Data Profiling
Data Preview
Connecting to Big Query requires following the steps below to completed.
To ensure a smooth experience configuring the connection.
A custom role for the decube service account is required to ensure we have correct access to the resources we need.
From the dashboard, click the hamburger (three lines indicating a menu), the top left of the browser next to the Google Cloud logo, and Search for IAM and Admin.
Find Roles and at the top, below the search bar, click + Create Role.
Fill in Roles as below
Title: decube data observability
ID: decube
Role launch Stage: General Availability
Click on Add permission, a popup will be displayed. Then on Enter property name or value type each of the permissions below, press enter and click on the checkbox to assign that permission to the decube data observability role.
When you are done assigning permissions, click Add to exit the popup, on Assigned Permission table, review the permissions granted to match above requirements.
Click Create.
Our reporting module requires additional permission and Google Cloud Asset API to be enabled. We suggest creating a new role and attaching it to the existing service account above.
For full report generation especially with multiple GCP projects, the service account will need additional role on the Organization level. Without these additional roles, the generated report will only contain queries from the current project
Go to GCP API and Services and click Library.
Search for “cloud asset api”.
Click Enable
4. Go to IAM and admin and Roles section click Create Role.
5. Grant the role with these permissions and click Create
Grant the Role to the service account attached with the Bigquery source.
For decube to connect with BigQuery, a JSON key file is required from a Service Account with the proper Roles assigned to it.
From the dashboard, click the hamburger (three lines indicating a menu), the top left of the browser next to the Google Cloud logo, and Search for API's and Services.
On that page, click on Credentials, at the top, click + Create Credentials and then select Service account.
Fill in the Service account details form, we recommend
Service account name: decube,
Click Create and Continue.
On Grant this service account access to project, click on Select a role, the on the _Filter _ search, type decube data observability created previously and click to assign this role to the service account. Click Continue.
Skip Grant users access to this service account form and click Done.
On the Service Account page you will see a Service Account named decube, click on it.
On the options below the Service Account name, click on Keys
Click Add Key and then select Create new key from the drop-down.
For the Key Type, choose JSON and click Create.
You will automatically download the JSON file and can refer to the location of the file on your browser, save it somewhere easily accessible like your Desktop.
Having to the Project that contains the intended BigQuery.
An account that can view and create
