# Google Bigquery ## Supported Capabilities {% tabs %} {% tab title="Supported Capabilities" %} **General** * **Metadata** — metadata extraction and display of asset information (tables, columns, schemas). Types collected: Schema, Table, Column, View * **Profiling** — data profiling on the Profiler tab * **Preview** — sample data preview * **Data Quality** — data quality monitoring and observability * **Configurable Collection** — selective ingestion of schemas/workspaces in Data Source Management * **View Table** — view tables, which are virtual tables based on SQL queries **Data Quality Monitors** * Freshness * Volume * Field Health * Custom SQL * Schema Drift **Lineage** * **View Table Lineage** — tracks virtual tables (views) and their data dependencies * **SQL Query Lineage** — maps data movement through SQL queries (SELECT, JOIN, INSERT, etc.) {% endtab %} {% tab title="Not Supported" %} **General** * External Table * Stored Procedure **Data Quality Monitors** * Job Failure **Lineage** * External Table Lineage * Foreign Key Lineage * Stored Procedure Lineage {% endtab %} {% endtabs %} ## Connection Requirements Connecting to Big Query requires following the steps below to completed.

### Prerequisite To ensure a smooth experience configuring the connection. 1. Having [access](https://cloud.google.com/iam/docs/granting-changing-revoking-access#iam-view-access-console) to the Project that contains the intended BigQuery. 2. An account that can view and create [Service Accounts](https://cloud.google.com/iam/docs/understanding-service-accounts) ### Creating decube Data Observability Role A custom role for the decube service account is required to ensure we have correct access to the resources we need. 1. From the dashboard, click the hamburger (three lines indicating a menu), the top left of the browser next to the Google Cloud logo, and Search for `IAM and Admin`. 2. Find `Roles` and at the top, below the search bar, click *`+ Create Role`.* 3. Fill in `Roles` as below * `Title`: decube data observability * `ID`: decube * `Role launch Stage`: General Availability 1. Click on *`Add permission`,* a popup will be displayed. Then on *`Enter property name or value`* type each of the permissions below, press enter and click on the checkbox to assign that permission to the decube data observability role. ``` bigquery.datasets.get bigquery.datasets.getIamPolicy bigquery.jobs.create bigquery.jobs.get bigquery.jobs.list bigquery.jobs.listAll bigquery.readsessions.create bigquery.readsessions.getData bigquery.readsessions.update bigquery.routines.get bigquery.routines.list bigquery.tables.get bigquery.tables.getData bigquery.tables.list resourcemanager.projects.get storage.buckets.get storage.buckets.list storage.objects.get storage.objects.list ``` 1. When you are done assigning permissions, click `Add` to exit the popup, on `Assigned Permission` table, review the permissions granted to match above requirements. 2. Click *`Create`*. ### Enabling Reporting Functionality Our reporting module requires additional permission and Google Cloud Asset API to be enabled. We suggest creating a new role and attaching it to the existing service account above. ``` cloudasset.assets.analyzeIamPolicy cloudasset.assets.searchAllIamPolicies cloudasset.assets.searchAllResources ``` {% hint style="warning" %} For full report generation especially with multiple GCP projects, the service account will need additional role on the Organization level. Without these additional roles, the generated report will only contain queries from the current project ``` bigquery.jobs.list bigquery.jobs.listAll ``` {% endhint %} 1. Go to GCP API and Services and click Library.

2. Search for “cloud asset api”.

3. Click Enable

4\. Go to `IAM and admin` and `Roles` section click `Create Role.`

5\. Grant the role with these permissions and click `Create` ``` cloudasset.assets.analyzeIamPolicy cloudasset.assets.searchAllIamPolicies cloudasset.assets.searchAllResources ```

6. Grant the Role to the service account attached with the Bigquery source.

### Getting a Service Account JSON Key File For decube to connect with BigQuery, a JSON key file is required from a Service Account with the proper Roles assigned to it. 1. From the dashboard, click the hamburger (three lines indicating a menu), the top left of the browser next to the Google Cloud logo, and Search for *`API's and Services`.* 2. On that page, click on *`Credentials`*, at the top, click *`+ Create Credentials`* and then select *`Service account`*. 3. Fill in the *`Service account details form`,* we recommend * `Service account name`: decube, 1. Click *`Create and Continue`.* 2. On *`Grant this service account access to project`*, click on *`Select a role`,* the on the \_`Filter` \_ search, type *`decube data observability`* created previously and click to assign this role to the service account. Click Continue. 3. Skip *`Grant users access to this service account`* form and click `Done`. 4. On the Service Account page you will see a Service Account named decube, click on it. 5. On the options below the Service Account name, click on `Keys` 6. Click *`Add Key`* and then select *`Create new key`* from the drop-down. 7. For the `Key Type`, choose `JSON` and click Create. 8. You will automatically download the JSON file and can refer to the location of the file on your browser, save it somewhere easily accessible like your Desktop.