How do I configure BigQuery Credentials?

This guide outlines the process of creating a service account in Google Cloud Platform, granting the necessary permissions, and adding credentials to the ThinkData Platform to connect to BigQuery.

Step-by-Step Guide:

Step 1: Create a Service Account in Google Cloud Platform.

Follow these steps to create a service account for accessing BigQuery:

1. Log in to the Google Cloud Console and navigate to IAM & Admin > Service Accounts.
2. Create a new service account: 
   •  Click Create Service Account.   
   •  Enter a name for the service account (e.g., `thinkdata-platform-access`).     
   •  Optionally, add a description.
   • Click CREATE AND CONTINUE and then on DONE.

Step 2: Assign BigQuery User Role to the Service Account at Project Level.

1. Navigate to IAM & Admin > IAM.
2. Click the Edit Principal option next to the service account you created.
3. In the Grant Access to Project dialog, enter the service account's email in the New Principals text box.
4. In the Select a Role dropdown, assign the BigQuery User role.
5. Click SAVE to grant the BigQuery User role to the service account.

Step 3: Create and Download a Key File.

1. Navigate to IAM & Admin > Service Accounts.
2. Click Grant Access under the VIEW BY PRINCIPALS section.
3. Click on the Service Account Name.
4. In the service account details page, go to the Keys section.
5. Click ADD KEY > Create new key and select JSON.
6. Save the downloaded JSON key file securely.

Note:

• This JSON key file will be used to add BigQuery credentials to the ThinkData Platform in Step 5.
• If the key file is no longer required after being added to the ThinkData Platform, securely delete it to avoid unauthorized access or potential security risks.

Step 4: Grant Dataset Permissions to the Service Account.

After creating the service account and assigning the BigQuery User Role, follow the steps below to grant the necessary permissions for accessing specific datasets or tables in the BigQuery project.

Step 4.1: Basic Permissions

The following permissions are required for the ThinkData Platform to connect, list, and virtualize data assets from BigQuery.

a. Grant access to individual tables in a dataset.

1. Grant MetaData Viewer access on the dataset.

• In the BigQuery Console, click the View Actions option next to the dataset that contains the table.
• Select Share > Manage Permissions.
• In the Share Permissions dialog, click ADD PRINCIPAL.
• Enter the service account email in the New principals text box and assign the `BigQuery Metadata Viewer` role using the Select a role dropdown.
• Click SAVE to grant metadata access to the specific dataset.

      2. Grant access to a specific table in the dataset.

• In BigQuery Console, click the View Actions option next to the specific table name.
• Select Share > Manage Permissions.
• In the Share Permissions dialog, click ADD PRINCIPAL.
• Enter the service account email in the New principals text box and assign the `BigQuery Data Viewer` role using the Select a role dropdown.
• Click SAVE to grant view access to the specific table in the dataset.

b. (Optional) Grant access to all tables in a dataset.

• In BigQuery Console, click the View Actions option next to the dataset name.
• Select Share > Manage Permissions.
• In the Share Permissions dialog, click ADD PRINCIPAL.
• Enter the service account email in the New principals text box and assign the `BigQuery Data Viewer` role using the Select a role dropdown.
• Click SAVE to grant view access to all the tables in the dataset.

Step 4.2: Additional Permissions for Managing Data Products

If the service account needs to create or manage data products, grant the additional `BigQuery Data Editor` permission at Dataset level by following below steps:

• In BigQuery Console, click the View Actions option next to the dataset name.
• Select Share > Manage Permissions.
• In the Share Permissions dialog, click ADD PRINCIPAL.
• Enter the service account email in the New principals text box and assign the `BigQuery Data Editor` role using the Select a role dropdown.
• Click SAVE to grant write access to the dataset.

Alternatively, you may create a new service account and assign the BigQuery Data Editor role directly if more precise access control is required.

Step 5: Add BigQuery Credentials to the ThinkData Platform

After creating the Service Account and granting the required permissions , add it to the ThinkData Platform to enable data access by following the steps below. This step only needs to be completed once as long as you continue to use the same 'user' associated with these credentials.

Steps to add BigQuery credentials to the platform:

1. Open the platform and navigate to the Credentials page:
   • Data > Credentials > Create Credential
2. Enter the user name these credentials belong to (or any other identifying name) in the Credential Name field.
3. In the Type dropdown, select BigQuery.
4. Copy and paste the contents of the JSON key file created earlier in Step 3 in the Client Secrets text box.
5. Click Save to store the credentials.

Step 6: Using the Credentials

Now, you can use these credentials with:

• Source Connection to virtualize a data asset from BigQuery, or
• Destination Connection to create a Data Product with BigQuery as the destination, using existing datasets from the ThinkData Platform.

Summary of Key Terms:

• Service Account: The entity used for authentication with BigQuery, typically a Google Cloud service account with JSON credentials.
• Permissions: The roles assigned to the service account define its actions, such as reading data, creating tables, or modifying datasets.