Google Big Query

Prerequisites

  1. Install latest Incorta version. Visit Install and Upgrade Incorta for instructions.
  2. Set up a Google Cloud Service Account. If you do not already have an account, you can create a Google Cloud account here.
  3. Create a Google Cloud Project. After setting up your Google Cloud account, create a new project for Incorta, then add a data set and table to this new Google Project. This will be used to test the connection between Google and Incorta. Visit Identifying Projects on the Google Cloud site for additional help with setting up your project.
  4. Download Big Query Driver: Before connecting a Google BigQuery data source, you must first download and configure the BigQuery driver for Incorta. The driver is in a JAR file. The BigQuery JAR file must exist in both the CMC and the Incorta Services installation path.
  5. Add BigQuery as Data Source: Follow the instructions to add a new data source. When asked for the specific parameters in the New Data Source dialog, you will need to have the following information to complete the connection.

BigQuery Connector Parameters

After you choose BigQuery as a data source, the New Data Source dialog will request these parameters:

  • Client ID: This is found in the Key File as client_id.
  • Private Key ID : You can find this value in the private_key_id field of your service account JSON file.
  • Project ID: Unique identifier assigned to each Google Cloud project found in the Dashboard page of the Google Cloud Console.
  • Google Service Account Email: Email address associated with the Google Service Account you are connecting. Example: <ServiceAccountName>@<ProjectName>.iam.gserviceaccount.com
  • Private Key Path: Authenticates the Service Key Path. Private key path should include the full path. Example:

    OAuthPvtKeyPath=D:\Incorta\connectors\feisty-truth-241410-c89e94d36056.json

  • Connection Pool: 30
  • Connection Properties: Path of the JSON file key downloaded from Google Cloud Service accounts. Refer to Using a Google Service Account for more details.

Download BigQuery Driver

  1. Download the Simba JDBC driver for BigQuery from this link.
  2. Extract the zip file, SimbaJDBCDriverforGoogleBigQuery42_1.2.1.1001, to the following installation paths:

    • installationpath/IncortaNode/extensions/connectors/bigquery
    • installationpath/cmc/extensions/connectors/bigquery
  3. Ensure that all files under bigquery are .jar only.

Configure BigQuery Driver

After downloading the Simba JDBC driver for BigQuery, you will need to configure the driver in order to authenticate the connection between Incorta and your Google BigQuery service account.

To configure service account authentication:

  1. Set the OAuthType property to 0.
  2. Set the ProjectID property to the name of your BigQuery project.
  3. Set the OAuthServiceAcctEmail property to your Google service account email address.
  4. Set the OAuthPvtKeyPath property to the full path to the key file that is used to authenticate the service account email address. This parameter supports keys in .pl2 or .json format.

See Using a Google Service Account on the Simba site for more details.

Download a Key File

Visit the Create Service Account Key page at the Google API console site to download a JSON key file for the Service account you want to connect to Incorta.

  • The Key file contains both the client ID and private key ID.
  • The Driver class for the connection is: com.simba.googlebigquery.jdbc41.Driver

The following is an example of a connection string authenticating a connection to a service account:

jdbc:bigquery://https://www.googleapis.com/bigquery/v2:443;
ProjectId=MyBigQueryProject;OAuthType=0;
OAuthServiceAcctEmail=<Service Email Address>;
OAuthPvtKeyPath=<Path to Key File>;

Example of Path to Key File parameter:

OAuthPvtKeyPath=D:\Incorta\connectors\feisty-truth-241410-c89e94d36056.json

Example of the Service Email Address parameter: OAuthServiceAcctEmail=bigqueryservice@feisty-truth-241410.iam.gserviceaccount.com

For additional information on managing BigQuery service account credentials, visit Setting Up Authentication for Server to Server Production Applications on the Google Cloud site.


© Incorta, Inc. All Rights Reserved.