Google Cloud BigQuery Bidirectional Integration

BigQuery is Google Cloud’s serverless, highly scalable, fully managed analytics data warehouse for large- or small-scale data. A variety of storage options are available, both in the cloud or for local files. With BigQuery, it is possible to quickly create, store, modify, find, and inspect your data and its corresponding data types.

This document presents a bidirectional integration between Google Cloud BigQuery and Collibra. The Google Cloud BigQuery to Collibra synchronization retrieves multiple projects, datasets, tables, columns, and data, which in turn are converted into Collibra assets. For the reverse direction – Collibra to Google Cloud BigQuery – the synchronization updates the Google Cloud BigQuery descriptions of the datasets, tables, and columns based on the information defined on the Collibra Platform. There is no need to enable the process manually – the built-in scheduler ensures that integrations are executed with consistency.

Use Cases

You can efficiently preserve the dataset structure from:

• a specific BigQuery project

or

•  multiple BigQuery projects.

From Google Cloud BigQuery to Collibra

This integration offers the flexibility to:

• Transfer and preserve data and metadata

The integration can seamlessly transfer various types of organization’s metadata from Google Cloud BigQuery projects into the Collibra cloud platform. This can range from BigQuery projects and datasets to individual columns. Each of these can contain values such as names, types, descriptions, and other details.

• Preserve the existing structure

With this integration, the Google Cloud Platform BigQuery structure can be maintained onto the Collibra cloud platform. In this way, the organization can have a complete copy of their Google Cloud Platform structure onto the Collibra cloud platform. This means that all new and existing systems will continue working together with any information present on the Google cloud platform.

From Collibra to Google Cloud BigQuery

When the organization’s systems rely on Collibra, the following needs may arise:

• Working with third-party systems with Google Cloud support

This integration makes it possible to work with external systems lacking Collibra support, provided Google Cloud is supported. With BigQuery as the middleman, the integration recreates the structure of the third-party system in the existing Collibra environment.

• Making use of external datasets

Google BigQuery datasets from Google and other companies are supplied in a multitude of types. This integration facilitates introducing the richness of public or other external data in Collibra. The integration provides the required structure inside existing Collibra cloud platform systems to accommodate this. 

Elements in Scope

This integration will contain the following elements 

• Google Cloud BigQuery Projects
• Google Cloud BigQuery Datasets
• Google Cloud BigQuery Tables
• Google Cloud BigQuery Columns

To receive support on this item, you can engage our Professional Services team or post any questions in the Data Citizens Community.