Dremio to Collibra Integration
Overview
Dremio is a platform that provides graph views for all your database sources, including external ones. You can easily organize and search through your databases by attaching tags. The four types of database sources that are supported are:
- datasets – data can be imported in multiple formats such as JSON, CSV, and many more
- virtual datasets (VDS) – these are reflections of physical datasets or other virtual datasets
- spaces – provide a way to group datasets by a common theme such as projects, departments, or geographic regions. VDS can be stored here.
- data lakes – these allow data imports from cloud services such as Amazon S3, Azure Storage, Google Cloud storage, and many more.
Data can be created and modified inside these database sources via SQL statements within the web UI.
The integration presented here makes it straightforward for you to transfer schemas, tables, table statistics, catalog and table tags from Dremio to your Collibra Platform instance.
A CMA file is also included. Once this CMA file is imported within your Collibra instance, it will ensure that all the custom prerequisites will be generated automatically.
Alternative ways to try out this integration are via the supplied JAR file or Docker image with the compiled build version of the integration. These contain all the required dependencies and source code. After a minimal number of required changes as described in this document, users can run the integration using one simple command.
Use cases
You can reliably preserve the database structures and tags from Dremio to Collibra by transferring:
- Catalogs, Table metadata, and Table statistics
- Table Tags of Datasets and Spaces.
Elements in Scope
The integration is designed to retrieve the following elements:
- Table metadata
- Table Statistics
- Table Tags
To receive support on this item, you can engage our Professional Services team or post any questions in the Data Citizens Community.
Media
More details
Release Notes
- Security Dependency Vulnerability Updates
- Dockerfile
- Integration Library version upgraded to v1.1.10
- Spring Boot version upgraded to v2.7.6
Compatibility
- Spring Boot Framework
- Collibra Data Intelligence Cloud
- Collibra Data Intelligence On-Prem
Dependency
- Java Runtime Environment 1.8
- Spring Boot Integration Library
- Spring Boot Framework
- Dremio JDBC Driver 19.1.0
License and Usage Requirements
Release History
Release Notes
- Included CMA File
- Included Asset IDs within the integration
- Dependency upgrades
Compatibility
- Spring Boot Framework
- Eclipse IDE
- Collibra Data Intelligence Cloud
- Collibra Data Intelligence On-Prem
Dependency
- Java Runtime Environment 1.8
- Spring Boot Integration Library
- Spring Boot Framework
- Dremio JDBC Driver 19.1.0
License and Usage Requirements
Release Notes
- Updated Spring starter parent dependency to version 2.6.6
- Updated Collibra Integration Library to version 1.1.5
Compatibility
- Spring Boot Framework
- Eclipse IDE
- Collibra Data Intelligence Cloud
- Collibra Data Intelligence On-Prem
Dependency
- Java Runtime Environment 1.8
- Spring Boot Integration Library
- Spring Boot Framework
- Dremio JDBC Driver 19.1.0
License and Usage Requirements
Release Notes
Initial release. Main features:
- synchronizes Dremio Table metadata to Collibra
- synchronizes Dremio Tag metadata to Collibra
- Dremio JDBC Connector Authentication
- Dremio HTTP Request Authentication
Compatibility
- Spring Boot Framework
- Eclipse IDE
- Collibra Data Intelligence Cloud
- Collibra Data Intelligence On-Prem
Dependency
- Java Runtime Environment 1.8
- Spring Boot Integration Library
- Spring Boot Framework
- Dremio JDBC Driver 19.1.0
License and Usage Requirements
See existing Q&A in the Data Citizens Community
Browse discussions with customers who also use Community Offerings from the Collibra Marketplace.
Start a New Topic in the Data Citizens Community
Collibra-hosted discussions connect you to other customers who use this app.
The following terms shall apply to the extent you receive the source code to this offering.
Notwithstanding the terms of the Binary Code License Agreement under which this integration template is licensed, Collibra grants you, the Licensee, the right to access the source code to the integrated template in order to copy and modify said source code for Licensee’s internal use purposes and solely for the purpose of developing connections and/or integrations with Collibra products and services.
Solely with respect to this integration template, the term “Software,” as defined under the Binary Code License Agreement, shall include the source code version thereof. Except with respect to the foregoing, all remaining terms of the Binary Code License Agreement shall apply to the license of integration template hereunder.
