Unsupported Screen Size: The viewport size is too small for the theme to render properly.

DBT Lineage to Collibra Integration

Published by: Collibra Marketplace
Latest version: 1.0.1
Released: August 3, 2022
Package Documentation
Community Offering

Your use of Community Offerings is subject to the Collibra Marketplace License Agreement. Read more.

Overview

For this specific integration (and all other Custom Integrations listed on the Collibra Marketplace), please read the following disclaimer:

  • This integration is a template that has been developed in cooperation with a few select clients based on their custom use cases and business needs.
  • While all effort has been made to encompass a range of typical usage scenarios, specific needs beyond this may require chargeable template customization.
  • With this in mind, we have made sure that the template is available as source code and readily modifiable to suit the client's particular use case.

Dbt is a service that combines modular SQL for data transformation using data pipelines. Analytics code can be deployed in a way that helps teams follow software engineering best practices like modularity, portability, CI/CD, and documentation. It is possible to produce table-level lineage using its lineage graph. However, column-level lineage is not yet supported.

This Spring Boot integration fetches compiled SQLs through a GraphQL call, converts the files, and writes files to lineage harvester path to process. Rather than representing dbt’s lineage graph on Collibra, the compiled SQL queries are extracted from dbt cloud services and placed in a SQL directory for the Collibra lineage harvester to process.

Use cases

This integration fetches compiled SQLs from the dbt platform written to the lineage harvester source path.

Elements in Scope

The integration is designed to retrieve the following elements:

  • SQL files

To receive support on this item, you can engage our Professional Services team or post any questions in the Data Citizens Community.

Media

More details

Release Notes
  • Lineage harvester methods removed from integration.
  • Extracted SQL timestamp name changed to model name.
  • Integration config resource bundle changed to externalize config class.
Compatibility
  • Spring Boot Framework
  • Eclipse IDE
  • Collibra Cloud
  • Collibra 5.7.5 and newer
Dependency
  • Java Runtime Environment 1.8
  • Spring Boot Integration Library
License and Usage Requirements
  • Collibra Catalog

Release History

Version 1.0.0
August 1, 2022
Release Notes

Initial release.

Main features:

This Spring Boot integration extracts compiled SQLs and uploads SQL files using the Collibra lineage harvester.

Compatibility
  • Spring Boot Framework
  • Eclipse IDE
  • Collibra Cloud
  • Collibra 5.7.5 and newer
Dependency
  • Java Runtime Environment 1.8
  • Spring Boot Integration Library
License and Usage Requirements
  • Collibra Catalog

Support is provided in accordance with the Marketplace Terms.

To receive support on this item, you can engage our Professional Services team or post any questions in the Data Citizens Community.

The following terms shall apply to the extent you receive the source code to this offering.

Notwithstanding the terms of the Binary Code License Agreement under which this integration template is licensed, Collibra grants you, the Licensee, the right to access the source code to the integrated template in order to copy and modify said source code for Licensee’s internal use purposes and solely for the purpose of developing connections and/or integrations with Collibra products and services.

Solely with respect to this integration template, the term “Software,” as defined under the Binary Code License Agreement, shall include the source code version thereof. Except with respect to the foregoing, all remaining terms of the Binary Code License Agreement shall apply to the license of integration template hereunder.

Reviews

Rating
Leave a review