Unsupported Screen Size: The viewport size is too small for the theme to render properly.

Collibra and Airflow Integration

Published by: Collibra Marketplace
Latest version: 1.0.1
Released: March 22, 2022
Contact Publisher
PackageDocumentation
Community Offering

Your use of Community Offerings is subject to the Collibra Marketplace License Agreement. Read more.

Overview

“Community Offerings” are offerings published by third parties or Collibra within the Collibra Marketplace community.  Community Offerings are free and made available “as is” by Collibra.  Your master commercial agreement with Collibra for the Collibra Service DOES NOT apply to your use of the Community Offerings (including any warranties, support services and service levels referenced therein).   Your use of Community Offerings is subject to the Collibra Marketplace License Agreement, as may be modified from time to time by Collibra.  Collibra encourages users to seek assistance for Community Offerings from peers and other Data Citizens via the Collibra Community platform.

For this specific integration (and all other Custom Integrations listed on the Collibra Marketplace), please read the following disclaimer:

  • This integration is a template that has been developed in cooperation with a few select clients based on their custom use cases and business needs.
  • While all effort has been made to encompass a range of typical usage scenarios, specific needs beyond this may require chargeable template customization.
  • With this in mind, we have made sure that the template is available as source code and readily modifiable to suit the client's particular use case.

Airflow is used to author workflows as Directed Acyclic Graphs (DAGs) of tasks. The Airflow scheduler executes your tasks on an array of workers while following the specified task dependencies. The rich user interface makes it easy to visualize pipelines running in production, monitor progress and troubleshoot issues when needed.

This integration will allow gathering of available DAGs and the corresponding DAG Tasks from Airflow and represent the relationship in Collibra so that orchestration can be shown how it fits in the data pipeline. If there are any changes in the DAG, DAG attributes, DAG Tasks or DAG Tasks attributes, the changes will be extracted and updated in Collibra. By bringing the orchestration in Collibra, the process can be brought under governance.

To receive support on this item, you can engage our Professional Services team or post any questions in the Data Citizens Community.

Media

More details

Release Notes

This Spring Boot Integration is updated to use collibra-core classes to exchange data with Collibra.

This integration will gather available DAGs and the corresponding DAG Tasks from Airflow and create relationship in Collibra so that data orchestration will be shown in Collibra. If there are any changes in the DAG, DAG attributes, DAG Tasks and DAG Tasks attributes, the changes will be extracted and updated in Collibra.

April 2002 update: upgraded to springboot framework version 2.6.6 in response to cve-2022-22965

Compatibility
  • Spring Boot Framework
  • Airflow API
  • Collibra Data Intelligence Cloud
Dependency
  • Airflow API
  • Java Development Kit v1.11
  • Spring Boot framework v2.6.6
License and Usage Requirements

Need help? We have a coaching session that can help you with:

  • Initial information on the integration and prerequisites for custom Springboot integrations.
  • Expert session on debugging and development support for custom Springboot integrations.

Book here

See existing Q&A in the Collibra Community

Browse discussions with customers who also use this app. 

Start a New Topic in the Collibra Community

Collibra-hosted discussions will connect you to other customers who use this app.

 

 

The following terms shall apply to the extent you receive the source code to this offering.

Notwithstanding the terms of the Binary Code License Agreement under which this integration template is licensed, Collibra grants you, the Licensee, the right to access the source code to the integrated template in order to copy and modify said source code for Licensee’s internal use purposes and solely for the purpose of developing connections and/or integrations with Collibra products and services.

Solely with respect to this integration template, the term “Software,” as defined under the Binary Code License Agreement, shall include the source code version thereof. Except with respect to the foregoing, all remaining terms of the Binary Code License Agreement shall apply to the license of integration template hereunder.

Reviews

Rating
Leave a review