Airflow is used to author workflows as Directed Acyclic Graphs (DAGs) of tasks. The Airflow scheduler executes your tasks on an array of workers while following the specified task dependencies. The rich user interface makes it easy to visualize pipelines running in production, monitor progress and troubleshoot issues when needed.
This integration will allow gathering of available DAGs and the corresponding DAG Tasks from Airflow and represent the relationship in Collibra so that orchestration can be shown how it fits in the data pipeline. If there are any changes in the DAG, DAG attributes, DAG Tasks or DAG Tasks attributes, the changes will be extracted and updated in Collibra. By bringing the orchestration in Collibra, the process can be brought under governance.
This Spring Boot Integration is updated to use collibra-core classes to exchange data with Collibra.
This integration will gather available DAGs and the corresponding DAG Tasks from Airflow and create relationship in Collibra so that data orchestration will be shown in Collibra. If there are any changes in the DAG, DAG attributes, DAG Tasks and DAG Tasks attributes, the changes will be extracted and updated in Collibra.
April 2002 update: upgraded to springboot framework version 2.6.6 in response to cve-2022-22965
- Spring Boot Framework
- Airflow API
- Collibra Cloud
- Airflow API
- Java Development Kit v1.11
- Spring Boot framework v2.6.6
License and Usage Requirements
- Collibra Catalog
No previous versions of this listing is available.
To receive support on this item, you can engage our Professional Services team or post any questions in the Data Citizens Community.
The following terms shall apply to the extent you receive the source code to this offering.
Notwithstanding the terms of the Binary Code License Agreement under which this integration template is licensed, Collibra grants you, the Licensee, the right to access the source code to the integrated template in order to copy and modify said source code for Licensee’s internal use purposes and solely for the purpose of developing connections and/or integrations with Collibra products and services.
Solely with respect to this integration template, the term “Software,” as defined under the Binary Code License Agreement, shall include the source code version thereof. Except with respect to the foregoing, all remaining terms of the Binary Code License Agreement shall apply to the license of integration template hereunder.