Unsupported Screen Size: The viewport size is too small for the theme to render properly.
Published by Collibra Marketplace

AWS Glue ETL Lineage Integration (M4)

Package Documentation
Certified Free

We have made the decision to transition away from Collibra Connect so that we can better serve you and ensure you can use future product functionality without re-instrumenting or rebuilding integrations. For more information, please reach out to your Customer Success Manager.

Learn more about different methods to build integrations in Collibra Developer Portal.

Overview

As ETL developers use AWS Glue to move data around, AWS Glue allows them to annotate their ETL code to document where data is picked up from and where it is supposed to land i.e. source to target mappings. It is those source to target mappings and data lineage information that is read from ETL scripts in Glue and sent to Collibra platform. This integration empowers ETL developers to properly communicate technical data lineage, directly from their code, without the necessity to blindly scan their code, which typically results in illegible data lineage diagrams.

Media

More details

Release Notes

Changes:

  • Updated the integration to use Collibra Connect Hub v2.
  • Due to the external ID length restriction, for the tables and columns, the hashed value is used.
Compatibility
  • Collibra Cloud
  • Collibra 5.7 and newer
  • AWS Glue ETL Lineage Connector
  • Collibra CyberArk Connector v1.1.0
  • Mule ESB 4.2.2
Dependency
  • Collibra Connect Hub 2.0.1
  • Java Runtime Environment 1.8
License and Usage Requirements
  • Collibra Connect
  • AWS Glue

Release History

Version 1.1.1
June 11, 2020
Release Notes
    Changes:

  • Fixed issue with incoming HTTP headers.
  • Updated the AWS Glue connector version to v1.2.0.
Compatibility
  • Collibra DGC 5.6
  • Collibra Platform 5.7
  • AWS Glue ETL Lineage Connector 1.2.0
  • Collibra CyberArk Connector 1.1.0
  • Mule ESB 4.2.0
  • Anypoint Studio 7.3
Dependency
  • Collibra Connect Hub 1.0.1
  • Java Runtime Environment 1.8
License and Usage Requirements
  • Collibra Connect
  • AWS Glue
Version 1.1.0
April 24, 2020
Release Notes
Compatibility
  • Collibra DGC 5.6
  • Collibra Platform 5.7
  • AWS Glue ETL Lineage Connector 1.1.0
  • Collibra CyberArk Connector 1.1.0
  • Mule ESB 4.2.0
  • Anypoint Studio 7.3
Dependency
  • Collibra Connect Hub 1.1.0
  • Java Runtime Environment 1.8
License and Usage Requirements
  • Collibra Connect
  • AWS Glue
Version 1.0.1
February 11, 2020
Release Notes

Changes:
Added CMA file for DGC 5.7.

Compatibility
  • Collibra DGC 5.6
  • Collibra Platform 5.7
  • Mule ESB 4.2.0
  • Anypoint Studio 7.3
  • AWS Glue ETL Lineage Connector 1.2.0
Dependency
  • Collibra Connect Hub 1.1.0
  • Java Runtime Environment 1.8
License and Usage Requirements
  • Collibra Connect
  • AWS Glue

Reviews

Rating
Leave a review