Unsupported Screen Size: The viewport size is too small for the theme to render properly.

Custom Technical Lineage python scripts

Published by: Collibra Marketplace
Latest version: v1.0
Released: November 20, 2023
Contact Publisher
PackageDocumentation
Community Offering

Your use of Community Offerings is subject to the Collibra Marketplace License Agreement. Read more.

Overview

For this specific integration (and all other Custom Integrations listed on the Collibra Marketplace), please read the following disclaimer:

  • This integration is a template that has been developed in cooperation with a few select clients based on their custom use cases and business needs.
  • While all effort has been made to encompass a range of typical usage scenarios, specific needs beyond this may require chargeable template customization.
  • With this in mind, we have made sure that the template is available as source code and readily modifiable to suit the client's particular use case.

This offering contains python scripts providing:

  1. Script converting custom technical lineage (v1) into the new batch format
  2. Classes and helper functions for creating custom technical lineage programatically

Converting existing custom technical lineage (v1) to new batch format

The script translate_to_batch_format.py will transform an existing custom technical lineage you have into the new format, you can use it by running:

python3 translate_to_batch_format.py <v1-dir> <target-dir> [–migrate_source_code]
v1-dir is an existing directory containing the lineage.json file and target-dir is a target directory in which the files will be created in the new format. Target directory will be created if it does not exist.
–migrate_source_code is an optional flag to invoke translation of source codes as well, because source codes are optional in the new format.
Classes and helper functions for creating custom technical lineage programatically
models.py contain classes that can be helpful when creating custom technical lineage programatically, especially in combination with the helper functions available in helper.py. This allows you to focus on the creation of your assets and lineage relationships while the correct formatting and the creation of the files is taken care of for you. There are 2 scripts levering these classes and helper functions that can be used as an example: example.py and example_with_props.py.

Requirements

The scripts make use of pydantic and have been tested with python version 3.11.

More details

Release Notes

This release provides the code via GitHub repo.

Compatibility
  • Collibra Data Intelligence Cloud
Dependency
  • N/A
License and Usage Requirements

Release History

Version v1
November 20, 2023
Release Notes
  • translation script converting custom technical lineage v1 into new custom technical lineage batch format
  • classes, helper functions and examples how to generate custom technical lineage programatically
Compatibility
  • Collibra Data Intelligence Cloud
Dependency
  • N/A
License and Usage Requirements

Reviews

Rating
Leave a review