Custom Technical Lineage python scripts
Overview
This offering contains python scripts providing:
- Script converting custom technical lineage (v1) into the new batch format
- Classes and helper functions for creating custom technical lineage programatically
Converting existing custom technical lineage (v1) to new batch format
The script translate_to_batch_format.py will transform an existing custom technical lineage you have into the new format, you can use it by running:
python3 translate_to_batch_format.py <v1-dir> <target-dir> [–migrate_source_code]
v1-dir is an existing directory containing the lineage.json file and target-dir is a target directory in which the files will be created in the new format. Target directory will be created if it does not exist.
–migrate_source_code is an optional flag to invoke translation of source codes as well, because source codes are optional in the new format.
Classes and helper functions for creating custom technical lineage programatically
models.py contain classes that can be helpful when creating custom technical lineage programatically, especially in combination with the helper functions available in helper.py. This allows you to focus on the creation of your assets and lineage relationships while the correct formatting and the creation of the files is taken care of for you. There are 2 scripts levering these classes and helper functions that can be used as an example: example.py and example_with_props.py.
Requirements
The scripts make use of pydantic and have been tested with python version 3.11.
More details
Release Notes
This release provides the code via GitHub repo.
Compatibility
- Collibra Data Intelligence Cloud
Dependency
- N/A
License and Usage Requirements
Release History
Version v1
November 20, 2023
Release Notes
- translation script converting custom technical lineage v1 into new custom technical lineage batch format
- classes, helper functions and examples how to generate custom technical lineage programatically
Compatibility
- Collibra Data Intelligence Cloud
Dependency
- N/A
License and Usage Requirements
Reviews
Leave a review