Automated, field-level lineage from your data pipelines into your data catalog.
Your governance and risk teams spend three weeks of every audit cycle tracing it from Pentaho .kjb / .ktr files to Collibra by hand. The Lineage Documentation Agent from Dataworkz does it continuously, at app, table, and column resolution. Reactive fire drill becomes always-on readiness.
30 seconds. The Lineage Documentation Agent scans a folder of Pentaho .kjb / .ktr files. Properties placeholders resolved. Collibra queried for existing metadata. Field-level lineage CSV generated, uploaded to Collibra, visual lineage auto-generated at app, table, and column level. Three weeks of analyst work, completed in minutes.
Audited controls. Your CISO reads the same posture your compliance team does.
Operates on ETL source files only. Safe for regulated environments.
App, table, and column resolution. The granularity your regulators expect.
Structured for direct regulator review. Every run logged and reproducible.
Regulators keep tightening lineage requirements: BCBS 239, CCAR, SOX. Yet most teams still trace it by hand across Pentaho ETL and the scripts native connectors miss. When the regulator asks, it turns into a multi-team fire drill with inconsistent answers across tools, repeated every reporting cycle, every model retrain, every inquiry.
Collibra and Alation map your warehouse cleanly. The lineage breaks upstream, in the data pipelines their native connectors don't fully parse: Pentaho ETL, SQL, and custom scripts in source control. The Lineage Documentation Agent from Dataworkz is the automated bridge: it extracts what those connectors miss and pushes it straight into your catalog. You don't replace your catalog. You finally fill it.
The answer is in your catalog before the regulator asks. BCBS 239, CCAR, SOX, answered at field level and continuously updated. The audit cycle stops being a build and becomes a review.
Lineage isn't only for audits. It's the everyday question: what breaks if I change this job? With field-level lineage live in your catalog, your team sees the downstream blast radius before a change ships.
The Lineage Documentation Agent is a GenAI agent from Dataworkz. Point it at your data pipelines, Pentaho ETL or custom scripts, and it does the rest. No connectors to build, no manual tracing.
Parse your Pentaho .kjb / .ktr files and their .properties companions.
Substitute every ${variable} placeholder from your properties files, building a fully resolved pipeline.
A language model reads the resolved pipeline and identifies sources, targets, datasets, fields, and transformation rules.
Where a placeholder can't resolve locally, the agent calls your data catalog API (Collibra, Alation) to fill the gap automatically.
Assemble app, table, and column-level lineage records: full source-to-target data flow paths.
Serialize to a standardized, catalog-ready CSV and upload via the Import API. No manual editing.
| Analyst tracing by hand | Catalog connectors alone | Lineage Documentation Agent | |
|---|---|---|---|
| Time per cycle | 2–4 weeks of analyst time. | Manual entry per asset. | Minutes per run. Continuous. |
| Pentaho ETL parsing | Manual. | Not native. | Native. |
| Properties resolution | Manual. Source of error. | Not handled. | Automatic. |
| Field-level resolution | Where the analyst gets to. | Where someone catalogs. | App, table, column. |
| Audit-cycle effort | Quarterly fire drill. | Quarterly cataloging review. | A review, not a build. |
.kjb and .ktr files parsed natively, plus custom scripts in source control. The transformation logic comes through.
Environment placeholders resolved automatically. No analyst rework.
Direct upload format for Collibra and Alation. No middleware required.
App, table, column resolution. Granularity your regulators expect.
New Pentaho jobs trigger lineage updates automatically. Audit-readiness stops being quarterly.
Operates on ETL source files only. Safe for regulated environments.
The Lineage Documentation Agent is one agent on the Dataworkz enterprise agentic AI platform. It operates on ETL source files only, not production data. Lineage stays where you specify. Output is structured for direct regulator review. Every run is logged.
Data catalogs are excellent once metadata is cataloged, but their native connectors don't fully parse the pipeline layer (Pentaho ETL, SQL, custom scripts), which is exactly where lineage breaks. The Lineage Documentation Agent is the automated bridge: it extracts the upstream lineage those connectors miss and pushes it straight into Collibra or Alation. You're not replacing your catalog. You're finally filling it.
No. The Lineage Documentation Agent operates on your Pentaho ETL source files only. Your production environment is not touched.
First-run timing depends on the size of your ETL codebase. Most enterprises see initial lineage in Collibra within two weeks of deployment. Continuous updates run on every change.
Yes. Output is structured at app, table, and column resolution for direct regulator review.
Pentaho is the lead integration for this program. It also extracts lineage from custom scripts (SQL, Python, Java) in source control, and the pattern extends to Informatica, Talend, and Alteryx per deployment. On the catalog side it targets both Collibra and Alation. Confirm your stack during evaluation.
20 minutes. Bring a Pentaho job from your environment. We'll walk through the lineage the agent produces at app, table, and column resolution, ready for Collibra.