Mainframe Lineage You Can Defend: Why Every Edge Needs Provenance

PDI Provenance Ledger: evidence beside every proposed edge

PDI methodology: the ledger below separates Verified, Inferred and Unresolved evidence states. These are the article’s recommended review semantics, not IBM-defined statuses or a claim that the current scanner exposes three identically named UI states. PDI product material describes source-line provenance and an explicit Unverified boundary.

Verified means the bounded relationship is supported by an identified source statement and has passed the stated review; it does not prove that the program executed. Inferred means a plausible relationship lacks sufficient direct proof and must not be counted as verified. Unresolved means a reference cannot yet be bound to a determinate relationship. Confidence labels are not invented statistical probabilities.

External metadata context: IBM’s current Db2 for z/OS SYSIBM.SYSDEPENDENCIES documentation describes catalog dependencies between objects, including supported column metadata. We cite that limited metadata/dependency context only. It is not proof of an application’s complete end-to-end lineage, runtime execution or PDI-specific provenance semantics.

All members, statements, timestamps and decisions in this example are fictional. A real ledger should retain the immutable source snapshot/hash, parser/rule version and authorized reviewer evidence behind each entry, with a new record when the evidence or status changes.

Treat each edge as a claim with evidence

A lineage edge asserts that one object or field relates to another through a program, job, transformation, or data movement. Record the endpoints, relationship type, source member, line number or statement, parser or extraction context, and verification status. Preserve the exact evidence reference so a steward can inspect the basis rather than trusting a diagram alone.

The provenance should be specific enough to distinguish similarly named programs and datasets. Include the repository or extraction batch and a stable path or identifier where available. If the evidence is a parsed statement, say so. If it is a manually confirmed relationship, record the reviewer and confirmation context separately.

Record the basis, not just the arrow

A reviewable edge names its source and target, relation type, extraction or scan reference, supporting member and statement, resolution state, and known limits. Keep evidence references stable enough for a reviewer to retrieve the exact material used for that scan.

Capture mainframe source context

Mainframe metadata may come from catalog unloads, DBD source, FDT reports, dictionary extracts, COBOL, JCL, Natural, or IDMS DML. The lineage record should identify the source member and its location in the extracted corpus, not just the program name shown in a catalog. Line and statement references make review practical when names are reused or source changes over time.

Tie a scan to a corpus version or content fingerprint and record the scanner build and parser configuration. If a member changes, the affected objects and edges can then be re-evaluated against the new source rather than appearing as unexplained changes in the catalog.

Use verified and unverified states deliberately

A verified state should mean that the defined evidence threshold was met for the claim. An unverified state should remain visible when a reference cannot be resolved, a required source is missing, or the relationship depends on information unavailable in the extracted material. Do not silently omit the edge or promote a plausible guess to a verified fact.

Make the reason actionable: unresolved object, missing member, unsupported construct, ambiguous name, or runtime-only behavior. A steward can then decide whether to provide additional evidence, request source-owner review, or accept a documented limitation. Status should communicate uncertainty in text as well as any visual styling.

Do not confuse parsing with runtime proof

A parser can identify a source statement that appears to read or update an object. That establishes a source-level relationship in the material analyzed. It does not, by itself, demonstrate that the statement ran in production, how often it ran, which dynamic target it resolved to, or what data values actually flowed at runtime.

Runtime proof requires suitable operational evidence such as execution logs, runtime instrumentation, job history, or other control records, and those sources have their own coverage and retention limits. Keep static, source-derived lineage distinct from observed runtime lineage. The distinction makes the catalog more credible, not less useful.

Build governance workflows around provenance

Use source-backed lineage to support impact analysis, migration planning, audit inquiry, and stewardship prioritization. Before changing a critical table or copybook, a team can inspect the programs and jobs connected to it, then route uncertain edges to owners for confirmation. The catalog becomes a queue for decisions rather than a decorative graph.

Set review policy by consequence. A low-risk analytical relationship may be acceptable as source-derived evidence; a control-critical dependency may require an owner confirmation or runtime corroboration. Record the policy, reviewer, and accepted limitations so future users know what “verified” means in that context.

Keep evidence stable through rescans

A rescan should make it possible to distinguish a changed source from a changed parser or extraction method. Store run identifiers, content fingerprints, tool versions, and the prior state needed to explain additions, removals, or status changes. Retain historic evidence according to access and retention policy, especially where source code or operational metadata is sensitive.

When evidence is corrected, preserve the earlier decision and the reason it changed. Avoid overwriting a bad edge without a trace; otherwise an auditor or engineer cannot understand why a past design decision relied on it.

Test lineage claims against real source

Choose a small but varied corpus: a straightforward static reference, a relationship with an ambiguous name, a missing source member, and a construct requiring runtime resolution. Ask source owners to inspect the cited member, line, and statement. Confirm that verified and unverified results reflect the evidence actually present, and document what additional evidence would change the status.

PDI’s Metadata Scanner – Mainframe overview describes source-line provenance and an explicit Unverified state. Its walkthrough shows the catalog experience; the whitepaper provides deeper engineering context. Use those resources to frame a technical review, then validate the behavior against your own extraction pipeline and governance policy. The strongest lineage claim is not the one with the most edges; it is the one whose evidence, scope, and limits can be independently understood.