WorksComputer Science
ProvCards: Artifact-Oriented Provenance for Heterogeneous Scientific Workflows
Abstract
Reproducibility in scientific workflows is hindered by heterogeneous components and execution environments, as well as fragmented provenance capture. Although existing approaches offer established semantics and workflow-specific solutions, provenance remains difficult to capture, exchange, and compose across workflow boundaries. We present ProvCards, an artifact-oriented approach that treats provenance as a first-class artifact grounded in the W3C PROV Data Model. Individual workflow components can generate cards that are stored, exchanged, reused, and composed independently of any workflow system or centralized execution context. ProvDecks group related cards into portable scopes while preserving their identities. ProvGraphs provide derived views for provenance reconstruction, querying, visualization, and analysis. Together, these representations support coherent, task-specific provenance views across heterogeneous components, facilitating traceability, reuse, and reproducibility. We implement ProvCards using a graph-based infrastructure and evaluate them in a scientific data-processing workflow. The results demonstrate independent provenance capture at the component level, composition across component boundaries, and reconstruction for querying and analysis. ProvCards provide a portable, composable, and incrementally extensible foundation for provenance management in heterogeneous scientific workflows.
Introduction
Provenance is increasingly important for verifying data and reproducing computational results. Modern scientific data processing, however, commonly spans databases, files, scripts, notebooks, workflow engines, external services, and distributed execution environments [2, 4]. Since no single system controls or observes the complete processing lifecycle, the provenance of a result is typically distributed across multiple technical and organizational boundaries. The participating systems already generate provenance-relevant information by design. Logs, execution traces, catalogs, and metadata records contain input and output references, commands, parameters, timestamps, software versions, responsible actors, and execution status. Yet this evidence is represented in system-specific formats and remains tied to the components that generated it. It therefore cannot readily be exchanged or composed into consistent end-to-end provenance. Existing approaches provide standardized provenance semantics, workflow-integrated capture, provenance processing, or workflow-level packaging, but they lack an independently manageable unit for incrementally capturing and composing provenance across heterogeneous components.
We address this gap with the ProvCard Model, an artifact-oriented approach grounded in the W3C PROV Data Model [10]. A ProvCard is an independently manageable unit of provenance capture and exchange. Each card represents a provenance-relevant entity, activity, or agent and can refer explicitly to elements captured by other components. ProvCards can be generated incrementally during or after execution and can be identified, stored, retrieved, and exchanged independently of a shared workflow environment. Their contribution lies not in introducing new provenance semantics, but in providing a portable operational unit for provenance that remains usable when no single system controls the complete data-processing lifecycle. Within this layered model, ProvCards form the atomic elements. Collections of related cards, called ProvDecks, establish portable scopes for composing and exchanging provenance while preserving the identity of the individual cards. ProvGraphs provide derived graph representations for reconstructing, querying, visualizing, and analyzing the relationships captured by the cards.
Figure 1 illustrates the proposed approach. The left-hand side shows data artifacts and processing activities distributed across independently operating systems, including a database, a script or notebook environment, and an external service. Each component generates ProvCards for the provenance elements it observes. Related cards can subsequently be collected in a ProvDeck, stored, and reconstructed as a ProvGraph. The highlighted elements illustrate an exemplary provenance question: the relevant path through the reconstructed graph identifies the data and processing steps that contributed to a selected result. Fragmented execution evidence thus becomes connected and queryable without requiring all processing steps to be executed or governed by a single workflow system.

Figure 1. From components distributed across heterogeneous systems to composable provenance artifacts. Independent data-processing components generate ProvCards for the provenance elements they observe. Related cards can be collected in ProvDecks, stored, and reconstructed as a ProvGraph for querying and analysis. The highlighted elements illustrate an exemplary provenance question and the provenance path providing its answer.
The main contributions of this paper are:
An artifact-oriented provenance model that defines an independently manageable unit of provenance capture and exchange, extending provenance management beyond the control boundaries of individual database and workflow systems (§ 4).
A layered representation consisting of ProvCards, ProvDecks, and ProvGraphs for incremental capture, cross-system composition, controlled sharing, and analysis (Sections 4.1 & § 4.3).
Formalized transformations and information-preservation properties between the three representations, including information reduction through abstraction, anonymization, and selective disclosure (§ 4.4).
An implementation and empirical validation demonstrating independent provenance capture, composition, reconstruction, and querying in a scientific data-processing workflow (§ 5–§ 7).
Related Work
To position the proposed artifact-oriented abstraction, we first examine how existing work addresses provenance semantics, capture, packaging, and lifecycle management. This comparison establishes the dimensions from which the challenges and requirements in § 3 are derived.
Provenance Models and Capture. The W3C PROV standard provides a uniform and machine-interpretable model for representing provenance. PROV-DM [10] distinguishes entities, activities, and agents, with relationships such as used, wasGeneratedBy, and wasDerivedFrom capturing dependencies between them. PROV-O [5] provides an OWL-based representation of these concepts. This family of standards establishes interoperable provenance semantics and serialization mechanisms, but they do not prescribe how provenance fragments generated independently by heterogeneous components should be captured, managed, and composed throughout their lifecycle.
Beyond such workflow-integrated capture, workflow engines, applications, operating systems, and supporting infrastructure generate logs and execution traces containing timestamps, commands, parameters, software versions, resource references, and execution status. These records constitute valuable provenance evidence even when provenance capture was not an explicit objective of the executing component. Their formats, identifiers, granularity, and semantics nevertheless differ across systems. Consequently, turning these records into consistent end-to-end provenance requires component-specific extraction and mapping.
Provenance Processing and Packaging. Provenance-processing frameworks operate on structured provenance representations. ProvToolbox [9], for example, supports the creation, validation, and transformation of PROV documents, while PROV-Template [7] enables declarative generation of provenance graphs. Such frameworks provide important processing capabilities but assume that the provenance information to be processed has already been mapped to an appropriate representation. Their primary purpose is therefore not to define a lifecycle for independently generated provenance fragments.
These approaches demonstrate that workflow provenance can be represented and exchanged independently of the technology used to execute a workflow. Their primary aggregation scope, however, is typically a crate, research object, or workflow execution. This scope may not yet be available when an individual component generates provenance evidence. This leaves open how independently generated provenance fragments can be managed before a complete workflow-level object is available and how they can later be incorporated into such an object.
Documentation. Documentation artifacts such as Datasheets for Datasets [3], Model Cards [8], Data Cards [11], and related dataset-documentation initiatives [6] improve the transparency and reuse of individual research assets. They describe properties such as an artifact’s intended use, composition, creation process, evaluation, limitations, licensing, and ethical considerations. Such documentation is related to provenance because it records contextual information about the origin and use of an artifact. Its primary focus, however, is the description of a dataset or model as a static resource. It does not generally represent the dynamic relationships among data, activities, agents, intermediate artifacts, and execution events that arise while a workflow is running. Documentation artifacts and dynamic provenance representations therefore address complementary concerns: the former communicate the context, intended use, and limitations of a resource, whereas the latter represent relationships among artifacts, processing activities, and responsible agents.
Controlled Disclosure. Provenance may contain sensitive information about datasets, users, workflow logic, and execution environments. Prior work distinguishes data, module, and structural privacy in scientific workflow provenance [1]. This motivates a distinction between provenance capture and disclosure: information may need to be omitted, abstracted, or anonymized before provenance is exchanged.
Research Gap. Existing approaches provide standardized provenance semantics, workflow-integrated capture, provenance processing, documentation, and workflow-level packaging. However, provenance evidence generated by independently operating components remains coupled to system-specific records or is managed only after a workflow-level aggregation scope has been established. The approaches discussed above do not make an independently generated provenance fragment the primary unit of lifecycle management and subsequent cross-system composition. Consequently, it remains difficult to persistently identify, store, exchange, and compose such fragments across heterogeneous system boundaries. To address this gap, the ProvCard Model introduces an independently manageable, artifact-level unit for provenance fragments grounded in PROV-DM. Its contribution concerns the granularity and lifecycle of provenance management rather than new provenance semantics. ProvCards are intended to complement, rather than replace, workflow-integrated capture, PROV documents, and workflow-level packages such as Research Objects. The resulting challenges and requirements are derived in Section 3; the model itself is introduced in Section 4.
Challenges and Requirements
Challenges. Existing approaches leave five challenges for provenance management in heterogeneous scientific workflows.
C1 Heterogeneous workflow environments. Scientific workflows combine workflow engines, scripts, databases, external services, and distributed infrastructures whose provenance mechanisms are often incompatible.
C2 Independent and fragmented provenance capture. Workflow components capture provenance independently, using different formats and levels of detail, which complicates composition and end-to-end reconstruction.
C3 Limited portability and composability of provenance. Although PROV-DM provides interoperable semantics, provenance is often embedded in workflow-specific records
or packages and cannot be exchanged and composed independently.
C4 Missing lifecycle for provenance artifacts. Independently generated provenance fragments lack explicit mechanisms for persistent identification, storage, retrieval, exchange, and composition.
C5 Provenance detail vs. capture overhead. Detailed provenance improves reconstruction and analysis but increases computational and organizational overhead. Capture must therefore remain lightweight for integration into existing workflows.
Requirements. Based on these challenges, we derive seven requirements for artifact-oriented provenance management across heterogeneous scientific workflows:
R1 Machine-readable and Extensible Representation. Represent PROV-DM entities, activities, agents, and relationships in a structured, machine-readable format extensible with domain-specific metadata.
R2 Engine Independence. Operate independently of workflow engines, programming languages, and software environments without requiring replacement of existing infrastructure.
R3 Lightweight and Incremental Capture. Allow workflow components to generate provenance artifacts independently and with limited overhead.
R4 Support for Heterogeneous Resources. Provide a common abstraction for datasets, scripts, databases, computational models, and external services.
R5 Provenance Artifact Management. Support persistent identification, storage, retrieval, and exchange independently of the generating environment.
R6 Portability and Composability. Preserve artifact identities and relationships during exchange and composition across systems and institutions.
R7 Provenance Reconstruction and Analysis. Support reconstruction of provenance graphs for dependency, lineage, and responsibility analysis.
The Provenance Model
We address the requirements identified in Section 3 through an artifact-oriented provenance model to independently capture, exchange, and compose provenance in heterogeneous workflow environments.
Design Principles. The design of the proposed provenance model follows five principles that support provenance capture and exchange across heterogeneous workflow environments:
Independence enables individual workflow components to generate provenance artifacts without requiring access to a global workflow representation or centralized provenance service.
Portability ensures that provenance artifacts can be stored and exchanged independently of the workflow system that generated them.
Composability enables independently generated provenance artifacts to be combined into larger provenance structures
while preserving the identity and semantics of the individual artifacts.
Immutability ensures that a provenance artifact represents a specific observation of a workflow component at a particular point in time. Changes to the underlying workflow, data, or metadata therefore result in new provenance artifacts rather than modifying existing ones.
Lightweightness allows provenance to be captured incrementally by individual workflow components without requiring a complete workflow representation or centralized provenance infrastructure.
Provenance Artifacts
The ProvCard Model introduces a layered, artifact-oriented representation for provenance management across heterogeneous scientific workflows. It preserves the provenance semantics of PROV-DM [10], while introducing an explicit artifact-level abstraction for provenance capture, composition, exchange, and analysis. The model distinguishes three complementary representations: ProvCards, ProvDecks, and ProvGraphs. ProvCards represent independently manageable provenance artifacts, ProvDecks provide a composition and exchange scope for collections of artifacts, and ProvGraphs provide a derived representation for provenance analysis.
ProvCards. A ProvCard represents the fundamental provenance artifact of the proposed model. Each ProvCard describes a single provenance-relevant element corresponding to an entity, activity, or agent according to PROV-DM. ProvCards are designed as independent and self-contained artifacts. For example, a measurement instrument may generate a ProvCard describing a raw dataset, while a processing tool creates another ProvCard describing a transformation activity. The relationship between both artifacts is established through explicit provenance references rather than through a shared workflow environment.
Let denote the set of PROV types: . Let denote the set of Prov subtypes, i.e., , where the first three labels define entities, and the last two labels define agents. code is a specialized activity. The available subtypes depend on the underlying ontology. Let further denote the set of supported provenance relation types based on PROV-DM: .
Definition 4.1 (ProvCard). Let denote the set of PROV types and the set of PROV relation types. Let denote a set of keys and a set of atomic values. Let be a unique identifier, a PROV type, a PROV subtype, and , a set of key-value pairs. Let further and be sets of relationships , where each relationship is defined as , with a unique identifier , and . Then, a ProvCard is defined as the tuple .
The set of key-value pairs consists of mandatory, conditionally mandatory, and optional provenance metadata. Mandatory attributes are id, location, name, and type. Conditionally mandatory attributes depend on the type of ProvCard. For entities, for example, fileFormat and fileSize are also mandatory. In contrast, attributes such as runID or comment are optional. Attributes describing directly connected ProvCards are likewise always mandatory. The formal definition separates the local description of a provenance artifact from the relationships connecting it to other artifacts. In particular, and allow a ProvCard to refer to provenance information that is represented by independently created ProvCards. The resulting model therefore does not require all provenance information to be captured within a single workflow execution or provenance record.
In addition to general provenance metadata, domain-specific characteristics or excerpts from the data may be represented as quality information. Such information extends the ProvCard without changing its underlying PROV semantics.
Definition 4.2 (Quality ProvCard). Let be a ProvCard. Let denote a set of quality-aspect identifiers, a set of quality attribute keys, and denote a set of atomic quality values. Let the set of structured quality values be defined as . Let , denote a finite set of quality aspects associated with the ProvCard. Then, a quality ProvCard is defined as the tuple .
Figure 2 illustrates how independently generated provenance artifacts can describe the components of a workflow. The workflow consists of two activities (missing_values.py, outlier.py), three datasets (data_raw.csv, data_intermediate.csv, data_final.csv), and one metadata set (config.txt). Each component can be represented by a separate ProvCard, while explicit relationships connect the cards to form the provenance structure. An excerpt of the ProvCards is shown in Table 1. Note that is a quality ProvCard.
Figure 2. Overview of the data preprocessing workflow used in the case study.
| ProvCards | |
| ProvDeck | |
| ProvGraph | |
Table 1. Structure and attributes of the different ProvCard types.
ProvDecks. Independently generated ProvCards can describe individual components of a workflow, but provenance exchange and analysis often require a defined scope containing multiple artifacts. To support this use case, related ProvCards can be combined into collections called ProvDecks. A ProvDeck represents a logical provenance scope for a set of artifacts associated with a common context, such as a workflow execution, experiment, dataset release, or provenance exchange scenario. It provides a composition and exchange @page 4/11 starts_mid=1 ends_mid=1 while preserving the identity and semantics of the individual artifacts.
Immutability ensures that a provenance artifact represents a specific observation of a workflow component at a particular point in time. Changes to the underlying workflow, data, or metadata therefore result in new provenance artifacts rather than modifying existing ones.
Lightweightness allows provenance to be captured incrementally by individual workflow components without requiring a complete workflow representation or centralized provenance infrastructure.
Provenance Artifacts
The ProvCard Model introduces a layered, artifact-oriented representation for provenance management across heterogeneous scientific workflows. It preserves the provenance semantics of PROV-DM [10], while introducing an explicit artifact-level abstraction for provenance capture, composition, exchange, and analysis. The model distinguishes three complementary representations: ProvCards, ProvDecks, and ProvGraphs. ProvCards represent independently manageable provenance artifacts, ProvDecks provide a composition and exchange scope for collections of artifacts, and ProvGraphs provide a derived representation for provenance analysis.
ProvCards. A ProvCard represents the fundamental provenance artifact of the proposed model. Each ProvCard describes a single provenance-relevant element corresponding to an entity, activity, or agent according to PROV-DM. ProvCards are designed as independent and self-contained artifacts. For example, a measurement instrument may generate a ProvCard describing a raw dataset, while a processing tool creates another ProvCard describing a transformation activity. The relationship between both artifacts is established through explicit provenance references rather than through a shared workflow environment.
Let denote the set of PROV types: . Let denote the set of Prov subtypes, i.e., , where the first three labels define entities, and the last two labels define agents. code is a specialized activity. The available subtypes depend on the underlying ontology. Let further denote the set of supported provenance relation types based on PROV-DM: .
Definition 4.1 (ProvCard). Let denote the set of PROV types and the set of PROV relation types. Let denote a set of keys and a set of atomic values. Let be a unique identifier, a PROV type, a PROV subtype, and , a set of key-value pairs. Let further and be sets of relationships , where each relationship is defined as , with a unique identifier, , and . Then, a ProvCard is defined as the tuple .
The set of key-value pairs consists of mandatory, conditionally mandatory, and optional provenance metadata. Mandatory attributes are id, location, name, and type. Conditionally mandatory attributes depend on the type of ProvCard. For entities, for example, fileFormat and fileSize are also mandatory. In contrast, attributes such as runID or comment are optional. Attributes describing directly connected ProvCards are likewise always mandatory. The formal definition separates the local description of a provenance artifact from the relationships connecting it to other artifacts. In particular, and allow a ProvCard to refer to provenance information that is represented by independently created ProvCards. The resulting model therefore does not require all provenance information to be captured within a single workflow execution or provenance record.
In addition to general provenance metadata, domain-specific characteristics or excerpts from the data may be represented as quality information. Such information extends the ProvCard without changing its underlying PROV semantics.
Definition 4.2 (Quality ProvCard). Let be a ProvCard. Let denote a set of quality-aspect identifiers, a set of quality attribute keys, and denote a set of atomic quality values. Let the set of structured quality values be defined as . Let , denote a finite set of quality aspects associated with the ProvCard. Then, a quality ProvCard is defined as the tuple .
Figure 2 (PDF p. 4). Overview of the data preprocessing workflow used in the case study.
Figure 2 illustrates how independently generated provenance artifacts can describe the components of a workflow. The workflow consists of two activities (missing_values.py, outlier.py), three datasets (data_raw.csv, data_intermediate.csv, data_final.csv), and one metadata set (config.txt). Each component can be represented by a separate ProvCard, while explicit relationships connect the cards to form the provenance structure. An excerpt of the ProvCards is shown in Table 1. Note that is a quality ProvCard.
ProvDecks. Independently generated ProvCards can describe individual components of a workflow, but provenance exchange and analysis often require a defined scope containing multiple artifacts. To support this use case, related ProvCards can be combined into collections called ProvDecks. A ProvDeck represents a logical provenance scope for a set of artifacts associated with a common context, such as a workflow execution, experiment, dataset release, or provenance exchange scenario. It provides a composition and exchange boundary without merging the individual ProvCards into a single provenance record. The cards retain their individual identities and semantics and can therefore remain independently reusable outside the ProvDeck. In this paper, composition denotes the collection of mutually compatible ProvCards into a common ProvDeck while preserving their identities and explicit provenance relationships. The model assumes that cards referring to the same provenance element use compatible identifiers and do not provide conflicting descriptions. Entity reconciliation and conflict resolution across independently administered identifier spaces are outside the scope of this work.
Definition 4.3 (ProvDeck). Let be a unique identifier and a finite set of ProvCards. Then, a ProvDeck is defined as the tuple .
The separation between ProvCards and ProvDecks is central to the artifact-oriented design. ProvCards can be generated and exchanged independently, whereas ProvDecks define which artifacts are considered together for a particular provenance context or exchange operation. A ProvDeck therefore establishes provenance scope without changing the semantics or identity of the contained artifacts.
Provenance Graphs
ProvGraphs provide a derived analytical representation of the provenance artifacts contained in a ProvDeck. They enable provenance queries, visualization, dependency analysis, and graph-based exploration without requiring the original ProvCards to be merged into a single artifact. ProvGraphs follow the graph semantics of PROV-DM [10], where nodes represent entities, activities, and agents and edges represent provenance relations. Thus, a ProvGraph is not an additional source of provenance information, but a derived view that makes the relationships encoded by a collection of ProvCards explicit for analysis.
Definition 4.4 (ProvGraph). Let be the finite set of ProvCards contained in a ProvDeck . Let be the set of nodes extracted from ProvCards, where each node corresponds to exactly one ProvCard . The following holds:
with a unique identifier, a PROV type, a set of provenance attributes. Quality aspects associated with a ProvCard are not part of the ProvGraph representation.
Let be the set of provenance edges extracted from the relationships contained in ProvCards. Each edge is defined as , with and denoting source and target identifiers and . Then, a ProvGraph is defined as the directed attributed graph .
The graph representation intentionally provides a projection of the information represented by the ProvCards. It preserves the identifiers, PROV types, provenance attributes, and provenance relationships required for graph-based analysis, while quality aspects and artifact-specific information are omitted.

Figure 3. Transformation between ProvCards, ProvDecks, and ProvGraphs within the artifact-oriented provenance model.
Provenance Operations and Transformations
We define four transformations for packing, reconstructing, and projecting provenance artifacts and characterize their information-preservation properties. These transformations are not necessarily invertible: packing may omit, abstract, or anonymize information, while graph extraction may exclude artifact-specific information. These preservation considerations also apply to quality information, which extends quality ProvCards without changing their underlying provenance semantics. Quality information may therefore be preserved, omitted, or anonymized during transformation; however, unpacking must reconstruct all quality information represented in a ProvDeck.
We first establish the following notation:
denotes the set of all syntactically valid (quality) ProvCards;
denotes the set of all finite subsets of ;
denotes the set of supported PROV relation types, including wasGeneratedBy, used, wasDerivedFrom, and wasAssociatedWith;
denotes the set of all valid ProvDecks;
denotes the set of all valid ProvGraphs.
Packing and Unpacking. For a finite set of (quality) ProvCards , we define a packing operation parameterized by a privacy policy as . For a given input set , the packing operation produces a ProvDeck , where is a unique identifier and is the set of ProvCards represented in the ProvDeck. Depending on , the represented cards may correspond directly to cards in or to representations of cards in in which information has been omitted, abstracted, or anonymized. Any deck-level metadata or organizational information is represented by the corresponding ProvCards and their provenance relationships rather than by introducing additional elements into the ProvDeck tuple. By default, packing preserves all provenance information contained in . Depending on , privacy-aware packing may omit individual ProvCards or attributes, or replace attribute values by anonymized representations. Thus, information reduction is not an inherent property of packing but results from the applied policy. This separation is important for interoperability: the same provenance artifacts can be exchanged losslessly when no information-reducing policy is applied, while policy-controlled representations can be created when the exchange context imposes privacy or disclosure constraints.
Let denote the provenance and quality information explicitly represented by , where is a finite set of ProvCards, a ProvDeck, or a ProvGraph. These information includes identifiers, PROV types and subtypes, attributes, quality aspects, and provenance relationships. ProvDeck identifiers and representation-specific ordering or serialization details are excluded because they organize the representation rather than describe provenance. Information preservation between two representations and is expressed by . Thus, all information represented by is preserved in , while may contain additional information. For a packing policy that only omits ProvCards or attributes, the packing operation satisfies . If preserves all information contained in , packing is lossless and . Policies that replace values through abstraction or anonymization produce representations derived from , but the explicitly represented information items do not necessarily form a literal subset of .
Conversely, we define the unpacking operation by , where is the finite set of ProvCards reconstructed from the information represented in . Unpacking is required to reconstruct all information represented in the ProvDeck without introducing additional provenance information. Formally, this requirement is expressed as . Thus, information intentionally omitted or abstracted during packing cannot in general be recovered during unpacking. If information has been anonymized, the corresponding anonymized representation must be preserved during unpacking, without implying recoverability of the original value.
Accordingly, is a reconstruction counterpart to rather than a classical inverse. In general, , because packing may intentionally omit, abstract, or anonymize information. Since unpacking reconstructs exactly the information represented in the ProvDeck, we have . If preserves all information, the reconstruction is lossless, including the quality information represented by the packed ProvCards.
Graph Extraction. The second transformation maps a ProvDeck to a graph representation. We define the graph extraction operation . For a given ProvDeck , the extracted ProvGraph is defined as . Its node set is . Thus, each ProvCard contributes exactly one graph node whose identifier, PROV type, and provenance attributes are preserved. The edge set is constructed from the provenance relationships represented by the ProvCards. For each pair of ProvCards , an edge is included in if or , where . Thus, the graph captures the provenance relationships whose source and target cards are contained in the ProvDeck. Quality aspects are stored as dedicated nodes in the Artifact Database (cf. § 4.5) but are not included in the ProvGraph produced by . Consequently, graph extraction preserves the core provenance structure while omitting .
Graph Reconstruction. Conversely, a ProvGraph can be used to reconstruct a ProvDeck. We therefore define the graph reconstruction operation . For a given ProvGraph , where each node is defined as , we first reconstruct a set of ProvCards . For each graph node , the corresponding ProvCard is reconstructed as , where is assigned according to the applicable ProvCard specification. The incoming and outgoing relationships are reconstructed from the graph edges: and .
Since quality aspects are not represented in the ProvGraph, they cannot be recovered by the graph reconstruction operation .
A ProvCard reconstructed solely from therefore contains the core provenance information represented by its graph node and edges, but not the quality information stored in the corresponding quality ProvCard. Recovering this information requires access to the corresponding ProvCards, ProvDeck, or Artifact Database. The reconstructed ProvCards are classified according to their PROV types and assembled into a new ProvDeck , where is a newly assigned unique identifier. The newly assigned ProvDeck identifier has no effect on the reconstructed provenance information and is therefore excluded from comparisons based on Info.
Graph reconstruction is therefore not a strict inverse of graph extraction. It preserves the core provenance information represented by the input graph, such that . The reconstructed ProvDeck may additionally contain information required by the applicable ProvCard specification, such as assigned subtypes. Its newly assigned deck identifier is representation-specific and is not considered part of the reconstructed provenance information. However, quality aspects and other artifact-specific information omitted during graph extraction cannot be recovered from the ProvGraph alone. Consequently, is semantically consistent with but need not be identical to the ProvDeck from which was extracted.
Information Preservation. The transformations can therefore be characterized by the information relation. For packing and unpacking, we obtain . For graph extraction and reconstruction, we obtain and . The latter relation expresses that graph reconstruction preserves the information represented by the graph, but does not imply recovery of information that was not represented in the graph projection.
Provenance Lifecycle. Together, these transformations support a provenance lifecycle in which different components may perform capture, composition, analysis, and preservation. Workflow components generate ProvCards during or after execution without requiring a global workflow representation. Related ProvCards can subsequently be aggregated into ProvDecks and transformed into ProvGraphs for querying, visualization, and dependency analysis. Because ProvCards and ProvDecks can be stored independently of the systems that generated them, the provenance remains available for later exchange, reuse, and analysis.
Provenance Infrastructure
Operationalizing this lifecycle requires infrastructure for generating, persisting, retrieving, and composing provenance artifacts.
ProvTemplate. When the structure of a preprocessing pipeline is known in advance, a ProvTemplate can be used to define the expected provenance structure. It consists of a predefined set of ProvCards, grouped within a ProvDeck, representing the expected workflow components and their relationships. During execution, the template is instantiated by removing unused ProvCards and relationships and populating the remaining cards with concrete provenance information. This reduces the effort required to collect and organize provenance metadata during runtime.
Artifact Database. The artifact-oriented model can be complemented by an optional persistent repository for collecting, indexing, preserving, and reusing ProvCards. The database is not required for provenance capture or exchange, but supports discovery, retrieval, versioning, long-term preservation, provenance analysis, workflow comparison, and reuse across heterogeneous workflow environments.
Implementation
To operationalize the proposed model and infrastructure, our implementation comprises four main steps (cf. Figure 4): (1) ProvCard creation, (2) storage in a graph database, (3) ProvDeck creation, and (4) ProvGraph generation. These steps involve three software components: a database component for storing and retrieving ProvCards, a transformation component for converting between ProvCards, ProvDecks, and W3C PROV-JSON, and a visualization component for generating ProvGraphs.

Figure 4. Software architecture with application-specific input (blue), provenance system (green) and provenance questions answering (red).
ProvCard Creation. Provenance extraction and ProvCard creation constitute the application-specific part of the implementation. We distinguish between two alternative integration strategies: In the first strategy, the executing component collects the relevant information, including input and output artifacts, responsible agents, execution timestamps, and optional quality information. This information is immediately transformed into ProvCards and stored in the Artifact Database. Consequently, provenance capture is interleaved with data processing, and the corresponding provenance representation is available as soon as the workflow step has completed.
In the second strategy, provenance information is reconstructed retrospectively from execution logs. During workflow execution, the participating components write provenance-relevant events, such as accessed inputs, generated outputs, timestamps, parameters, and responsible processes, to their respective log files. After the workflow has completed, workflow-specific extractors parse these heterogeneous log files, map the extracted information to the ProvCard model, and import the resulting ProvCards into the Artifact Database. The two strategies differ in the time and location of provenance extraction but produce the same standardized representation. Direct capture requires explicit instrumentation of the executing workflow but avoids a separate ingestion phase and makes provenance available immediately. Retrospective extraction introduces less provenance-specific logic into the workflow itself,
Listing 1: JSON serialization of a quality ProvCard for an adaptive missing-value imputation activity.
{
"id": "prov::activity::missing_value_imputation_01",
"type": "activity:code",
"name": "Adaptive missing-value imputation",
"location": "workflow://missing_value_imputation_01",
"quality": {
"imputation_gap_37": {
"primary_recommendation": "seasonal",
"applied_method": "linear",
"decision_status": "fallback"
}
},
"input": [
{
"id": "prov::data::input_series",
"type": "data",
"edge": "used"
}
]
}
but its completeness depends on the information recorded in the available logs and requires workflow-specific parsing and mapping.
Provenance Model. Once created, the ProvCards provide a standardized representation of the extracted provenance information based on our provenance model. All subsequent components are implemented independently of the underlying use case and workflow system. This separation establishes a clear boundary between application-specific provenance extraction and application-independent provenance processing. Consequently, the same processing pipeline can be applied to different workflow systems without modifying the use-case-independent components.
ProvDeck Creation. A ProvDeck comprises a set of related ProvCards and can be serialized as a JSON file. Each ProvCard is represented by a JSON object, as illustrated in Listing 1. For quality ProvCards, is represented by the quality object. Its property names correspond to quality-aspect identifiers , while the associated property values represent the structured values . While the JSON representation provides a portable exchange format for complete ProvDecks, the individual ProvCards and their relationships are persistently stored in the Neo4j database.
Artifact Database. The artifact database stores provenance artifacts in a graph-oriented representation. This representation supports efficient traversal of provenance paths, dependency analysis, and graph queries while preserving the semantics of the underlying W3C PROV-DM model. Each ProvCard (describing an entity, activity, or agent) is mapped to a graph node. The corresponding PROV type determines the node label, while the metadata attributes in the attribute set are represented as node properties. The relationships specified in and are materialized as directed graph edges representing PROV relations such as used, wasGeneratedBy, wasDerivedFrom, and wasAssociatedWith.
Quality Information. Quality information represented by quality ProvCards is stored separately from the corresponding provenance nodes. While is conceptually associated with the ProvGraph node representing the ProvCard, the physical Neo4j representation stores each structured quality aspect in a dedicated node to facilitate querying and aggregation. Each quality aspect is connected to its associated provenance node through a has_quality relationship. The identifier identifies the quality aspect, while the key-value pairs contained in are represented as properties of the quality node.
ProvGraph Generation. The final implementation step generates a graphical representation of the provenance information. For this, a W3C PROV-JSON file is generated and converted into a directed graph using the prov library. To improve readability, entity labels containing file paths are reduced to their corresponding file names before rendering.
Case Study
To evaluate the proposed provenance framework, we consider a data-processing workflow that performs two consecutive preprocessing steps on a weather dataset. The workflow (cf. Figure 2) is deliberately kept simple to illustrate how provenance information is captured, stored, and queried while still covering different types of provenance questions.
Dataset. The workflow processes daily wind speed, sunshine duration, air pressure, and temperature. Missing values, encoded as −999, are imputed by missing_values.py, after which outlier.py identifies implausible measurements. The resulting raw, intermediate, and final datasets form the entities of the provenance path shown in Figure 2.
Provenance Capture. The implementation evaluated in this paper applies the direct-capture strategy described in Section 5. The preprocessing workflow and its provenance instrumentation are implemented in a single Jupyter notebook. After each processing step, a dedicated cell creates the corresponding entity, activity, and agent ProvCards, records their PROV relationships, timestamps, and optional quality information, and stores them in the Neo4j-based Artifact Database. ProvCards are thus generated incrementally during workflow execution. Retrospective extraction from log files is supported as an alternative strategy but is not implemented or evaluated in this case study.
ProvDeck Creation. The ProvCards generated throughout the preprocessing workflow are collected into a complete ProvDeck, preserving their individual identities and relationships. The size of a ProvDeck depends on the number and detail of its ProvCards. In this case study, the serialized complete ProvDeck has a size of 6 kB.
Provenance Querying. Provenance questions are translated into Cypher queries and executed against the Artifact Database. The graph-oriented representation allows the queries to traverse provenance relations between entities, activities, and agents. Depending on the query, the result can consist of individual provenance attributes, a set of provenance elements, or a connected subgraph of the stored provenance graph.
The resulting subgraph is reconstructed as a ProvDeck, preserving the individual provenance artifacts and their relationships, and represented as a ProvGraph for graph-based analysis and visualization. For queries that only require a textual result, the query result can alternatively be returned as a string. This allows the same provenance representation to support both machine-readable provenance artifacts and human-readable query results. As an example of a graph-based provenance query, we consider the question : How was the dataset processed before outlier detection? Answering this question requires following the provenance relations from the outlier detection activity to its input artifacts and further back to the preceding processing activity and its inputs.
The corresponding Cypher query retrieves the provenance elements and relations required to reconstruct the processing history preceding the outlier-detection activity. The resulting query-specific ProvGraph is shown in Figure 5b. It shows that weather_data_raw.csv is used by missing_values.py, which generates weather_data_intermediate.csv. This intermediate dataset subsequently serves as the input to outlier.py. Because the question concerns processing before outlier detection, the output of that activity, weather_data_final.csv, is not included in the query result. In contrast, Figure 5a shows the complete Artifact Database representation, including quality information and the final dataset weather_data_final.csv.

Figure 5. Graph-based answering of the provenance question : How was the dataset processed before outlier detection? (a) Complete graph representation of all ProvCards stored in the Artifact Database for the case-study workflow, including quality information. (b) Query-specific ProvGraph containing only the provenance elements required to answer the question.
Validation of the Provenance Model
We validate three aspects of the proposed model: requirement coverage, the information-preservation properties defined in § 4, and the reconstruction and querying of independently captured provenance. Performance, scalability, capture overhead, and cross-system portability are outside the scope of this evaluation.
Requirement Coverage. Table 2 summarizes how the implementation and case study address requirements R1–R7.
| Req. | Addressed by | Evidence |
| R1 | ProvCards | Structured representations capture PROV elements, relationships, and optional quality information. |
| R2 | ProvCards | Direct capture was integrated into the existing notebook workflow without replacing its infrastructure. |
| R3 | ProvCards | Workflow steps generate ProvCards independently of a global provenance representation. |
| R4 | ProvCards, ProvDecks | Datasets, metadata, activities, and agents are represented through a common abstraction. |
| R5 | ProvCards, ProvDecks | ProvCards are uniquely identified, stored in Neo4j, retrieved, and collected into ProvDecks. |
| R6 | ProvDecks | Independently serialized, mutually compatible cards are collected into a common ProvDeck while preserving their identities and relationships. |
| R7 | ProvGraphs | ProvDecks are transformed into ProvGraphs that reconstruct the expected processing path and support graph-based queries. |
Table 2. Requirement coverage and evaluation evidence.
Transformation Validation. We validated the implemented transformations using the ProvCards generated for the weather-data preprocessing workflow. First, we packed the cards for the raw, intermediate, and final datasets, the missing-value imputation and outlier-detection activities, and the associated agents into a ProvDeck using the identity policy. After unpacking the deck, we compared the reconstructed cards with the original cards in terms of identifiers, PROV types and subtypes, attributes, relationships, quality-aspect identifiers, and quality values. After ordering semantically unordered collections by their identifiers, keys, and relation types, no differences were observed.
We then transformed the ProvDeck into a ProvGraph and reconstructed a ProvDeck from that graph. The graph preserved the provenance path from weather_data_raw.csv through missing-value imputation and weather_data_intermediate.csv to outlier detection and weather_data_final.csv, together with its graph-relevant attributes and relationships. Comparing these elements with the reconstructed ProvDeck revealed no differences. Quality aspects and other information excluded by the graph projection, as well as the newly assigned ProvDeck identifier, were excluded from this comparison. The results are consistent with the information-preservation properties defined in § 4.4; they do not establish those properties for arbitrary implementations or information-reducing policies.
Independent Capture and Provenance Questions. The ProvCards generated at different stages of workflow execution were successfully connected through compatible identifiers and collected into a ProvDeck without maintaining a shared in-memory ProvGraph during preprocessing. The complete ProvGraph reconstructed from these cards contained the expected processing path from weather_data_raw.csv through missing-value imputation and weather_data_intermediate.csv to outlier detection and weather_data_final.csv. The query-specific ProvGraph in Figure 5b contained only the prefix required to answer and therefore excluded the final dataset.
Following the provenance-question classification of [12], we evaluated 14 questions covering how, where, who, what, which, when, and why. For example, one question asks how the dataset was processed before outlier detection (Figure 5b). The complete set of questions and queries is available in the Git repository (see below). Eleven questions were answered entirely from the recorded provenance attributes, quality information, and relationships; three required supplementary context from the notebook, filesystem, or execution environment. All 14 answers were consistent with the known workflow and its recorded execution context. These results show that independently generated ProvCards support workflow reconstruction and provenance querying, while the range of answerable questions depends on the contextual information captured.
Discussion. The validation is limited to direct provenance capture in a small Jupyter-based workflow and a single execution environment. Although the architecture supports retrospective generation of ProvCards from logs, its completeness depends on the available log information and workflow-specific extraction and mapping. Evaluating this strategy, as well as capture overhead, scalability, and cross-system portability, remains future work.
Conclusion and Future Work
This paper introduced ProvCards, an artifact-oriented approach for capturing and composing provenance across heterogeneous scientific workflows. ProvCards represent independently manageable provenance artifacts grounded in PROV-DM, ProvDecks provide scopes for their composition and exchange, and ProvGraphs provide derived representations for querying and analysis. The formalized transformations make explicit which provenance and quality information is preserved or omitted, including information reduction through privacy-aware packing and graph projection.
Our implementation and case study demonstrate that provenance can be composed through compatible identifiers, and queried as a connected provenance graph. The results provide initial evidence for the feasibility of the approach, but are limited to a small workflow and direct capture within a single environment.
Future work will evaluate capture overhead, storage requirements, query performance, and interoperability in larger distributed workflows and across independently administered systems. We also plan to investigate reusable extraction adapters, privacy-aware provenance exchange, controlled disclosure, and extensions of the Artifact Database for discovery, versioning, workflow comparison, and long-term preservation.
Code and Data Availability. The source code, executable Jupyter notebook, Cypher queries, generated ProvDecks, and ProvGraphs used in this evaluation are available at https://gitlab.com/TanjaAuge/provcardmodel. The underlying weather data originate from the German Weather Service and are available at https://opendata.dwd.de.
References
References
- [1]Susan B. Davidson, Sanjeev Khanna, Sudeepa Roy, and Sarah Cohen-Boulakia. 2010. Privacy Issues in Scientific Workflow Provenance. In Proceedings of the 13th International Conference on Extending Database Technology. 225–236. doi:10.1145/1739041.1739069DOI
- [2]Alban Gaignard, Hala Skaf-Molli, and Khalid Belhajjame. 2020. Findable and reusable workflow data products: A genomic workflow case study. Semantic Web 11, 5 (2020), 751–763. doi:10.3233/SW-200374
- [3]Timnit Gebru et al. 2021. Datasheets for datasets. Commun. ACM 64, 12 (2021), 86–92. doi:10.1145/3458723DOI
- [4]Carole Goble, Sarah Cohen-Boulakia, Stian Soiland-Reyes, Daniel Garijo, Yolanda Gil, Michael R. Crusoe, Kristian Peters, and Daniel Schober. 2020. FAIR Computational Workflows. Data Intelligence 2, 1-2 (2020), 108–121. doi:10.1162/dint_a_00033DOI
- [5]Timothy Lebo, Satya Sahoo, Deborah McGuinness, Khalid Belhajjame, James Cheney, David Corsar, Daniel Garijo, Stian Soiland-Reyes, Stephan Zednik, and Jun Zhao. 2013. PROV-O: The PROV Ontology. http://www.w3.org/TR/2013/REC-prov-o-20130430/
- [6]Shayne Longpre et al. 2023. The Data Provenance Initiative: A Large Scale Audit of Dataset Licensing & Attribution in AI. arXiv:2310.16787 [cs.CL]. https://arxiv.org/abs/2310.16787
- [7]Danius Michels et al. 2014. PROV-TEMPLATE: A Template System for PROV Documents. https://openprovenance.org/prov-template-2014-06-07/
- [8]Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Wasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, and Timnit Gebru. 2019. Model Cards for Model Reporting. In Proceedings of the Conference on Fairness, Accountability, and Transparency (Atlanta, GA, USA) (FAT* ’19). Association for Computing Machinery, New York, NY, USA, 220–229. doi:10.1145/3287596.DOI
- [9]Luc Moreau, Belfrit Victor Batlajery, Trung Dong Huynh, Danius Michaelides, and Heather Packer. 2018. A Testing System to Generate Provenance . IEEE Transactions on Software Engineering 44, 02 (Feb. 2018), 103–121. doi:10.1109/TSE.2017.2659745DOI
- [10]Luc Moreau and et al. 2013. PROV-DM: The PROV Data Model. https://www.w3.org/TR/prov-dm/.
- [11]Mahima Pushkarna et al. 2022. Data Cards: Purposeful and Transparent Dataset Documentation for Responsible AI. In FAccT ’22. Association for Computing Machinery, 1776–1826. doi:10.1145/3531146.3533231DOI
- [12]Laura Waldersdorfer, Dominique Hausler, and Tanja Auge. 2025. Provenance Question-based AI Transparency and Accountable AI Governance. In AAAI 2025 Workshop on AI Governance: Alignment, Morality, and Law. https://openreview.net/forum?id=EFsyy6DqCM
