May 2024
ProvTracer
Automatically captures digital artifact provenance using a multimodal LLM and knowledge graph serialization.
Overview
Desktop application for capturing, modeling, and querying AI/ML output provenance using PROV-O and SPARQL; automated capture via multimodal LLM. Built for dissertation research on reproducibility auditing and lineage tracing in defense AI pipelines.
The question
How can a software work session leave a useful, queryable record of how its artifacts were produced? ProvTracer brings together software traceability and Semantic Web provenance.
From work session to knowledge graph
The desktop application captures session context and uses a multimodal language model to help describe the work. It serializes provenance using an ontology based on PROV-O and BFO, linking small time-slice graphs into a larger session record.
The resulting graphs support SPARQL queries, including comparisons between task timestamps. They can also be supplied to a language model for qualitative questions about a session.
Explore the implementation
The repository includes the Python/PyQt6 application, its ontology, example case studies, and a walkthrough of the recording and analysis workflow.