Gehani, A., Kim, M., & Malik, T. (2010, June). Efficient querying of distributed provenance stores. In Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing (pp. 613-621).
Abstract
Current projects that automate the collection of provenance information use a centralized architecture for managing the resulting metadata – that is, provenance is gathered at remote hosts and submitted to a central provenance management service. In contrast, we are developing a completely decentralized system with each computer maintaining the authoritative repository of the provenance gathered on it. Our model has several advantages, such as scaling to large amounts of metadata generation, providing low-latency access to provenance metadata about local data, avoiding the need for synchronization with a central service after operating while disconnected from the network, and letting users retain control over their data provenance records. We describe the SPADE project’s support for tracking data provenance in distributed environments, including how queries can be optimized with provenance sketches, pre-caching, and caching.