ABSTRACT
We describe a framework for managing live research data involving two major components. First, a system for the scalable scheduling and execution of automated policies for moving, organizing, and archiving data. Second, a system for managing metadata to facilitate curation and discovery with minimal change to existing workflows. Our approach is guided by four main principles: 1) to be non-invasive and to allow for easy integration into existing workflows and computing environments; 2) to be built on established, cloud-aware, open-source tools; 3) to be easily extensible and configurable, and thus, adaptable to different academic disciplines; and 4) to integrate with and take advantage of infrastructure and services available on academic campuses and research computing environments. These principles give our solution a well-defined place along the spectrum of research data management software such as sophisticated electronic lab notebooks and science gate-ways. Our lightweight and flexible data management framework provides for curation and preservation of research data within a lab, department or university cyberinfrastructure.
- Declan Butler. 2005. A new leaf. Nature 436 (06 07 2005), 20--21.Google Scholar
- I. Foster. 2011. Globus Online: Accelerating and Democratizing Science through Cloud-Based Services. IEEE Internet Computing 15, 3 (May 2011), 70--73. Google ScholarDigital Library
- Apache Foundation. 2019. Apache Airflow. https://airflow.apache.org. (2019). {Online; accessed 20-February-2019}.Google Scholar
- Arcot Rajasekar, Mike Wan, Reagan Moore, and Wayne Schroeder. 2006. A prototype rule-based distributed data management system. (2006).Google Scholar
- R. W. Watson. 2005. High performance storage system scalability: architecture, implementation and experience. In 22nd IEEE / 13th NASA Goddard Conference on Mass Storage Systems and Technologies (MSST'05). IEEE Computer Society, Washington, DC, USA, 145--159. Google ScholarDigital Library
- Nancy Wilkins-Diehr. 2007. Special Issue: Science Gateways --- Common Community Interfaces to Grid Resources. Concurrency and Computation: Practice and Experience 19, 6 (2007), 743--749. arXiv:https://onlinelibrary.wiley.com/doi/pdf/10.1002/cpe.1098 Google ScholarDigital Library
Index Terms
- A Lightweight Framework for Research Data Management
Recommendations
Data management and curation practices: the case of using DSpace and implications
ASIST '15: Proceedings of the 78th ASIS&T Annual Meeting: Information Science with Impact: Research in and for the CommunityData management and curation is a new challenge with the emerging trend of data-dependent scholarly research. Due to the lack of common standards and best practices, current data management and curation practices have been varied. This poster presents a ...
Data Management Workflows in Interdisciplinary Highly Collaborative Research
PEARC '22: Practice and Experience in Advanced Research ComputingData curation is an important aspect in research projects. Effective data management is critical for data curation, and it not only contributes to the success of projects but makes research outputs findable, accessible, interoperable and reusable. We ...
Applying iRODS to the Brain Image Library
PEARC '18: Proceedings of the Practice and Experience on Advanced Research ComputingRecent advancements in optical microscopy and specimen preparation have greatly improved the specificity and sensitivity of observation. Imaging instruments have become readily available and produce data at GB/s rates. These capabilities now allow ...
Comments