Research Data and Metadata at Risk: Degradation over Time

Research data and metadata - usually derived from an experiment or series of experiments - is in the hot focus of the researcher during the experiment and the subsequent interpretation, paper writing and publishing. But once the researcher has moved on to their next effort, this data is very much at risk. Often (read 'the-rule-rather-than-the-exception'), the data is not properly managed and archived, and resides as a single copy on the researcher's desktop or maybe research server.

In addition, the metadata is minimal or non-existent, and if it does exist is only interpretable by the researcher and their colleagues or students. Over time, the chance that this data will be lost or useful knowledge about it forgotten by the researcher increases, and the information content of the data and metadata rapidly decreases. Some events can seriously accelerate this decrease: data loss (media failure, computer replacement, other serious accidents or failures, etc); change of careers and retirement; and the death of the researcher.

This common scenario was first described in published form in 1997 (to my knowledge) in a paper entitled: Nongeospatial Metadata for the Ecological Sciences (citation below). It included a very expressive diagram, which I have re-created for another paper (which was used in the draft of the paper but it was decided not to use the diagram in the final version), and you can see it below:





A higher resolution jpeg image can be found here.
A high resolution PDF can be found here.

The diagram was created using LaTeX and TikZ. The source files can be found at the github repo.


From (only in draft): 
de la Sablonnière, Auger, Sabourin and Newton. 2012. Facilitating Data Sharing in the Behavioral Sciences. Data Science Journal. Volume 11, 23 March 2012. DOI 10.2481/dsj.11-DS4
   
After: 
Michener, W., J. Brunt, J. Helly, T. Kirchner & S. Stafford. 1997. Nongeospatial Metadata for the Ecological Sciences. Ecological Applications 7:1:330-342 DOI: 10.1890/1051-0761(1997)007[0330:NMFTES]2.0.CO;2

 

The Michener paper is excellent and very much before its time. Its many insights generalize to other research domains.

Comments

Post a Comment

Popular posts from this blog

Java, MySql increased performance with Huge Pages

[Resources updates: 2010.07.07 , 2009.11.03 , 2009.05.27 ] Long running, large memory, high performant applications often have special needs with respect to their memory management. On Linux, Solaris and other modern OSes, the translation look-aside buffer ( TLB ) - whose page size of 4k for many CPUs/OSes - becomes a scalability issue in these extreme conditions. In order to get around TLB scalability issues, huge page sizes are used to reduce the impact on performance. This can be of use to installations with large scale Java, MySql and other large memory applications. 300% improvement : " Well, in my case, I was able to achieve an over 3x improvement in my EJB 3 application, of which fully 60 to 70% of that was due to using large page memory with a 3.5GB heap. Now, a 3.5GB heap without the large memory pages didn't provide any benefit over smaller heaps without large pages. Besides the throughput improvements, I also noticed that GC frequency was cut down by two-third
Read more

IBM on Linux: "Lean, clean, and green"

IBM developerWorks has an article ( Linux: Lean, clean, and green: How GNU/Linux is becoming more eco-friendly - 26 May 2009) which examines some of the Green benefits of the Linux operating system. It focuses primarily on the low resource demands Linux has on systems (as well as its support for older systems), thus extending the life of machines that would otherwise be junked. Also discussed is virtualization and aspects of the Linux OS that reduce power consumption in servers. Additional Green Linux and Open Source resource: LessWatts.org - Saving Power on Intel systems with Linux Ten ways Linux can turn you green . 2009 Green Computing With Open Source Software . 2009 Open Source is Already Naturally Green: Fewer Lawyers, Fewer Showers, More Real People Add Up to Big Green Wins . 2009 Is Linux the Greenest Operating System . 2009 Go Green, Save Green with Linux . 2008 Linux captures the 'green' flag, beats Windows 2008 power-saving measures 2008 Canadian Green Party
Read more

Mars Inc. Cacao Genome Database claims Open Access, public domain: falls short

This initially looked very promising: Mars , along with a number of collaborators (USDA, IBM, Clemson University Genomics Institute; Public Intellectual Property Resource for Agriculture at the University of California-Davis; National Center for Genome Resources; Center for Genomics and Bioinformatics at Indiana University; HudsonAlpha Institute for Biotechnology; and Washington State University), have sequenced the cacao genome and released it "Open Access" and "public domain" for the benefit of all, at a site called the Cacao Genome Project : McLean, VA –Today, Mars, Incorporated, the U.S. Department of Agriculture-Agricultural Research Service (USDA-ARS), and IBM released the preliminary findings of their breakthrough cacao genome sequence and made it available in the public domain . - From the Mars Inc. press release 15 September 2010 A quote from the Independent article on the release ( First rice, then wheat – now cocoa genome unravelled 15 Sep 2010) f
Read more