Environmental Data Management at NOAA: Archiving, Stewardship and Access

A recent publication available via National Academies Press – Environmental Data Management at NOAA: Archiving, Stewardship and Access is, IMHO, a must-read for anybody gathering any type of data (not just at NOAA as the title implies). For those strapped for time, a quick 4-page PDF of the report brief is also available for download.

We need to be thinking about the principles identified in the report and how they apply to our individual research endeavors going forward. Our data can potentially become quite useful to others researchers and research projects. Those projects might be in a completely unrelated discipline and/or research area for which the data were initially gathered, so a little forethought is required when defining the metadata.

The principles laid out in the report are well thought-out and pertain to pretty much any data-gathering endeavor.

The principles from the book are:

"Environmental data should be archived and made accessible"
"Data-generating activities should include adequate resources to support end-to-end data management"
"Environmental data management activities should recognize user needs"
"Effective interagency and international partnerships are essential"
"Metadata are essential for data management"
"Data and metadata require expert stewardship"
"A formal, ongoing process, with broad community input, is needed to decide what data to archive and what data not to archive"
"An effective data archive should provide for discovery, access, and integration"
"Effective data management requires a formal, ongoing planning process"

Much more detailed explanations and examples are provided in the report itself, so it’s definitely worth a read.

Implementation Issues

This topic is near and dear to the research that I am interested in. Keep in mind that I’m abstracting the principles identified for NOAA to the work of individual researchers and/or their institutions. Issues that come to mind are:

How can we leverage existing and emerging technologies to help make the data available and more accessible for use by others. These uses can include data mash-ups that use the data as one or more components via web services, GIS technologies and trusty old data file downloads among other undefined or not-yet-invented uses.
How can we ensure that the data holdings are discoverable and available in a standards compliant format.
What tools and techniques can we develop that will make the job of data archiving and accessibility easier to accomplish? These tools should be open source and freely available to all.
How can we do the above across agency, institution and country boundaries? If I know where my data are and how to get access to it, I’m certainly not going to pay to make it easier for "those other guys" to get access to it.
Where will the research funding necessary to move forward with these principles come from?

To their credit, most researchers would like to include provisions to support the principles identified in the report. There are several obstacles to following through with these good intentions, however. For example, when the pressure comes to trim the funding requested for a research proposal (let’s face it, funding is tight) – data management, dissemination and accessibility is often the first thing to get whacked. It’s simply not necessary to accomplish the goals of most research projects. In addition, most institutions probably don’t have a "Research Data Management Department" whose mission it is to assist with "archiving, stewardship and access" as it relates to research data.

I could continue on with several more pages of ramblings about how the principle objectives can be achieved, but for now I’ll let you look them over and see how they apply to your research endeavors.

Enjoy!

Doug White