Dr. James J. O’Brien, Mr. Shawn R. Smith, and Dr. Mark A. Bourassa
Center for Ocean-Atmospheric Prediction Studies
The Florida State University
Tallahassee, FL 32306-2840
Program Manager: Michael Johnson, NOAA/OCO
Related NOAA Strategic Plan Goals:
Goal 2. Understand climate variability and change to enhance society’s ability to plan and respond.
Goal 3. Serve society’s needs for weather and water information.
The central activity of the U.S. Research Vessel Surface Meteorology Data Assembly Center (DAC) is the continued development of the Shipboard Automated Meteorological and Oceanographic System (SAMOS) initiative (http://samos.coaps.fsu.edu/). The SAMOS initiative focuses on improving the quality of and access to surface marine meteorological and oceanographic data collected in-situ by automated instrumentation on research and merchant vessels. Over the past year, the SAMOS DAC has (1) recruited 10 additional vessels to provide routine observations, (2) developed a research-quality data evaluation system, (3) expanded tools to display data quality, (4) improved user access to data, and (5) continued international liaison activities. In addition, the DAC continues delayed mode quality evaluation of observations from the Ronald Brown and Ka’Imimoana. The DAC activities focus primarily on NOAA Strategic Plan Goals 2 and 3 by providing high quality weather and near surface ocean data for use in validating satellite products, global air-sea flux analyses, and model fields.
The DAC was established at the Florida State University specifically to coordinate the collection, quality evaluation, distribution, and future archival of SAMOS data. A SAMOS is typically some form of a computerized data logging system that continuously records navigation (ship’s position, course, speed, and heading), meteorological (winds, air temperature, pressure, moisture, rainfall, and radiation), and near ocean surface (sea temperature and salinity) parameters while the vessel is at sea. Measurements are recorded at high-temporal sampling rates (typically 1 minute or less). The DAC collaborated with the Woods Hole Oceanographic Institution (WHOI) to design a ship-to-shore-to-user data pathway for U.S. research vessel SAMOS data. In the past, the data flowed from ship to shore only in a delayed-mode with a 3 month to 2-year lag between collection and availability to the user community. The new vision supports automated data transmission from each ship to the DAC on a daily basis. A “preliminary” version of the SAMOS data is available soon after receipt by the DAC. The preliminary data have undergone common formatting, metadata enhancement, and automated quality control. Visual inspection and further scientific quality control are now operational and result in a “research” quality SAMOS product. Plans are being developed to further enhance distribution services for the quality-controlled data in formats that meet user needs. Data exchange agreements are in place with two national data archive centers to ensure that the original and quality controlled data are maintained for future generations of scientists.
Over the past year our efforts have focused on the recruitment of additional vessels to the SAMOS initiative and the continued development of the data quality procedures. Recruitment resulted in an additional 10 vessels reporting for the period covered by this report. The SAMOS data quality evaluation system is now operational for both preliminary and research SAMOS data products. Many upgrades have been made to both our public access web site and our internal data base tools. Throughout the year, DAC personnel have been actively promoting the SAMOS Initiative through meetings and working groups, including the International Marine Technicians Symposium (INMARTECH) at WHOI in October 2006 and the 4th WMO/IOC JCOMM Ship Observation Team (SOT) meeting in Geneva, Switzerland in April 2007. Finally, we continue our delayed mode data processing for the NOAA vessels Ronald Brown and Ka’Imimoana.
1. 2. Vessel recruitment
Recruitment of additional vessels to participate in the SAMOS Initiative was very successful during the reporting period. Ten new vessels were recruited (Table 1) in the past year. These vessels now routinely contribute SAMOS observations when they are at sea. Collaboration with NOAA’s Office of Marine and Aviation Operations (OMAO) resulted in eight new recruitments. OMAO developed a SAMOS data transmission applet as part of the version 4.0 release of their scientific computing system (SCS). As the new SCS was installed in the fleet, the NOAA vessels began transmitting data to the DAC. In addition, Co-PI Smith attended the annual UNOLS RVTEC meeting and INMARTECH in October 2006 and had good conversations with several additional vessel operators. Most expressed interest in participating in SAMOS, but initiating new data transfers is still difficult in these times of tight operational budgets for research vessels.
Table 1: Ships transmitting observations to SAMOS DAC during the period
July 1, 2006 - June 30, 2007.
Vessel Call Sign Operator Number of ship days of data
Atlantis KAQP WHOI 290
Gordon Gunter WTEO NOAA 2
Healy NEPP USCG 6
Henry Bigelow WTDF NOAA 21
Hi’Ialakai WTEY NOAA 45
Ka’Imimoana WTEU NOAA 7
Knorr KCEJ WHOI 229
Lawrence Gould WCX7445 NSF/Raytheon 32
Miller Freeman WTDM NOAA 122
Nancy Foster WTER NOAA 74
Oscar Dyson WTEP NOAA 93
Ronald Brown WTEC NOAA 71
3. Daily SAMOS data processing
Preliminary processing of SAMOS observations received via daily email messages from participating research vessels is now an operational activity at the DAC. During the reporting period, 992 days of shipboard meteorology data were processed for the 12 recruited vessels (see Table 1). Preliminary processing (Figure 1) starts once the data file arrives at the DAC as an attachment to an email. Each email attachment is unpacked, the data provided are verified that they conform to the format and parameters expected for the individual vessel, and finally the data are blended with vessel specific metadata and are converted to a common netCDF format. The data for each day are then passed through an automated quality evaluation program and data quality statistics are calculated prior to the file being posted for users on the SAMOS web and ftp sites. The entire process from arrival at the DAC to distribution of the preliminary data files is fully automated. Preliminary files appear on the data distribution site within 5 minutes of their arrival at the DAC (typically shortly after 0000 UTC).
A sample of the spatial distribution of data received, processed, and on-line for 1 January – 21 May 2007 is shown in Figure 2. Prior to 2007, the only two ships providing data were the Knorr and Atlantis. A rapid increase in vessels participating in SAMOS began in early 2007, leading to the present 12 recruited vessels (Table 1).
Each individual data file has been augmented with extensive metadata that is stored in a ship profile database. In addition, the shipboard database uses a strict version control to track individual data files received from their original email attachment to the final files released to the public (Figure 1). Individual data quality statistics are stored in the ship database and these can be accessed through the data availability link on the SAMOS web site (http://samos.coaps.fsu.edu/html/data_availability.php). A sample of data quality graphics for the Knorr is provided in Figure 3.
4. Delayed-mode SAMOS processing
The preliminary processing of SAMOS observations is fully operational (see 2 above). Due to data logging problems on the ship or communication dropouts, some data arrive several days after they were collected. Often the data are noted to be missing by the analyst at the DAC and arrive after the analyst notifies the vessel technician at sea. In addition, data for a single day may be fragmented and may arrive in multiple files attached to a single email. As a result, the DAC developed a method to merge multiple files for a single observing day into a combined, delayed-mode data file. This merged file undergoes additional automated and visual data quality evaluation and is then released as a “research-quality” SAMOS data file for the particular observation day (Figure 1).
The process to merge multiple files for a single observing day and the visual data quality evaluation are now operational. The merge program is designed to eliminate duplicate records from the files being merged. Duplicates are eliminated based on a series of rules that take into account the automated quality control applied to the preliminary data files. The merge process is fully automated and the merged files are tracked within the file-tracking database. Currently the merge occurs 10 calendar days after the observation day (when the preliminary data should arrive at the DAC). Using the database, the analyst can easily reference the original file pieces that were merged to create a single data file for each observation day. Once merged, a summary of the data quality flags on the new file is produced and stored in the database.
Each day, the data quality analyst at COAPS reviews the latest merged files and conducts a visual quality evaluation. The visual analysis is accomplished using SVIDAT, a graphical user interface developed by COAPS programmers. SVIDAT allows the analyst to review, add, or modify data quality flags on the merged files. Once the analyst is satisfied with the data quality, the file is saved and posted automatically to the SAMOS ftp and web sites. This process also updates all necessary tracking information in the ship database and creates the copies of the original, preliminary, and research quality files for delivery to the national archive centers.
5. Continued delayed-mode evaluation of NOAA ship data
The DAC continues to evaluate the quality of the meteorological observations collected by the NOAA vessels Ronald Brown and Ka’Imimoana. We have received data from the Ronald Brown for the period February – December 2006. These data have been converted to our internal netCDF format and are awaiting quality control. For the Ka’Imimoana, we have received data for the period September 2004 – August 2007. The data for September 2004 – December 2005 have completed visual quality inspection and their data quality report is being written. Data from January – September 2006 have been converted to our format and are awaiting visual quality control. The remaining Ka’Imimoana data are awaiting conversion and quality evaluation. We note that in 2007 both the Ronald Brown and Ka’Imimoana were recruited to the SAMOS initiative and will no longer be providing delayed-mode data to the DAC. The outstanding delayed-mode data for the Ronald Brown and Ka’Imimoana will be evaluated and released by mid-2008. Future data for these vessels are being processed as part of the SAMOS initiative.
6. Public access to observations and metadata
A web presence for SAMOS is accessible at: http://samos.coaps.fsu.edu/. The pages provide information on the SAMOS initiative as a whole, provides links to relevant literature, and access to past SAMOS workshops. Through these pages, the DAC provides access to the preliminary quality controlled data for all 12 ships currently recruited to the SAMOS initiative. A metadata portal allows users to access ship- and parameter-specific metadata along with digital photos and schematics of participating vessels. Both the metadata portal and data access are user searchable. Criteria include searches by vessel and the observation dates. The web site also provides access to desired SAMOS parameters, accuracy requirements, and training materials.
The data distribution system is under constant development. New in the reporting period is the “Data Availability” link under “Data Access”. This new page provides the user with more options to select desired data than the older “Data Download” page. Through “Data Availability” users can select one or more ships and a time range to determine whether observations are available. The data are displayed graphically by day and each day is color-coded according to the overall quality of the days observations. The user can then select to download data of one or more ships and has the option of selecting only good quality or all available data. Drill down capacity allows the user to select a single day and ship from the graphic to display detailed data quality graphics (similar to Figure 3). A new metadata interface is being developed and plans call for the data to be accessible via a THREADS server at COAPS. These new tools will be added to the web site for users when testing is complete.
7. Liaison activities
The SAMOS DAC serves as the coordination office for the entire SAMOS initiative. In this capacity, DAC personnel facilitate U.S. and international collaborations on topics ranging from data accuracy, data acquisition and exchange, training activities, and data archival.
Foremost of these activities was participation by co-PI Smith at the International Marine Technicians Symposium (17-19 October 2006) at WHOI. The symposium provided an opportunity to introduce the activities of and plans for the SAMOS initiative to the international research vessel community. Throughout the week, Mr. Smith participated in dialogs with ship operators as part of the SAMOS vessel recruitment effort. Mr. Smith also visited one of the NOAA OMAO vessels in port at WHOI and received a demonstration of the NOAA OMAO SAMOS data applet in SCS 4.0. This meeting led directly to the recruitment of many of the NOAA vessels in 2007. The symposium was also the venue where the “A Guide to Making Climate Quality Meteorological and Flux Measurements at Sea” was first released to the technician and operator community. This guide was co-authored by Frank Bradley (CSIRO) and Chris Fairall (NOAA/ESRL) and was a collaborative effort between the WCRP Working Group on Surface Fluxes and the SAMOS initiative. Several scientists participating in SAMOS activities contributed to the guide.
Co-PI Smith also was invited to participate at the Fourth Session of the WMO/IOC JCOMM Ship Observation Team meeting in Geneva, Switzerland (16-21 April 2007). The SOT is responsible for overall coordination of the Voluntary Observing Ship (VOS) program and the meeting brought together VOS participants from many countries. Participants included several countries that are working towards full automation of their standard VOS meteorological reports. Mr. Smith presented the activities of the SAMOS initiative and had productive discussions with the VOS automation community. They were keenly interested in the SAMOS approach to data quality evaluation and long-term access to observations for future climate research.
DAC personnel attended other meetings throughout the year to promote both the stewardship of SAMOS data, but also the scientific application of these data.
Smith, S. R., 2006: A Partnership between Shipboard Oceanic and Atmospheric Data Programs. EOS, Trans Amer. Geophys. Union, 87, 463, 466.
Smith, S. R., 2007: Shipboard Automated Meteorological and Oceanographic System (SAMOS) Initiative. Report for 4rd session of the JCOMM Ship Observation Team meeting, 16-21 April 2007, Geneva, Switzerland, 2 pp.
Smith, S. R., R. Keeley, and T. Delcroix, 2006: Report of the 1st Joint GOSUD/SAMOS Workshop. UCAR Joint Office for Science Support, Boulder, CO, USA, 63 pp. [Available from COAPS, The Florida State University, Tallahassee, FL 32306-2840].
Bourassa, M. A., and S. R. Smith, 2007: Improving air-sea flux estimation with a new wave-dependant parameterization and high-quality research vessel observations. ONR Progress Review – Southeast Region, Tallahassee, FL, USA, 1-3 May 2007.
Smith, S. R., J. Rolph, and M. A. Bourassa, 2007: Progress of the Shipboard Automated Meteorological and Oceanographic System (SAMOS) Initiative Data Assembly Center. Climate Observation Division 5th Annual System Review, NOAA, Silver Spring, MD, USA, 5-7 June 2007.
Smith, S. R., 2007: The SAMOS Initiative. 4th session of the JCOMM Ship Observation Team, Geneva, Switzerland, 16-21 April 2007, CDROM.
Josey, S. A. (presented by S. R. Smith), 2006: Evaluation of Air-Sea Fluxes. 2nd CLIVAR Global Synthesis and Observation Panel meeting, La Jolla, CA, USA, 8 and 9 December 2006.
Smith, S. R., 2006: The SAMOS data assembly center. International Marine Technicians Symposium, Woods Hole, MA, USA, 17-19 October 2006.
Smith, S. R., F. Bradley, and C. Fairall, 2006: A guide to making climate quality meteorological and flux measurements at sea. International Marine Technicians Symposium, Woods Hole, MA, USA, 17-19 October 2006.
Summary of Interaction with NOAA
The SAMOS initiative is a collaborative effort that encompasses a number of U.S. and international partners. The DAC routinely interacts with a number of NOAA facilities to achieve the goals of the initiative. Foremost is our routine participation in the annual system review of the NOAA Climate Observation Division (COD; formerly Office of Climate Observation). Through COD, we collaborate with other centers focusing on mooring, drifters, and other components of the ocean observing system.
We actively collaborate with several groups at the NOAA Earth System Research Laboratory (ESRL) in Boulder. Chris Fairall’s group has been an active participant in the SAMOS initiative through both the development of the roving standard flux system (for onboard instrument comparisons) and the “A Guide to Making Climate Quality Meteorological and Flux Measurements at Sea”. Through a collaboration with Scott Woodruff at ESRL (and Steve Worley at NCAR), the DAC has developed hourly sub-sets of WOCE era research vessel observations that are now available as a supplement to the International Comprehensive Ocean Atmosphere Data Set (ICOADS). This code is currently being modified to allow data received at the SAMOS DAC to be contributed to ICOADS.
We continue to have an ongoing and productive partnership with Office of Marine and Aviation Operations (OMAO). Through recent collaborations with Doug Perry, Tom Stepka, and Dennis Shields, the DAC has started receiving routine SAMOS transmissions from eight NOAA vessels. As a result, the former delayed-mode data transfers from the Ronald Brown and Ka’Imimoana are now fully automated through the SAMOS initiative.
We also continue to interact with the National Oceanographic Data Center (NODC) and the National Climatic Data Center. Through NODC, we plan to establish a routine archive for SAMOS observations. We plan to transfer SAMOS data to NCDC in support of the SURFA project, which will evaluate surface fluxes in operational numerical weather forecasting models.
Summary of Education and Outreach Activity
The DAC does not presently participate in outreach activities. We do annually participate in the FSU Young Scholars Program (YSP). This year we hosted and directed the research activities of two high school students. The YSP students spend six weeks on the FSU campus taking classes and conducting directed research. We have been involved in the YSP program since 1998.
The SAMOS initiative has a number of educational goals. The focus of the training activities has been the production of a “A Guide to Making Climate Quality Meteorological and Flux Measurements at Sea”. The handbook is aimed at the sea-going research community and ships’ technical staff. Topics include information on preferred sensor location, calibration, in-situ comparisons, documentation, metadata, bulk flux methodology, and measurement error. Both hardcopy and digital versions of the handbook were produced by NOAA/ESRL and distributed at INMARTECH 2006.