Advanced Computing, Mathematics and Data
Research Highlights
April 2008
Code Improvements Enhance Analysis of Global Cloud Resolving Model Data
Tools to subset and visualize the petabyte data set sizes that will be produced by the Global Cloud Resolving Model. Enlarged View
Pacific Northwest National Laboratory (PNNL) researchers, in collaboration with the developers of the open-source NetCDF Operators (NCO), recently added capabilities for processing geodesic grid data and optimized performance to support efficient manipulation of data sets consisting of many files of tens to hundreds of gigabytes in size. The improved and optimized tools enhance the ability to analyze data generated by the Global Cloud Resolving Model (GCRM).
The GCRM models the entire globe on a fine enough scale (2-4 km) to accurately model cloud behavior, which is widely agreed to be a major source of uncertainty in existing models. Results from the GCRM will be used as the basis for increasing the accuracy of climate models that can be run more efficiently at coarser resolutions to simulate longer periods of time. To reach this scale the model makes use of a geodesic grid. The grid has the desirable characteristic that each cell on the globe is roughly equal in size while having the same number of neighbors.
The Community Access to Global Cloud Resolving Model Data and Analyses project SciDAC Scientific Application Partnership has enhanced the popular NCO data manipulation software to support data that is generated on the geodesic grid. Working with the NCO team, PNNL researchers identified a major performance bottleneck within the core NCO code that affected processing of large data sets. This code has been reworked, improving the performance by a factor of up to 500 times for certain operations. These capabilities are now being added to other tools within the NCO tool suite. The tools will be used to provide server-side data reduction.
Acknowledgment
Sponsor: DOE's Office of Advanced Scientific Computing Research.
Research Team: Karen Schuchardt (PNNL), Bruce Palmer (PNNL), Jeff Daily (PNNL), Todd Elsethagen (PNNL), Annette Koontz (PNNL); Charlie Zender (University if California-Irvine), Henry Butowsky (UCI).
EMSL Involvement: Some of this work was conducted in the Environmental Molecular Sciences Laboratory, a Department of Energy national scientific user facility located at the Pacific Northwest National Laboratory.
