1. Introduction
Ensemble forecasting traditionally denotes the running of several forecasts from the same initial time, with the only differences among these forecasts being in their initial conditions consistent with analysis uncertainty. The divergence of the individual ensemble members provides quantitative information on the likelihood of possible outcomes. Ensemble techniques yield significant improvement in predictive skill at the medium-range over higher-resolution, deterministic forecasts. Currently, ensemble predictions are run operationally at several of the major forecast centers throughout the world.
A major challenge of the ensemble prediction is condensing the large amount of information contained in its voluminous output into a simple, user-friendly form. This includes providing forecasters with succinct and operationally relevant information that displays the essential aspects of the range and likelihood of alternative scenarios. In particular, it is important to provide the forecaster with the flexibility to display individual predictions, subsets of individual predications, measures of central tendency (mean, median, etc.), and quantifications of uncertainty via ensemble spread and probability distributions for user selected fields, levels and locations.
The Fleet Numerical Meteorology and Oceanography Center (FNMOC) runs operational ensemble forecasts that are generated by a version of the Navy Operational Global Atmospheric Prediction System (NOGAPS). NOGAPS is a global, spectral model that includes physical parameterizations for radiation, boundary layer processes and a representation of precipitation. NOGAPS provides skillful medium-range guidance that support operational users of FNMOC products.
2. Project objectives and accomplishments
The FNMOC ensemble system consists of 10 members, each of which is a 10-day NOGAPS forecast. These members include the 0000 UTC operational forecast, a lagged forecast from the previous 1200 UTC operational forecast, and 8 forecasts based on perturbations generated by a breeding cycle. The 8 bred-mode members are run with a horizontal resolution of T63 (triangular truncation at wave number 63) to 10 days and the operational forecasts are performed at T159 to 6 days, then truncated to T63 for days 6-10.
The analysis time for the EFS is 0000 UTC and the output products are available at 12 or 24 hour intervals, depending on the output parameter. The output fields are interpolated to mandatory pressure or height levels and onto a uniform one-degree latitude/longitude grid. Due to data storage constraints, only a limited set of fields was retained for post-processing at the University of Arizona. The output fields available to the University of Arizona include geopotential height at 1000- and 500-mb, air temperature at 850-mb, winds at 19.5-m above ground level (AGL) and 12-hour precipitation accumulations. These data are transmitted from the FNMOC via the Internet to the University of Arizona where they are archived for future efforts regarding verification of the ensemble predictions and calibration of the model output.
2.1 Project objectives
In this project, we used output from operational ensemble forecasts generated at the FNMOC to achieve the following project objectives:
The group at FNMOC (M.A. Rennick) was responsible for providing grids from their EFS and the University of Arizona (S.L. Mullen and L.M. Farfan) was in charge of developing software to generate statistics and real-time weather displays of the FNMOC grids. The post-processing phase of the ensemble forecasts was performed on a workstation at the University of Arizona. Data arrive to this workstation within a few hours after the forecasts are executed. The data are then decoded, sorted, archived, analyzed and converted into a format tailored for several graphical packages.
2.2 Research and development activities
During the project, we performed the following activities and accomplishments:
a) Static displays of EFS
Maintained the daily operation of a static web-site on the internet, which presented the basic statistics of the current forecast. The statistical parameters shown included ensemble means, medians, standard deviations, and extremes for 500 mb heights, 1000-500 mb thickness, air temperature at 850 mb, winds at 19.5 m AGL and accumulated precipitation at 24 h intervals. The user could first select a combination of one field and statistical parameter and then a pre-defined GIF image that was already on the computer disk was displayed in the user web-browser. In addition to the basic statistics, the probability of selected fields (e.g., wind speeds, precipitation amount) that exceed critical thresholds were shown. The computation of probabilities used only the 8 forecast members that were initialized with the perturbed conditions.
In the static portion of the web-site, the graphical products were pre-defined by the developer. They were generated one time per day during the period 1400-1600 UTC and remained without any modification until the next forecast cycle was processed. This portion of the site presented the geographical distribution of the essential atmospheric parameters and provided a simple overview of the temporal evolution of these fields throughout the forecast period. The geographical area shown in this section included the contiguous United States and portions of Canada, Mexico, the eastern Pacific and western Atlantic Oceans. The code that generated these images was relatively simple and made use of C-Shell scripts that generated plots of the same combination of fields and parameter every day. However, the number of fields and areas available were somewhat limited and the user could not request displays that were not available on a pre-defined list of geographical areas.
b) Inclusion of the NCEP/NCAR reanalysis climatology
Climatological means, variances and probabilities from the NCEP/NCAR reanalysis were used for normalization of ensemble spreads, computation of forecast anomalies and verification with respect to the long-term observations. The reanalysis dataset was generated from global data for a period of 40 years (1957-96) and included atmospheric fields at a horizontal resolution of about 2.5-degrees. For the purposes of our study, we computed the climatology of atmospheric fields for the period 1979-1996 and, in order to compare these fields with the FNMOC ensembles, the climatological grids were interpolated onto the uniform 1-degree grid.
The normalized spreads are useful for estimating limits of deterministic skill of the ensemble forecasts associated with significant weather events such as tropical cyclones and middle-latitude systems that develop over the oceans and can be disruptive to the naval operations. In the web-page displays the geographical area shown is the Northern Hemisphere and includes ensemble means, anomalies and standard deviations.
c) Interactive displays of EFS
In the interactive portion of the web-site, the graphical products available offered some degree of flexibility to the user. In this case, the displays were generated when the user submitted a request. The user was allowed to select from a group of certain options such as statistical parameter, vertical level, geographical coverage and forecast time. The geographical areas that were available include global coverage as well as hemispheric displays of the Atlantic, Pacific and Indian Oceans and the Northern and Southern Hemispheres. In addition, the user had the choice of displaying individual ensemble members.
The basic operation of the interactive displays required a sequence of steps that were performed for each user request, generation and display of selected fields. The first component in this process was the submission of a user request via a web-browser where the available options are presented in a simple form that makes use of the Hyper Text Markup Language (HTML). This included a list of the forecast periods, model fields, statistical parameters and map projections. Once the user submitted a request, code based on the PERL language and C-Shell scripts generate a Graphics Interchange Format (GIF) image and this image was displayed on the web-browser using HTML. The code made use of the input parameters selected by the user and executed programs using the available graphical packages.
The graphical packages that we used were the General Meteorological Package (GEMPAK), the Interactive Data Language (IDL) and the graphical package from the National Center for Atmospheric Research (NCAR-graphics). In order to evaluate the performance of these packages, we developed software that generated similar graphical products with each package. Depending on the efficiency of the package, it took a relatively short period of time (a few seconds) to a relatively long wait (about a minute) to process the request, generate the image and display it on the user web-browser. We found that GEMPAK was by far the slowest and give unacceptable delays (up to one minute) whereas IDL and NCAR-graphics were much faster (a few seconds) and were comparable in speed.
A popular application is the display of all the ensemble members in the form of "spaghetti" diagrams. At a given forecast time, these diagrams showed the geographical distribution of a single contour value for each of the ensemble forecasts and included the climatological information derived from the NCEP/NCAR reanalysis. In this section, the user was asked to provide the desired field and contour value, the forecast time and the geographical coverage of the display.
Also included in the interactive portion of the web site was the capability of generating time series of the ensemble forecasts for an individual station, so-called forecast plumes. These plots were useful to determine the time evolution of a given forecast field for a fixed station selected by the user. In a single image, the evolution was displayed for all 8 ensemble members, the ensemble means and, for the purpose of climatological reference, the reanalysis time series for the same location. Another type of display related with time series consisted in showing the behavior of specific fields along ship tracks. These tracks followed the projected motion of a ship and presented the predicted fields at different times.
d) Forecast validation and verification
An important component of this project was the objective evaluation of the utility of the FNMOC ensemble products through rigorous verification and calibration techniques. We performed activities related to the verification of the FNMOC forecasts against (surface and upper-air) observations in selected oceanic basins and against the FNMOC initial fields (or analyses). Initially, we computed mean and root-mean-square (RMS) errors and identifying the performance of the ensemble forecasts. Some of these errors were displayed in a section of the static web-site, along with the display of scattergrams of forecast-observations pairs. The verification of the probability forecasts was performed via the Brier Score and Ranked Probability Score.
e) Prototype system delivery
We initiated the delivery the EADS prototype system to the FNMOC for preliminary evaluation. The scripts that generate the interactive displays were transferred to the FNMOC's computer system via the Internet. Due to the restrictions imposed by the Navy to external users, the University of Arizona group was not able to provide assistance in the installation of the prototype system and in the evaluation of the code's performance on that system.
3. Summary of University of Arizona and FNMOC exchanges
The participants of this project have been able to interact in the development of graphical displays of the ensemble forecasts. The FNMOC provided assistance to initiate the design of the interactive web-site via the documentation on the initial version of their model display and verification system. This information was used to design the fundamental portions of the current EADS and to start coding the interactive display system. In addition, the FNMOC provided valuable suggestions and ideas to modify the current display system according to their needs and available computer resources.
4. Presentations and publications
Farfan, Luis M., S. L. Mullen and M. A. Rennick, 1999: Analysis, Display and Verification of an ensemble prediction system. Preprints 13th Conference on Numerical Weather Prediction. Denver, Colorado, 15-19 September.
5. Summary of benefits and problems encountered
5.1 Benefits to the university
The faculty and students at the Department of Atmospheric Sciences (University of Arizona) welcomed the availability of the output from the FNMOC EFS products. These products were used by S.L. Mullen in his synoptic meteorology course for their 3-5 day forecast exercises via the static and interactive versions of the web-site. In addition, these products were available in GEMPAK format for visualization with graphical interfaces, such as GARP. We found that our web-site ensemble products were regularly used by several agencies in the United States, Australia and South America. There was also a significant transfer of knowledge from this project to other COMET projects. In particular, the real-time MM5 forecasts for Arizona, performed in conjunction with the National Weather Service (NWS) Forecast Office in Tucson.
The project was hampered by a few problems:
5.2 Benefits to the FNMOC
As a result of this project, the FNMOC ensemble forecast system received daily scrutiny that would have been impossible with FNMOC resources alone. FNMOC modelers and customers were able to better understand the ensemble behavior due to the enhanced web displays developed by UA. The verification displays and comparisons to climatological variations have been a particularly welcome addition to the existing FNMOC products.
The most important specific transition has been the software to support the interactive generation of time series (plume) displays at user-defined locations. UA developed this software, transferred it to FNMOC, and worked with FNMOC to make it compatible with the operational constraints of the FNMOC system. The product satisfies a repeated requirement levied by our customers to provide long-range guidance along ship tracks. The structure of the interactive interface can be generalized to support other requirements as well.
Concur with UA analysis of problems associated with this project. Close interactions between remote partners are difficult, but were made possible through mutual dedication. The lack of operational reliability of university network and computer systems is not unexpected. Rather than attempt to change this, partnerships should be designed to anticipate lost and delayed data. Procedures should be implemented to recover from such events.