Iowa State Univ.: "Eta ensemble forecasts of elevated nocturnal convection"

Final Report

SECTION 1: PROJECT OBJECTIVES AND ACCOMPLISHMENTS

1.1 In this 1-year project, we proposed to:

(a) develop an inventory of elevated convection forecast failures/successes
(b) generate eta ensembles from mixed convective schemes
(c) validate ensembles with subjective and objective methods
(d) perform real-time ensemble output during warm-season 2002

We have accomplished all of these tasks.

a) Our inventory consists of 35 MCS cases and 3 null cases from 2002. The MCS cases may be broadly classified into three categories: MCSs originating in front range (4 cases), MCSs developing in-situ (15), and various non-classic MCSs that developed in the vicinity of surface warm fronts and fronts aloft in the central U.S. (16). These classifications were suggested as important in the proposal because the different initiation mechanisms associated with them might result in one type of system being more consistently forecasted better than the other type.

In addition to the 2002 events, we created an inventory containing 7 cases from the warm seasons of 1999 and 2000, along with 17 cases occurring in 2001.

b) During the first 6 months of the project, we tested an ensemble made up of 9 members that used changes in the KF convective adjustment time step and trigger function, along with the KF control and BMJ control runs. We found that this ensemble system did not have significant spread, with all of the KF variants resembling closely the control KF case.

After consultations with Jack Kain (NSSL/SPC), we tested five other versions developed by him, 3 of which were run along with the BMJ and KF control runs in real-time over the summer of 2002 to assist forecasters at the NWS DMX office.These versions of the KF scheme included:

(i) an alternative form of the downdraft that includes a downdraft mass entrainment rate based upon the grid point relative humidity and lapse rate. The entrainment rate decreases as grid point relative humidity increases and as grid point lapse rate approaches moist adiabatic. Maximum entrainment rate occurs for RH=0% and lapse rate equal to the dry adiabatic lapse rate,

(ii) a modification to include a positive temperature perturbation in the updraft if the grid point sounding there is nearly moist adiabatic,

(iii) a version using both of the above plus two additional modifications described below. The two below have been tested individually but are not run individually in real-time.

(iv) a change in the computation of the updraft radius, which is important in the calculation of entrainment of environmental air. Instead of a constant value of 1500 m, the radius can vary from 100 to 3000m as a function of the grid-point vertical velocity in the 300mb layer above the LCL. Descent results in smaller radii, with ascent greater than 4 cm/s resulting in a radius of greater than 2000 m,

(v) a change in the way evaporation and melting is calculated in the updraft. The adjustment to melting allows it to occur in at most a 400 mb deep layer, whereas in the original formulation, the melting rate was directly related to the updraft mass flux. The adjustment to evaporation rate relates it to the grid point mixing ratio deficit, instead. of the mixing ratio deficit of the downdraft (original formulation) which constrained it to the relative humidity of the downdraft, which is set to 10%.

The real-time ensemble run during summer 2002 (with members i-iii plus the control KF and control BMJ) seemed to provide better spread and potentially improved forecasting skill. For some individual cases simulated in research mode, we used as many as 15 ensemble members.

c) We subjectively evaluated three of the case studies in some detail. All three cases involved convection that initiated aloft over the central U.S. - a large MCS, a small MCS, and widespread, discrete storms. In both MCS cases, there were large areas (1/4 the size of IA) of rainfall exceeding 1.0" per 24-hours in the observations that was not forecasted in the unmodified KF run. Only ensemble members that emphasized parameterized precipitation increased the areal coverage of precipitation. Most simulations seemed to accurately forecast the timing of convective initiation.

A feedback form was completed in June 2002 in order to facilitate discussion concerning model performance and difficulty of forecasts for the events occurring in real time during the 2002 warm season. The form was only used a limited number of times due to time constraints with the operational forecasters and occasional computer problems, but analysis of the feedback will be useful in design of future projects.

Objective verification focused on events that occurred in August 2002, which was a particularly active month in which 14 cases were catalogued. We computed equitable threat score for six ensemble members for thresholds of 0.01, 0.02, 0.05, 0.1, 0.15, 0.25, 0.35, 0.5, and 1.0 inches, averaged over 13 cases with 39 6-hour periods (00-06, 06-12, 12-18h). The members included:

bmj - betts-miller-janic
kf - unmodified kf
dd - kf with downdraft
tp - kf with temperature perturbation
rad - kf with radius modification
all - kf with all modifications

The ETSs were:
.01 1.0

The relatively low ETSs for high precipitation rates indicates continued difficulty of these model variants to both simulate high rates of precipitation and correctly position this precipitation. The ETSs are somewhat higher than those found using the standard KF scheme (slightly older version) in a study by Gallus and Segal (2001) examining 20 MCS events during the summers of 1998-2000. The "all" simulation did worse than the others. None of the KF variations improved greatly upon the KF scores, with the biggest improvements being in the radius version. Unlike the Gallus and Segal study, the KF runs earned higher ETSs than the BMJ for light rainfall amounts.

Average ETS values for the 2002 events using the KF variants were very similar to the ETS for the original KF implementation in the ETA model. However, ETS compared on a case-by-case basis shows large variability with the KF variants showing much improved accuracy in some cases. We have examined the degree of similarity between the KF variants and original KF implementation with linear regression analysis. The ETS scores for each variant were regressed against the ETS scores for the original implementation for all thresholds. The R-squared value is a measure of the linearity between the variants and original implementation with values approaching 1 indicating the variant produces either forecasts that are identical to the original implementation or forecasts with bias that is identical in all cases.

Below are R-squared values for a .01 threshold:
bmj 0.525
dd 0.765
tp 0.972
rad 0.950
all 0.519

The R-squared values show that the kf scheme with all modifications and the bmj run differ the most from the control kf. Modest differences occurred with the downdraft modifiction; otherwise, the perturbations did not differ much from the control. All of the 9 ensemble members first tested (described above) had R-squared values around .9 or higher, indicating little spread.

These results suggest that the bmj run and the downdraft run in combination with the original implementation could form an ensemble of equally likely forecasts.

We have examined the utility of ensemble forecasts made with these variations. We computed the rank histogram of this ensemble for each case in August 2002. The average rank histogram is:

0.128117949   0.098766667   0.10615641   0.157766667   0.509182051

This indicates that the ensemble tends to underpredict precipitation amount and has insufficient spread to capture the observed precipitation pattern. While the R-squared values indicate that strides have been made toward producing useful operational ensembles from variations of convective schemes, the rank histogram indicates more work is needed before a final product is realized.

The tendency was for all ensemble members to underestimate precipitation amount. The frequency for no precipitation is shown below:

obs 0.66041958
bmj 0.639412587
kf 0.633025641
dd 0.639114219
all 0.531608392

The frequency for less than 11 mm (about 0.5" or less) follows:

obs 0.296923077
bmj 0.305986014
kf 0.329202797
dd 0.332428904
all 0.450041958

d) During the mid-late summer of 2002, a web page was designed for the DMX-NWS office where the current ensemble run off of 00 UTC data could be seen with forecasts out to 48-hours, along with the previous day's ensemble run, and a small archive of runs from earlier this year. The real-time products were made available to operational forecasters by 6 am. We attempted to have them available by 4 am to better assist the morning forecaster, but other computational demands on our systems often delayed the output to 6 am.

1.2 The NWS goals were to:

(1) Utilize the ensembles near-real-time as operational forecast input during warm-season 2002

(2) Provide subjective verification of the ensembles and forecaster insight via the feedback form

Regarding (1), Ensembles were available for forecaster use via the web page, and they were utilized occasionally in the forecast process. Since ensemble data usually became available after operational forecast deadlines, opportunities to use the ensemble output in real time were sometimes limited due to time constraints. If ensembles were not available for the 4 a.m. forecast process, forecasters were asked to look at the output during the morning hours prior to forecast updates.

Regarding (2), The feedback form provided an effective means for forecasters to provide insight into ensemble quality, and assist with verification. Forecasters tended not to fill out the form if the ensemble data was not available in time for their real time use. The archive capability allowed forecasters to go back during quieter weather to review the data, providing insight as to whether model solutions were helpful or added value from an operational perspective.

2.1 The primary focus of the work adhered closely to the goals of the original proposal. The collaboration with NWS-DMX on this project, as with other COMET projects, continued to strengthen our understanding of the needs of the operational community.

The time available to conduct local research is limited at the forecast office, so projects such as this are extremely valuable, and they keep the lines of communication open to all scientists at ISU, not just those principally involved in the project. Our presence at ISU continues to attract 2 or 3 Student Volunteers each semester. One student embarked on a Senior Research project with a DMX forecaster as mentor. Craig Cogil and Karl Jungbluth of the DMX Office provided a lecture and lab on weather radar for ISU in April. NWS forecasters continue to collaborate with ISU professors, researchers and students to host the Central Iowa NWA Severe Storms and Doppler Radar Conference.

SECTION 3: SUMMARY OF BENEFITS

3.1 Although an important part of the research was an investigation into the best model and ensemble configurations in general to help predict elevated convection, a large benefit of the work was the real-time weather forecasts for precipitation made available to the NWS on the Internet. The refined system with 5 members showing more substantial spread was available in real time over the mid and late summer months. At times during this period, the output should have been helpful to the forecasters, as it appeared to offer solutions that better resembled what ended up happening. Also of benefit during the project were informal meetings with forecasters to explain the webpage and ensemble design and receive feedback on how it might be improved.

A final benefit was the ability to work directly with the author (Kain) of the primary convective parameterization scheme being modified to test possible improvements. Dr. Kain was eager to help because he wanted to see these variations tested for improved elevated convection forecasting but lacked the resources locally at SPC/NSSL to do the frequent testing.

3.2 It takes a project like this for many forecasters to understand the significance of model parameterizations and the impact of convective schemes upon a given model's output. Another benefit is that forecasters seen that it takes a significant amount of time and research to improve the forecast models, and that they can be involved. NWS forecasters have output from a number of numerical models at their disposal, and they receive training on the basics of numerical weather prediction, but this project allowed them to take a step forward in their knowledge. NWS Des Moines was fortunate to have the Eta ensembles available to forecasters on a daily basis. Forecasters were introduced to the basics and benefits of short-range ensemble prediction, which is a tool that some are considering for the first time. Benefits to forecast accuracy would be even greater if ensemble output was available in AWIPS for direct comparison to other models. Discussions are underway with ISU to make this happen.

SECTION 4: PRESENTATIONS AND PUBLICATIONS

Anderson, C. J., W. A. Gallus, Jr., R. W. Arritt, and J. S. Kain, 2002, IMPACT OF ADJUSTMENTS IN THE KAIN-FRITSCH CONVECTIVE SCHEME ON QPF OF ELEVATED CONVECTION, Preprints, 15th conference on Numerical Weather Prediction, San Antonio, TX, Aug 12-16, 23-24.

Anderson, C. J., W. A. Gallus, Jr., R. W. Arritt, and J. S. Kain, 2002, IMPACT OF ADJUSTMENTS IN THE KAIN-FRITSCH CONVECTIVE SCHEME ON QPF OF ELEVATED CONVECTION. Presentation, The World Weather Research Programme's (WWRP) International Conference on Quantitative Precipitation Forecasting (Reading, UK, 2-6 September 2002).

Anderson, C. J., R. W. Arritt, and W. A. Gallus, Jr., 2003, THE USE OF SEVERAL DIFFERENT TYPES OF SHORT-RANGE ENSEMBLES TO PREDICT WARM SEASON CONVECTIVE SYSTEM RAINFALL. Journal of Hydrology, (manuscript under preparation - submission planned Feb. 2003)

SECTION 5: SUMMARY OF PROBLEMS ENCOUNTERED

None