SUNY-Albany: Empowering the forecast consumer: An investigation of citizen need for, and the technology for communicating, process-centered weather information"

Final Report

We hereby submit a report of findings from a three-stage research project carried out between March 1998 and June 2001, titled "Empowering the Forecast Consumer." The overall goal of the project was to develop prototype forecast products for the general public that would provide consumers with a way to make their own assessments of the likelihood of weather conditions directly affecting their plans, rather than relying on forecaster predictions alone (whether NWS forecasters, or commercial forecasters). The goal of empowering consumers in that way arose from the findings from focus group interviews in a pilot study we conducted that some portion of consumers do not take weather forecasts at face value, but seek additional corroborating or disconfirming information. This tendency seems to arise from the value consumers place on preserving normalcy and keeping their own counsel, coupled with their general distrust of the accuracy, completeness, or specificity of forecasts from official sources (especially commercial media sources). If NWS products are created that give consumers better tools with which to supplement forecasts (as they already do) to make their own assessments of risk, the benefits would be (a) heightened credibility of and reliance on NWS products, and (b) improved weather related plans and decision-making on an everyday basis that might carry over to consumer responses to emergency weather advisories and warnings.

This report is organized in six sections as follows:

1. Overview and rationale for the project as presented in the original proposal.

2. A summary of findings from focus group interviews in the first phase that were aimed at eliciting consumer needs for forecasts and weather information, the way consumers use forecasts and weather information, and their assessments of the products currently available.

3. A description of and rationale for prototypes of new forecast products that were developed in the second phase of the project, in response to focus group interviews in the first phase.

4. A summary of focus group interviews carried out in the third phase, to elicit assessments of the prototype forecast products we developed from the citizens who would use them.

5. Software development issues and implementation issues regarding these prototype forecast products that emerged from focus group interviews in the third phase.

6. Summary and concluding discussion

1. PROJECT OVERVIEW AND RATIONALE

Problem

Improving the effectiveness of communication to the public about weather events is as vital to the NWS mission as improving forecast accuracy. This is most critically true in emergency situations, when hazardous weather conditions threaten or are imminent. NWS responsibility for communication in emergency weather situations has led to studies of ways to improve communication and coordination between NWS and the various agencies, officials, and communications media who play a role in public safety during emergency situations (Adams, 1995). It has also led to studies of public response to messages during emergency situations about weather conditions, to find what can be done to make those messages most effective in bringing about the desired public response (McLuckie, 1974; Carter, 1980; Leik, Carter, & Clark, et al., 198 1; Mileti & Sorenson, 1990).

However, improving the effectiveness of emergency weather communication with the public may depend on more than improving the messages produced specifically during emergency situations. Citizens' daily experiences with weather forecasts, and their subjective perceptions of forecast accuracy, may produce relevant habits of response that carry over to emergency situations. We conducted a pilot study of citizens' understandings and uses of weather information under the COMET Partners Program (Sanders & Westergard, 1996) using focus group methodology. The value of focus group interviews is that they can "tap into" not just what uses people make of weather information and forecasts, but more importantly, how they understand and reason about weather information and forecasts, use them to make weather-related plans, etc. The results of focus group interviews are thus not easily or usefully quantified (in contrast with polling methodologies), but are best stated in terms of dominant themes and concerns that the analyst infers from transcripts of discussions.

In our pilot study we conducted focus group interviews in the Albany, New York area of particular consumer constituencies. For persons most likely to be actively involved with weather-related planning, we interviewed groups of, respectively, farmers, building contractors, and outdoor recreationists (golfers); for persons unlikely to have a standing concern with weather events, we interviewed groups of, respectively, parent-teacher organization (PTO) members and senior citizen residents of an assisted living facility.

Our central finding was our informants did not trust, and thus do not rely solely and directly, on professional weather information and forecasts in making weather-related plans. This does not mean that our informants ignored weather communications—to the contrary, they tended to seek out and attend to weather communications on a regular basis. However, as explained below, their beliefs and attitudes made it an uncertain and contingent matter what response to weather communications they would make when adverse weather conditions were forecast (where we mean by " adverse weather condition" not only hazardous conditions, but any condition that for an individual consumer would negatively affect his or her plans).

Most of our pilot group informants displayed a weak understanding of the forecast process and of weather events, but they felt certain that weather events are inherently unpredictable, and thus felt that forecasts are inherently unreliable despite the best efforts of meteorologists and despite recent gains in forecast accuracy. This was the basis of their distrust. Added to that was their perception that weather information sources tend to exaggerate and sensationalize possible adverse weather (referring mostly to commercial media sources; 60% of our informants made no mention of the National Weather Service at all as a direct or indirect source of weather information).

These beliefs alone would explain why our informants tended to not rely solely and directly on expert sources of weather information and forecasts in making weather-related plans. But in addition we inferred two values that informants have that also promote a reported tendency by many to second-guess official forecasts. One value is to "preserve normalcy" — a resistance to altering routine activities or changing plans (with the notable exception of senior citizens, who were generally ready to forego activities and plans in favor of avoiding any extra difficulty or any risk to safety). This desire would promote close attention to weather communications, in order to be able to make plans that would not have to be changed later because of unexpected weather. But paradoxically it would promote caution about accepting official forecasts at face value, when respondents are skeptical about forecast accuracy.

A second value we inferred is to "keep their own counsel" — to decide for themselves what their level of risk is and what response to make. Informants seemed to reserve it as their option to apply personal judgment, past experience, and current priorities to decide what weight to give a weather forecast and how to respond to it in carrying out their activities and plans, ranging from the possibility on one extreme of either disregarding it entirely, to the other extreme of acting on it directly. Informants reported a tendency to treat weather forecasts from expert sources as only preliminary. They go onto second guess them after seeking additional corroborating or disconfirming evidence (detailed below).

This mirrors findings that during weather emergencies, citizens tend to seek additional, corroborating information when advisories and warnings are issued, rather than respond immediately and directly to those communications (Leik, Carter, & Clark, et al., 1981; Mileti & Sorenson, 1990). However, our findings suggest a new explanation for that finding. Instead of regarding this tendency to seek corroborating information as being specific to emergencies, a product of "denial" or stress about how much is at risk, it seems to involve the tendencies to "preserve normalcy" and "keep their own counsel" that have been cultivated by consumers' daily, non-emergency, experience with weather communications.

The further corroborating or disconfirming evidence that pilot group informants reported they relied on in keeping their own counsel differed across individuals, and ranged from the application of folklore and proverbs (regarding the behavior of animals, plants, clouds, etc.) to " rules of thumb" (one informant felt that weather events usually occur a day after the forecast predicts) to alternate information sources (most notably, the Farmer's Almanac) to amateur meteorology (projecting from the reported location and movement of weather systems and events further west the likelihood and timing of future weather events "here" over the next several days).

Our findings indicate that improving the effectiveness of the communication of weather information depends on increasing the willingness of citizens to trust and rely on expert sources, especially in emergency situations. There seem to be two main sources of public resistance to this. One is the tendency of citizens to keep their own counsel. The other is the conviction among citizens that weather information and forecasts are inherently unreliable.

Solutions

It does not seem feasible to attempt reducing the desire of citizens to keep their own counsel, even if that were an appropriate focus of NWS effort. Rather, the practical and appropriate goal for NWS is to reduce the evident distrust citizens have of professional weather information and forecast sources, thus increasing the chances that in keeping their own counsel they will rely solely and directly on expert sources of weather information and forecasts, and abandon other, less scientific, evidence they rely on now. There are two broad stratagems that could be adopted for this purpose.

One possible stratagem would be to undertake an information/public relations campaign to heighten public awareness of recent scientific and technological advances that are leading to rapid gains in forecast accuracy. However, our pilot study (Sanders & Westergard, 1996) does not make it seem likely that this would result in a noteworthy improvement. A number of our informants already feel that there have been gains in forecast accuracy. But this did not change their conviction that weather events are inherently unpredictable, and that weather information and forecasts are therefore inherently unreliable no matter how much the rate of error has decreased. This is because as long as they consider that no matter how much the science improves, there is still a possibility of error, they can reasonably have reservations about every forecast. In addition, the values of "preserving normalcy" and "keeping one's own counsel" remain. Those values will lead many informants to second guess forecasts as long as they perceive there to be the potential for forecast error, no matter how small. Hence, a campaign to improve the credibility of forecasts by educating consumers about how much the science has improved is not likely to be effective.

The alternate stratagem, which our project set out to investigate, has two intertwined aspects. One is to provide consumers with a sound scientific basis for making their own assessments of risk — to facilitate and guide, rather than resist, the efforts of citizens to "keep their own counsel" in deciding what weather conditions might actually occur, and what they should do in response. The other aspect of the stratagem is to change the form and content of weather communications so as to reduce or eliminate those aspects that potentially foster citizen's conviction that official sources of weather information are untrustworthy.

We have devised a single communication solution that lends itself to achieving both ends, one that new communications technologies make feasible and attractive, and that is consistent with and facilitated by the recent development of the Interactive Forecast Preparation System (IFPS).

First, based on our prior study (Sanders & Westergard, 1996) we hypothesized that the conviction that weather information and forecasts are unreliable is strengthened on a daily basis by what we refer to here as conclusion-centered weather information and forecasts. Most weather information and forecasts consist of conclusions that meteorologists have reached on the basis of relevant data and models as to what is likeliest to happen. When conclusion-centered statements about what "will" occur are presented to citizens, they create a higher likelihood that people will experience forecasts as being in error, no matter how qualified they are with probabilities of precipitation (POPs) and the like. This is partly because no matter how technically accurate a forecast, it cannot be detailed enough to anticipate the planning needs of a mass audience of consumers involved in a wide variety of activities. It is also because, based on feedback from our informants, forecasts are subjectively experienced as being wrong whenever (a) the forecast led the person to make or change weather-related plans, and (b) actual weather events were different enough from what the forecast predicted that the person's planning was nullified (e.g., activities were cancelled that need not have been, or activities were planned that had to be cancelled). For example, suppose that rain is forecast in an area with a 70% probability, and a person adjusts plans with the expectation that it win "probably" rain. If the person is located in the 30% of the area that did not get rain, however, it will seem that the adjustment of plans was unnecessary and that the forecast was wrong. In that way, a conclusion-centered forecast could seem wrong or misleading to whatever proportion of persons in an area were led by a forecast to make plans that they find they need not or should not have made after all.

This cannot be remedied by making conclusion-centered statements weaker and more qualified, because that would only weaken their usefulness to the public for making weather-related decisions, and that in turn would reinforce the public's conviction that weather information and forecasts are unreliable. The alternative is to replace conclusion-centered weather communications with what we call process-centered information and forecasts. By process-centered messages we mean depictions of evolving weather and alternative ways atmospheric dynamics could influence it, making it possible for citizens to "see" weather conditions evolving in advance, under alternative scenarios, and so they can make their own assessments of certainty and uncertainty about the potential for conditions that would favorably or adversely affect their plans. Such messages would have at least two benefits. First, they would make it vivid and meaningful to citizens how much our scientific understanding lets us know with certainty. Second, it would make evident that there is a sound basis for making assessments of uncertainties and contingencies. Thus, for example, suppose that instead of a conclusion that there is a 70% chance of rain, forecasters gave citizens a depiction of the increasing potential for rain in terms of evolving weather, and different ways that might vary, as atmospheric dynamics affect it. Citizens could not then experience it as a failure of professional forecast sources if it did not rain at their particular location. Nor could they reasonably blame the forecast for being "wrong if they make their own assessments of risk and plan accordingly.

The other dimension to be taken into account here is the evident tendency of citizens to keep their own counsel. There are a variety of psychological, cultural, and practical reasons why persons in general might do this; U.S. citizens in particular have a strong cultural foundation for keeping their own counsel (the cultural importance of self determination and personal freedom) for resisting dependence on the authority of expert sources in undertaking activities and making plans. In addition, conclusion-centered weather information and forecasts often make it a practical necessity for citizens to keep their own counsel. It is impossible to provide information and forecasts to the public at large that are detailed enough to apply directly to the diverse weather-related activities and plans of individuals. Details and variations in weather conditions that matter to commuters are different from ones that matter to recreational boaters and different in turn from ones that matter to a contractor, etc. Hence, it is unavoidable for individuals to have to extrapolate from the weather information and forecasts publicly available, no matter whether they are conclusion-centered or process-centered ones, how their specific activities and plans might be affected, and what adjustments they can or should make. In that case, the willingness of citizens to rely on professional weather information and forecast sources depends on how useful and informative they find them for the purpose of extrapolating how their own weather-related activities and plans might be affected. A conclusion-centered message is less informative for this purpose than a process-centered one.

Process-centered messages can more readily empower citizens to keep their own counsel, to know what to look for and be able to make and adjust their own judgments of the level of risk involved to their own activities and plans. Empowering citizens in this way stands to reduce the current tendency reported by our informants to seek additional, often unscientific, corroborating or disconfirming evidence about future weather.

But it is not an easy matter to make process-centered messages accessible and understandable to audiences that lack specialized meteorological training. To achieve this requires that we reverse our thinking about the way forecast messages utilize text versus graphics. Text is currently the primary medium and graphics are secondary as illustration or in "shorthand" for text-based weather information and forecasts. This is workable as long as the forecast products are conclusion-centered, because such messages do not have to involve technical specifics, and they can be communicated in words with considerable brevity (e.g., "A sunny start tomorrow, but clouding up with rain likely the afternoon; highs in the 70s with light variable wind"). However, if we want forecast messages to be process-centered, providing consumers with information about atmospheric trends and dynamics on which forecasters' predictions are based, texts do not work as the primary medium. Text-based information about trends and dynamics would be too technical, jargon laden, and long for consumers to understand or use. In order to show trends and dynamics as they bear on the potential for the occurrence, timing, magnitude, and duration of specific weather conditions and events, visual displays are far more efficient than a text-based message would be. We thus end up making graphics the primary medium, and text-based messages secondary — summaries or paraphrases of what has been depicted graphically.

The Solution We Have Investigated

Hence, we propose that graphics be considered the primary or initial format for communicating process-centered messages, with text developed secondarily as descriptive of the graphics. Such accompanying texts could then be modified for application in text-only media (print as well as radio — see the example in section 5). Appropriate graphic representations offer the potential to provide accessible and understandable weather information to the public, without requiring more time to present and consume than most conclusion-centered messages in words. Obviously, the technology is available for communicating forecast products primarily graphic form — not only television, but also increasingly the Internet.

Graphics that adequately represent atmospheric processes will necessarily differ from most graphics currently used in televised weathercasts and on the Internet. Graphic displays now being used in forecasts do not provide tools with which consumers can readily assess the certainty or uncertainty with which particular weather conditions and events will affect them, they merely illustrate what conditions and events are being predicted in the text-based forecast. The exception is the graphic displays of hurricane strike probabilities that are now being provided, and of particular interest among our informants and greatly valued by them, radar displays of the movement and intensity of currently occurring storms.

Hence, new ways have to be developed, and their usefulness for citizens tested, of graphically depicting weather as an evolving process, in such a way that consumers can make their own assessments of their risk of encountering adverse weather — adverse with respect to their own unique activities and situation — and plan accordingly. These depictions need to be understandable and accessible by the public at large, informative about the certainties and uncertainties associated with evolving weather under alternative scenarios. At the same time, they have to be forecast products like all NWS products that can be produced in a timely way, and are usable and accessible by the public at large from the NWS on the Internet, and derivatively (through descriptions of the graphics) on NOAA Weather Radio, as well as through commercial media sources.

Our goal in this project was to investigate the desirability, feasibility and means of shifting from conclusion-centered to process-centered weather information and forecasts.

2. SUMMARY OF PHASE I METHODS AND FINDINGS

The goal in this initial phase was to find whether the findings of the pilot project about the ways forecast consumers use weather communications, and what their needs are, generalize across geographic areas of different climates, and across consumers in terms of varying needs for weather information.

Procedure

In this phase and also the third phase of the study, as well as our pilot study, we relied on focus group interviews to find out about consumers' needs, understandings, and uses of weather information and forecasts. As explained above in discussing our pilot study, focus group data do not lend themselves to quantification, but are best reported in terms of dominant themes and issues that the analyst finds from a close reading of the transcribed discussions. But, also noted above, relying on focus groups, rather than individual interviews or questionnaires, has a distinct advantage, even though it is less efficient, and limits sample size. Informants talking to each other in a focused discussion of their practices, needs, values, etc. rather than talking to an interviewer, are likely to (a) stimulate each other to make observations, disclosures, reactions, evaluations, and the like that might not occur to them in response to an interviewer; (b) reveal influences, values, constraints on practice, needs, reasoning, and the like that a fixed set of interview questions might not tap.

The logistics of recruiting informants and arranging for a meeting time makes it costly to assure a representative sample, but for our purposes, sampling was not an issue. In this project, groups were formed in diverse geographical locations, and participants were drawn from diverse interest groups (for example, agricultural, or recreational), thereby avoiding any particular bias in composition across groups. Further, what we learn from focus group interviews is how people from representative types of constituency think about the subject at hand, and what they take into account. The informal check on how much reliance to put on informants' comments is whether there are recurrent, consistent issues, concerns, values, needs, and the like across informants and groups. If there is notable inconsistency in response across informants in an initial set of interviews, that would indicate a need for further, more careful sampling, and more closely managed interview protocols. In the case of our interviews there has been a high degree of internal consistency within each of what we have identified as two distinct types of consumer: proactive consumers (who are consistently engaged in weather related planning) and passive consumers (who are relatively insulated from weather events and more inclined to accept official forecasts at face value, and not second guess them).

Focus group interviews were carried out in and around four U.S. cities: Miami, Florida; Cincinnati, Ohio; Manhattan, Kansas; and Bozeman, Montana. These four sites were selected as differing in the range of conditions, and types of severe weather conditions and hazards, that residents contend with (tropical storms and freezes, in Florida; rains and flooding in the Ohio River area around Cincinnati; tornadoes and blizzards in Kansas; extreme cold in Montana).

In each of those geographical areas, five focus groups were formed, as follows: an agricultural group; a builders/contractors group; a recreational group (made up of people who engage in weather-dependent recreation, such as outdoor sports or boating); and two groups from parent-teacher organizations from contrasting socioeconomic neighborhoods. Across the groups in each area, there were roughly an equal number of men and women, though the actual proportion of men and women in each group varied widely. The method of recruiting participants was to contact organizations that represent particular constituencies, and asking them to recruit members in return for a monetary contribution to the organization. For the most part, this involved the help of cooperative extension organizations, builders' associations, recreation clubs (boaters, golfers, etc.), and the parent-teacher organizations of schools in the K-8 range.

It should be noted that our sampling of informants by gender, and by socioeconomic environment, was not systematic or rigorous. Local focus group organizers whom we hired were asked to consider those factors in the groups they formed, but the imperative was to form groups who could meet with us in the same time period, and there was little opportunity to obtain or utilize demographic information about each group's membership. The intent was simply to have enough variety in the social composition of our focus groups to find out whether gender, or socioeconomic environment, were possible factors in the way consumers use weather communications, that would have to be investigated more systematically.

Interview topics were: (1) sources of and reasons for preferring weather information and forecasts; (2) understandings of the forecast process; (3) evaluations of forecast accuracy and reliability, (4) uses of forecasts and weather information; (5) whether forecast information is taken "at face value" or supplemented or qualified, and if supplemented or qualified, on what basis; (6) what counts as emergency or hazardous weather conditions, and what use is made of weather information and forecasts under those conditions.

Findings

Our focus group interviews were designed to promote a discussion among informants about how they understand weather information and forecasts and how they use them. Our method of analyzing these discussions was to group participants' comments on particular topics introduced by the focus group leader, and then make a judgment about what concerns, attitudes, issues, and the like were expressed consistently enough that they should be reported. Our findings then are presented below in the form of statements of what informants expressed as to their preferences, wants, and reasoning about, say, displaying forecast models as line graphs, not as percentages as to approval versus disapproval, agree versus disagree, etc.

The main research question in this first phase was whether our pilot group findings held up across geographically diverse groups, specifically whether there is a residual skepticism about NWS and commercial forecast products, whether consumers who are skeptical second-guess the forecasts, and if so what they base this second-guessing process on. In brief, we found that much of what we learned from our informants in the Albany, New York area held up in the responses we elicited from informants in the four locales we visited (the Miami, Florida, area; the Cincinnati, Ohio, area; the Manhattan, Kansas, area; and the Bozeman, Montana, area). However, the new data enriched and deepened the picture we got from our pilot study informants.

In our pilot study we found that there was a tendency among some persons in each of the five kinds of groups we interviewed to use official sources of information and forecasts as a starting point, but to not trust their reliability and to try to second guess them by practicing amateur meteorology, using natural signs based on folklore, supplemental sources like the Farmers Almanac, etc. However, we found in this larger group of studies that we could characterize those consumers likely to do this, and those unlikely to. We found that the tendency to second-guess forecasts was most prominent among consumers whose activities, and often whose financial interests, were directly dependent on weather conditions (especially farmers and builders). Such consumers, whom we classify as proactive about weather information and forecasts, were the ones who characteristically use official forecasts as starting points, and seek additional information to corroborate or second-guess them. Other consumers, whom we anticipate make up a majority of the citizenry, are what we classify as passive consumers. Passive consumers are those whose self-interest is relatively unaffected by weather conditions, and who tend to take weather forecasts and weather information at face value. While the additional focus group interviews we carried out in this first part of the project gave us information consistent with what we learned from our pilot study, there were new, more specific findings of importance. In summary, these are:

• As noted, the tendency to second-guess official sources of information and forecasts is most prominent among those with a high level of everyday concern about weather conditions — the proactive consumers. In contrast, informants we classify as passive consumers either take official information and forecasts at face value and act in accordance with them, or do not consistently monitor official sources of information and forecasts at all. These consumers do not perceive themselves as being very significantly impacted by, or faced with costs depending on weather, aside from such matters as how to dress for the day. The exception would be odd occasions involving travel plans, or the location of a planned outdoor activity, and the like.
  
  
• There is a widespread tendency among all forecast consumers to rely on their acquired "local knowledge" about the way weather develops and occurs in their locality to supplement, qualify, or supplant — to interpret or qualify — official information and forecasts. Afternoon thunderstorms in Miami during the summer are expected to be brief and inconsequential; snow in Bozeman, MT is considered to be widely variable and dependent on local conditions created by nearby mountains, with forecasts that originate from nearby cities widely discounted because they are presumed to not take local topological characteristics into account.
  
  
• Proactive consumers want more "raw data" about current weather dynamics on which to base their own judgments of how they might be affected, and fewer interpretations and predictions from forecasters. There are repeated statements by our informants to the effect that they would like the television weathercasters to stop talking and step out of the way of the radar imagery being shown.
  
  
• Proactive consumers want a reliable basis for making weather-related plans and decisions 48-72 hours in advance, and do not consider official forecasts for conditions that far in advance to be reliable.
  
  
• Proactive consumers need information about particular conditions, not all conditions. Primarily they need information either about anticipated wind, temperature, or precipitation 48-72 hours in advance.
  
  
• Proactive consumers vary widely in their technical understanding of meteorology, and the time and patience they have for studying weather information and forecasts.

We did not find any systematic differences across geographic/climatic regions in the needs people have for weather information, and the ways they use forecast products. We did find a wide range in persons' understandings of weather and weather forecasting, from "sophisticated" to "naive," that seem tied to two factors: (1) how dependent their livelihood is on weather conditions; and (2) how connected they are to new technologies — to cable-television, Internet, NWS, and private vendors' weather data and forecasts.

We did not find any indications that men and women vary within or across user groups in any systematic way in regard to needs, uses, or understandings. Nor did we find any indication of such differences among informants from differing socioeconomic environments (based on comparing the two PTO groups in each city, as well as comparisons of urban versus rural informants, and eastern, Midwestern, and western informants).

Discussion/ Conclusions

The focus group interviews conducted in this first phase confirm what we concluded from our pilot study. There is a need for NWS products that empower consumers to judge for themselves the relative certainty that particular weather conditions and events will occur, and make their own assessments of risk accordingly. Based on our focus group interviews, we expect that the primary users of such products would be those we classify as proactive consumers.

Proactive consumers need different forecast products than are currently available to naive consumers ("naïve" in the sense of not having technical meteorological understanding). It is in the NWS interest to supply such products. Proactive consumers will evidently seek additional information with which to corroborate or second-guess forecasts, regardless of what NWS does. But NWS can provide them with better, more scientific information to use for that purpose, and in so doing, will bolster the quality of such second guessing, and will create greater consumer confidence in NWS products and in the underlying science. Greater confidence from proactive consumers has the potential to be diffused through their social networks to larger population segments.

It is clear from what our proactive informants revealed about their practices, and expressed wants, that they need and want forecast products that are "unmediated" — an uninterrupted presentation of possibilities and processes, not the forecasters' "conclusions," interpretations, selections, emphases, etc. Moreover, from their comments, we infer that what proactive consumers want are tools that will let them assess risk — let them project, relative to their own, specific situations and decision needs, what the level of certainty or uncertainty of specific forecast conditions is. This applies to being able to project not just the certainty or uncertainty that specific conditions will occur (pertaining to concerns with temperature, wind speed and/or wind direction, precipitation, and humidity), but certainty or uncertainty as to timing, duration, and magnitude.

While those who tend to be proactive consumers consistently are probably a numerical minority of the general population, possibly a small minority, it should be considered that a much larger population segment is likely to act as proactive consumers on a more occasional basis, when they have a weather dependent activity planned, but fall into the passive category otherwise. Hence, we conclude that there is a sufficiently large consumer need for weather communications that empower consumers to keep their own counsel, and a sufficiently large benefit — to the quality of consumer decision-making, and to the credibility of NWS as an information source — to make development of prototypes of such products worthwhile.

3. PHASE II METHODS AND RESULTS

The second phase of this project involved the development of prototype forecast products that would respond to the needs for weather/forecast information expressed in focus group interviews. Based on focus group interviews in the pilot study and first phase of this project, we considered that these prototypes had to meet at least four conditions, and that it would require graphical products to meet them:

• They have to stimulate consumers to make their own judgments and assessments of the potential for particular weather conditions of interest to occur, and prevent consumers from adopting "someone else's" forecast.
• They cannot require consumers to have or to develop technical, meteorological, expertise in order to utilize them.
• They cannot require a large investment of time to comprehend.
• To maximize their simplicity, and maximize their utility for consumers, whose planning is contingent on specific weather conditions rather than a composite, separate forecast products are needed for temperature, precipitation, wind, and humidity.
• Products need only be in two dimensions to simplify consumer understanding.

Procedure

The plan for this phase was for Sanders and Westergard to collaborate with Richard Jesuroga of the Forecast Systems Lab to create prototype forecast products to meet the goals identified above. Before meeting with Mr. Jesuroga, however, Dr. Sanders and Mr. Westergard felt it important to develop some preliminary talking points, and explored possible solutions using AWIPS resources.

The graphical depiction on AWIPS of model forecasts for temperature, wind, precipitation, and the like, seemed to be a promising point of departure. Being able to "see" wind conditions (speed and direction) or temperature conditions as moving fields with defined boundaries, direction, and speed, was certainly more informative than what most consumers can learn from generally available forecast products. It would not require additional technical expertise to comprehend and utilize such information. And such information lends itself to making one's own decisions, given one's ability to judge from the size, speed, and timing of a particular field when and for what duration one would be affected.

Giving consumers access to model forecasts seemed to us a means of achieving our two primary goals:

• Providing consumers with tools with which to assess risk and make plans and decisions would be met by giving them access to graphic displays of two or three different model forecasts. The extent to which model forecasts converged as to the occurrence, duration, timing, and magnitude of specific conditions in a specific location could be regarded as an indication of relative certainty about those conditions. The extent to which the model forecasts diverged would provide consumers with an indication of relative uncertainty (as to an event's occurrence, timing, duration, and/or magnitude), as well as the likeliest range of possibilities (the early versus later possible times of onset, the lower or higher possible magnitudes, the shorter or longer possible durations). We felt that the ideal would be to overlay graphical displays of forecast models so that consumers would be exposed to a single screen to points of divergence and convergence in the model forecasts.
  
• Being responsive to the proactive consumers' tendency to " keep their own counsel" and to want forecast products that are unmediated — "unbiased" and "uninterrupted" by forecasters.

It turned out that Jesuroga and his team had already been working for some time on "dissemination" graphics compatible with our thinking. Jesuroga and his team had concentrated on the needs of emergency managers, and state and local officials, but intended also to eventually make such products accessible to the general public. Their goals, and the software and graphics they had already developed, were quite consistent with our own thinking.

However, there were also differences in our two approaches because of differences in our projected consumers — between the needs of emergency managers and state and local officials on one hand, and the commercial and recreational interests of our proactive informants on the other. Once adverse weather occurs, it is too late for action by proactive consumers, but is the time for action by emergency managers and civil officials. Hence, emergency managers and civil officials take special actions "in response to current weather conditions, and are most concerned with recent trends and current conditions. In contrast, proactive consumers with commercial or recreational interests want to minimize and delay special actions as much as they can, and want to make decisions about whether special actions are needed well in advance of the occurrence of adverse weather.

Hence, in contrast with our interest in model data 24-72 hours in advance, Jesuroga had been relying on observational data over the prior 12 hours to the present time; and because he was concentrating on observational data, Jesuroga and his team had not worked out how to depict the convergences and divergences among alternate forecast models, as we envisioned.

In collaboration with the FSL team, we were able to come up with a simplified suite of computer model-based graphics which approximately satisfies the information and decision needs that our proactive users articulated. We believe this suite will allow nonmeteorologists to see the three parameters of most interest — temperature, wind, and precipitation — as predicted by the AVN and NGM models. That suite of products is available in raw form, and the team at FSL showed us preliminary examples of what the resulting graphics might look like. We also devised a way of overlaying these models to represent on a single display their convergences and divergences.

While the products described above are an excellent first step, we still would like to see graphics developed to allow forecasters to present a primary forecast with multiple secondary, less probable, forecasts. A demonstration provided during our visit of the new Interactive Forecast Processing System (IFPS), currently being fielded at NWS forecast offices, left us enthusiastic about the NWS's future ability to refine our graphical forecast products to best serve our proactive users.

Results

On the basis of experimenting with different methods of display at the Forecast Systems Lab, the initial prototype we developed was a display of two forecast models, using contours to differentiate fields. By using contours, it was possible to overlay forecast models, with the contours of each model a distinctive color (e.g., yellow contours for the AVN model, and green contours for the NGM model). This was also compatible with the software and display options FSL had independently developed. The displays were limited to two forecast models because displaying three or more overlaid models created such visual clutter that we were confident it would be outside the capability or patience of consumers to work through the display.

At the same time, there was some concern that using contours to distinguish fields is not as comprehendible by inexperienced consumers or as familiar as color fields are, especially when models are overlaid. However, no solution was apparent at the time as to how color fields for different models could be overlaid without losing the distinction between the models within any given field. The alternative would be to make forecast models available, displayed in color fields as separate screens, with consumers able to switch back and forth between them.

The prototype products developed at FSL to be shown to focus groups in Phase 3 were thus of two kinds:

1. Displays in contours of two overlaid forecast models, capable of being looped, or examined frame by frame, out to 48 hours, displayed on a regional scale, and on a state scale to indicate the potential for zooming in or out.

2. Displays in color fields of two forecast models on separate screens, capable of being looped, or examined frame by frame, out to 48 hours, displayed on a regional scale, and on a state scale to indicate the potential for zooming in or out.

3. a. values in temperature fields were expressed in degrees Fahrenheit
   b. values in precipitation fields were expressed in hundredths of an inch (rain) or inches (Snow)
   c. values in wind fields were expressed in knots of wind speed, with vector arrows to indicate direction and speed

During sessions at FSL, we also discussed the potential for and value of including in these products text produced by forecasters providing analysis in lay terms regarding (a) atmospheric and topological contingencies that might influence forecast conditions in terms of intensity, timing, and the like, and (b) reasons to be more or less confident about one or the other of the model forecasts. We decided, however, that we would not include text in our prototypes, both to find out whether consumers expressed a desire for some such aids, and to keep the focus group interviews focused as much as possible on the value of the prototype graphics.

However, after returning from our meetings at FSL, we created a third prototype to be presented to informants for the sake of providing informants; with more choices, for the purpose of stimulating more informative discussion about the usefulness and understandability of these prototypes. The third prototype we devised was a representation in the form of line graphs of model forecast conditions (whether for wind, temperature, precipitation, etc.) at a specific geographical place. Forecast models were differentiated by giving each line on the graph, representing a different model, a different color.

Examples of each of the prototype graphic displays of forecast models are provided in the Appendix.

4. PHASE III METHODS AND RESULTS

The third phase of this project was intended to elicit feedback about the prototype forecast models we had developed from the same cross section of consumers as in Phase I. Before seeking assessments of and feedback about these prototypes from informants in Phase III, we first pilot-tested them, and an interview protocol on three focus groups in the Albany, NY area: a group of farmers, a group of recreational boaters, and a group of university students. Those interviews did not indicate that there were any problems with the displays we would be showing that required modifying the prototypes, except for issues of readability because of font size or color contrast that we attempted to remedy.

Based on our work at FSL, our original intention was to get feedback on prototypes that would be accessed via the Internet from each interview site on an FSL-created web site. This would give us that focus group members would see model forecasts for their region that were current, and would be able to see how the graphics could be manipulated by users (e.g., zooming in, looping, and so forth). However, as a backup, we also created simulated versions of these prototypes, taken from AWIPS resources, which were stored as files that the focus group leader carried with him. It turns out that we ended up using these simulated versions of the prototypes for all groups because (a) despite considerable efforts by local programmers, we could not get the software provided by FSL to function for us on our local sever as we needed it to, and (b) it became increasingly uncertain that we could count on Internet access in many of the sites at which focus group interviews were scheduled.

Procedure

We returned to the same four cities that were research sites in Phase I of this project (Miami, FL; Cincinnati, OH; Manhattan, KS; Bozeman, MT, and interviewed informants from the same four constituencies as in the first study (agricultural, building/contracting, recreational, and PTO). Because we did not find any indication of differences across each pair of PTO groups in Phase I, we only interviewed 1 PTO group in this third phase.

Focus groups were formed, as in Phase 1, by contacting organizations that represent particular constituencies, and asking them to recruit members in return for a monetary contribution to the organization. Hence, cooperative extension organizations were contacted for help in recruiting farmers, growers, and ranchers; builders' associations were contacted for help in recruiting builders; recreation clubs (boaters, golfers, etc.) were contacted for help in recruiting informants with recreational interests; and the parent-teacher organizations of schools in the K-8 range were contacted for help in recruiting their members.

It was not of concern one way or the other whether we interviewed the same informants in Phase III as we had in Phase I. The issues for informants to discuss in Phase III were entirely different from those they discussed in Phase I, and it seemed unlikely that discussion would be biased or contaminated depending on whether informants had taken part before or not. However, to the extent to which we can tell from what was said in the transcribed Phase III interviews, it does not seem that many informants took part in Phase I interviews also.

The focus group interviews in Phase III had two parts. The first part was "educational" about these prototype products rather than being an interview. Each of the three types of graphic display was presented and explained, and then informants' questions about them were answered. The second part was an interview that centered on three issues: (a) whether these forecast products would be useful, and if so, why and how, (b) whether informants would be willing to make use of the Internet to get access to them; (c) what feedback and suggestions informants had about the contents and understandability of each graphic, and how they could be improved.

It is worth noting that questions our informants asked during the "educational" portion of each focus group interview tended to be about two topics more than any other. One set of questions was about model forecasts, specifically about how they are produced. The second set was about the role of forecasters. It seems clear that the 20-30 minutes spent on "education" had a clear, and arguably a desirable, effect on helping informants understand (a) the "science" that goes into producing model forecasts, (b) what real-time data contribute to producing model forecasts, and (c) what the work and the role of the forecaster is. On the latter point, most informants as a result of this session came to understand forecasters as making expert judgments about the relative merits of specific model forecasts (as opposed to, say, guessing what might happen locally based on what events were being reported at other stations further west). It was an unexpected byproduct of this research to find that making these prototypes available can provide a relatively brief and accessible way to educate the public about how to understand and evaluate weather information and products.

Findings

Overall Assessment of the Prototype Graphics

To begin with the findings of most direct relevance, we found that proactive consumers (referring now primarily to such users as farmers and contractors, and many of those with recreational interests) had a favorable overall response to these prototype products, with various qualifications and exceptions on which we will elaborate below. At the same time, they indicated that they would use them in conjunction with currently available forecast products, not as an alternative to them.

In brief, all consumers indicated that they find the currently available forecaster predictions to be valuable and that these would continue to be their initial source of weather information and forecasts. But consumers also indicated that they would consult and value the kinds of tools for assessing risk that overlaid model forecasts do give them when (a) the standard forecast predicts "adverse conditions" (relative to the individual consumer's plans and activities), and (b) those adverse conditions would have a financial cost, or personal cost of some kind, depending on whether they occur, when, with what magnitude, or what duration.

Of the three types of graphic they were shown, informants consistently regarded forecast maps with overlaid contours to be the hardest to read and use. They found the line graphs displaying overlaid model forecasts for a locality to be the easiest to comprehend. However, they found these less informative than forecast maps with color fields. Although only one informant explicitly commented on having both types of graphic available (line graphs, and maps with color fields), we infer there would be the following predominant patterns of use. For those intending to travel from their present location, the forecast maps would be utilized instead of line graphs for localities. For those staying in their present location, there would be two modes of use. (1) Line graphs would be consulted to get an initial overview of weather trends, the potential for adverse weather, and the evident congruence or divergence among models; in the event of a potential for adverse weather and a degree of divergence between models, users would turn to forecast maps to study them for the size of fields, and their predicted movement and timing. (2) Forecast maps would be consulted first to get an initial overview of regional model forecast trends /dynamics, and then a line graph would be consulted (preferably accessed by "clicking" on a locale on the map) for a clear and comprehensible summary of the models' local forecasts.

On the Issue of Usefulness

With few exceptions, our informants positively evaluated the usefulness of these prototypes. The sentiment among most informants was that these greatly expanded the amount of information consumers would have available, and thereby would enrich and improve their basis for making weather-related plans. Not surprisingly, the most consistent positive response came from informants with commercial interests in weather conditions (farmers, builders), a more qualified but still generally positive evaluation from informants with recreational interests, and the most qualified positive responses from informants without a consistent or definer interest in weather conditions (members of Parent-Teacher Organizations). These latter tended to take the position that while they personally would have limited need for such products, they could imagine others who would benefit from having access, such as farmers. This range of evaluations from strongly to modestly positive also tends to parallel the distribution of informants from proactive to passive.

On the Issue of Frequency of Use

While informants had a range of predictions about how regularly they would make use of the kinds of forecast products we presented to them, there was general agreement that they would be consulted intermittently, not on a daily basis. Farmers might consult them in advance of a decision to spray their crops; builders might consult them in advance of a decision about whether to pour concrete; parents might consult them in advance of a decision of when, where, or whether to go on a family outing. There was general agreement that these products would be consulted and be useful whenever a consumer had to make consequential plans and decisions that an adverse turn in the weather could jeopardize (consequential in the sense of having financial or personal costs if the plans fell through). The rest of the time informants indicated that they would rely on the kind of bottom line, conclusion-centered forecast products currently available, valuing them for simplicity, ease of access, and speed of comprehension — and as alerting them to the possibility of adverse conditions that would lead them to consult the kinds of forecast products we were showing them.

On Informants' Readiness to Use the Internet to Gain Access to Such Products

The pattern of responses on this point is relatively simple. Those who already have Internet access indicated that they would welcome having these prototype graphics available to them. Those who do not currently have Internet access (especially those who have not opted to get access even when it is available to them locally) indicated that they would not change their media-use habits in order to gain access to these products.

One informant in particular made an observation whose substance is important. She noted that it takes time to study these graphics, to discern what the potential weather conditions of interest might be. She accompanied this with the comment that consumers need to be able to "stop" images, and move back and forth, at will, and not have that under anyone else's control. Accordingly, she emphasized that the Internet, not television, is the only suitable medium for these products.

The responses from informants on this point thus lead to the conclusion that these products are most suited to an Internet environment. However, it will be increased adoption of Internet technologies that leads consumers to these products. We cannot expect that being able to access these products will lead consumers to adopt Internet technology.

Informants' Advice about Improving the Prototype Graphics

The majority found the use of color fields to display model forecast data the easiest to comprehend and the most informative, even though they were told that at present we do not have a way to overlay alternate models of that kind.

Some informants preferred having model forecasts displayed as line graphs (though a number of informants indicated that bar graphs would be preferable) as long as one were not traveling and were concerned with local conditions only. They regarded them as most easily and quickly comprehended. However, there was disagreement on this point from a number of informants who also expressed an interest in local conditions alone. This latter group still favored a display in terms of color fields because they felt such displays are more informative about the dynamics in time and space of particular weather conditions (e.g., the size of a particular field, the speed of its movement, whether they were located near a boundary or a center, etc.).

Individual informants made suggestions for improving these products, especially displays of forecast models in color fields. Their suggestions follow, in no particular order.

• It is desirable to find away to overlay color fields from different models. One informant suggested a way of doing this: distinguish the models not by color, but by ways of filling in fields, where one model might fill in a field with vertical bars, and another model might fill a field with horizontal or diagonal bars.
• Include icons in color fields indicative of types of precipitation and severity of the kind one sees in some forecast graphics on television (lightning bolts, snowflakes vs. rain drops, etc.).
• When providing forecast data for precipitation, adjust units indicating amounts in a field in a particular period of time depending on whether precipitation will be rain or snow.
• Provide means of zooming in to the county level and out to at least a regional level.
• Include selected city names on maps to provide consumers with ways of easily locating themselves.
• Provide the capability to click on any geographical location on a map, and have a line graph or bar graph of overlaid forecast model data for that place displayed.
• Add forecast model data for jet stream tracks and pressure systems and fronts.
• Make time displays easily visible and comprehensible, especially when frames are looped. One informant suggested adding a bar across the top or bottom of the screen that varies from yellow to black and back again with the passage of time from day to night.

Negative Reactions

There were also some negative evaluations of these prototype products. They were expressed by relatively few informants, and were scattered across the different groups of informants we interviewed rather than clustering in particular groups. There were two primary, and two less prominent, reasons for negative evaluations of their usefulness.

1. Some negative evaluations were that the prototype graphics did not seem to provide anything more than is already available from television or Internet sources. These remarks indicate that for some informants, it is more difficult than for most to distinguish the (intended) uses of these prototypes from uses of the forecast graphics currently available on television and the Internet. Instead of recognizing that these prototype graphics make alternative model forecasts available, so the consumer can compare them to make his or her own assessments of risk these informants seemed to think of these prototypes as simply providing forecasts (predictions).

2. The second most common reason for negative evaluations was the amount of time and effort it could take to examine these prototype graphics, and come to a conclusion about the likelihood or risk of adverse weather conditions. Some informants felt they could not take the time, others felt that they would not have the opportunity to access the Internet often enough or at the times for making relevant plans and decisions.

3. A third negative evaluation, provided by only one informant (a boater in Miami) is that the prototype graphics are as vague as current forecast products about the conditions of interest to be expected in very specific locations, the likely timing and duration of weather events, and so on.

4. Finally, one other informant (a grower from the Miami area) raised an issue that (as we discuss) may be a critical one. He noted that graphics of the size and complexity of the ones we have devised are very slow to load and manipulate through any Internet connection by modem and phone line. He predicted that this would discourage many potential users.

5. SOFTWARE DEVELOPMENT AND IMPLEMENTATION ISSUES

In this section, we identify specific steps that need to be taken to bring the prototype forecast products we tested to a user-ready condition, and the specific steps needed within NWS to implement and utilize them.

Software Development

The suggestions and evaluations of our informants make clear that these forecast products need to have specific features that necessitate upgrading and adding to current software. We have restated our informants' comments below in the form of specific goals and software modifications a programmer needs to address:

• make graphic displays visually meaningful at a glance:
  - color fields for forecast maps displayed in such a way as to overlay contrasting models
  - icons within color fields indicative of precipitation types, storm conditions, etc.
  - bar graphs for overlaid forecast models
• display time lines clearly, readable while also examining map
• make products user controllable in regard to:
  - zoom-in capability down to county level
  - split screen capability to display either alternate models, or forecast maps and graphs of overlaid model forecasts
  - overlay model forecast maps or display each model separately
  - click on point on model forecast map to view graph of model forecasts for that place
  - select page by page view versus loop; select speed of loop
• maximize speed with which web pages load by minimizing file size and maximizing program efficiency

Given that there is a good software foundation already in FSL and NWS products to build on, we anticipate that only a modest amount of programmer effort is needed to achieve these goals. Reasons for NWS to make this investment will be discussed in the concluding discussion (Section 6).

Implementation Issues

The most significant implementation issue that these forecast products involve is the change they necessitate for both forecasters and consumers in the way they think about "the message" — about what NWS's products are intended to "tell" consumers. In accord with this change in thinking about "the message," we will also discuss specific operational changes that providing these forecast products require, including the production of text for weather radio and other text-based media.

Before getting into matters more specifically, it may help to contrast recent innovations in NWS forecast products for the public with what we are recommending. As a case in point, NWS sites in the Eastern Region have recently added a graphical version of the forecast to their web sites. The new forecast provides iconographic representations of a timeline for a 36-hour period for forecast temperature, humidity, wind and precipitation events, accompanied by a representation of the probability of precipitation for each 12-hour period. An extended forecast in iconographic form follows for subsequent days and nights, without timelines. These iconographic products are clear, and likely more accessible and informative than text products.

However, while this new graphical forecast product seems to move substantially in the direction our research has considered, there are important — essential — differences.

• Unlike our prototypes, the new products continue the current practice of providing forecaster conclusions to the public, without a representation of the range of potential variability that is comprehended by the models underlying the forecast. This conceals relevant information from the public that would facilitate individual planning.
• Unlike our prototypes, the new products do not make available geographical representations of fields nor display the rate and direction and movement, or relative size of fields, over time. Hence, they do not enable a consumer to judge whether he or she is located near the boundary of a field or in its center; where he or she will be located with reference to a field's boundaries over time; or what potential variability the speed, size, and location of particular fields there is. Again, this conceals relevant information that would facilitate individual planning.
• In striving to be as clear, simple, and uncomplicated as possible, these products fail to empower forecast consumers. Instead they continue making consumers fully dependent on forecaster judgments, and continue making forecasters seem fully responsible, and in error each time the actual weather does not bear out a consumer's weather-related planning based on the forecast.

On the other hand, the IFPS concept is a strong step in the right direction, requiring only minimal additional effort to achieve the goals we are suggesting.

• IFPS requires a change in how the forecaster perceives the forecast process, which is an ideal beginning to the changes we are proposing. To make the most of IFPS, forecasters must begin to think of the forecast in terms of a gridded, four-dimensional field of data for each of the various parameters. We suggest they could quite easily go the extra step, and perceive each of those data fields as representing their best judgment of the most likely course of development, bracketed by an array of less likely possibilities. Given that perception, along with an understanding of the needs of the proactive consumers we found, forecasters should have no trouble producing Area Forecast Discussions in the format proposed elsewhere in this report.
  
  
• If implemented well, IFPS could readily produce textual forecasts in the parameter-based format proposed elsewhere in this report. In addition, computer model forecasts are already available in comparable gridded field formats, so the automated production of the full range of products we propose would be a logical progression from IFPS implementation.

Re-Educating Forecasters and Consumers

The current understanding of "the forecast message" is that forecasters "tell" consumers what the weather is, and what it is going to be, as a matter of scientific fact. But forecasts are not a matter of scientific fact, they are theoretical projections coupled with forecaster experience and judgment — the result of computing what future conditions and events in a location are likely to be based on forecaster evaluation of current conditions and assessment of various computer models of the atmosphere.

The main negative consequence of thinking about forecasts as a matter of "telling" consumers what the weather is going to be is that it creates a potential gulf between forecasters and consumers. It leads forecasters to think of consumers as passive and helpless, dependent on weather forecasters alone. This is likely to make forecasters feel consumers cannot be trusted and should not be empowered — that it is entirely the forecasters' responsibility what consumers think and how they respond to forecast conditions, based on what information the forecasters provide and how they elaborate and conceptualized that information. But forecasters cannot meet that level of responsibility unless they provide consumers with exactly the information and the detail they need, and can guarantee their correctness. This is not possible, not technically and not practically (forecasts in sufficient detail would be indefinitely long and hard to comprehend). In addition, such a stance by forecasters runs against the desire of consumers, proactive ones at least, to keep their own counsel. By pursuing that course, forecasters invite consumers to evaluate forecasts and forecasters as right or wrong, accurate or unreliable. That, in turn, leads any consumer who evaluates forecasts as always being potentially fallible to either ignore forecasts and strive to be prepared for any turn to unofficial, mostly unscientific, sources of "prediction" in order to improve their personal weather-related decision-making and planning.

Implementing forecast products of the kind we have devised and tested with consumers requires that this way of thinking be changed among both forecasters and consumers. Instead of "telling" consumers what the weather will be, the goal is to give nontechnical consumers as much information as possible about possibilities and contingencies, e.g., the extent of divergence and convergence among two or more model forecasts, as well as a representation of the dynamics and geographical coverage of forecast weather events — to enable the consumers to judge whether there is a potential for conditions to occur that, in reference to their specific situation, would necessitate making or changing weather-related plans and decisions.

If forecasters understand that their goal is to help consumers make such judgments, and if consumers for their part understand that NWS products are available to help them do that, there will be two positive results. (1) As long as forecasters feel entirely responsible for what consumers know and how they respond, the tendency is to tell consumers less, to withhold qualifications and complexities from consumers that might undermine the forecasters' authority and distract the consumer from a focus on matters that the forecaster has judged consumers should focus on. If that sense of "responsibility" and "control" is removed, then forecasters will be more open to maximizing the information consumers can obtain, making weather information and forecasts maximally applicable across diverse decision needs of consumers in diverse activities and situations. (2) Correspondingly, if consumers consider NWS products as sources of information on which they can base their own assessments of risk, they will no longer evaluate NWS products as right or wrong — instead they will evaluate them as more or less informative and more or less useful. Developing a consumer focus on forecast products' usefulness (decision relevance) and informativeness — not just their scientific accuracy — is as important for NWS as it is for consumers. Currently, consumers judge forecasts that are simply not informative enough for their purposes to just be inaccurate.

However, achieving this shift in the relationship between forecaster and consumer depends on "educating" forecasters to stop thinking that it is their responsibility to "control" what consumers know and how they respond to weather information and forecasts, and promote thinking about how to give consumers as much information as possible to facilitate their personal planning and decision-making. It was objected in one meeting we had with NWS management personnel that forecast products like our prototypes would give consumers "too much information," and that this would either confuse consumers or increase the likelihood that they would misuse the information. We have already given reasons why we regard this way of thinking about consumers and about NWS products as counter-productive.

It is clear from our focus group interviews, however, that consumers also will need to be "educated" to stop thinking about forecast products of the kind we devised as one more way of "telling" them what the weather is going to be. It took 20-30 minutes of presentation and explanation in our focus group interviews to get that idea across to informants. While most did comprehend and embrace this shift, a few did not get it (for example the one comment we had that the informant felt he was already getting the information our graphics provide from forecast maps on cable television).

Indeed, what was said and implied by our informants in several instances is that what they most want (including those who understood that these products empower them and facilitate their ability to plan and make decisions) is what NWS and other information sources are currently trying to give them — simply to be told what the weather will be, with perfect accuracy, sufficiently far in advance that they can plan adequately without having to ponder contingencies. But current science does not quite enable perfect accuracy, and even if it did, it would be impossible to make forecasts sufficiently detailed that they would meet the widely divergent planning/ decision needs of a mass audience.

Consumer education can be achieved in three ways. First, a public information campaign can be launched to alert consumers to the NWS intention to provide them with decision-making tools, rather than bottom line predictions, along with directions for accessing these new products on the Internet. Second, tutorials for using these tools can be included on the web sites at which these new products are available. Third, protocols for forecaster texts (especially on weather radio, as discussed below) can be changed so that they provide information regarding model forecast divergences and convergences, not just predictions. This last item requires attention to forecaster education. (In case we seem to be contradicting ourselves, note that the consumer education project called for here is entirely different from the one we discussed in the overview section of increasing NWS credibility by educating consumers about scientific improvements in the forecast process. The goal here is to attract consumers to, and teach them to use, a new decision-making tool, that would guide and facilitate their efforts to second-guess official forecasts. The goal of the consumer education strategy we previously discussed and were skeptical of was to attempt to change consumer beliefs and values so they would stop trying to second-guess official forecasts.)

Forecaster education can be achieved primarily by providing forecasters with new tasks and new protocols. In accomplishing these new tasks, forecasters will be led to rethink their goal instead of trying to produce a maximally accurate forecast, and enhance its credibility by not promoting consumer access to underlying uncertainties and contingencies, the forecaster's role will be to provide consumers access to precisely any such uncertainties and contingencies, and to position their forecasts as judgments within the context of divergent models.

The first task for forecasters in this regard is to select two models for the current period that will be accessible to consumers through the forecast products we have devised and tested. The forecaster's task in this regard will be to select those two models that show, in the forecaster's judgment, the greatest plausible range of potential conditions and events — not the models that most closely correspond to and corroborate the forecaster's own predictions.

The second task for the forecaster will be to produce text (for distribution through weather radio, cable television feeds, etc) that functions in the way that the forecast products we have devised are intended to function — to provide consumers with the ability to judge the degree of certainty and uncertainty regarding forecast conditions and events, and make their own assessments of risk. This would involve a slight modification of the current protocol for producing forecast text. The first part of a text would consist of the forecaster's predictions (but organized as separate wind, temperature, precipitation, and humidity forecasts, rather than composite forecasts of conditions for the day, night, etc); the second part would be to summarize where different model forecasts converge and diverge, with brief explanations and analyses of changes in the current as opposed to prior forecasts. An example of the first part of such text is:

ZONE FORECAST:
For the Lake George/Saratoga region,
Wind: wind is likely to be from the southwest today and tonight, becoming west tomorrow morning, and northwesterly in the afternoon. Wind speeds are expected to moderate today, between 8 and 12 miles per hour, diminish overnight, and then increase tomorrow from about 10 miles per hour in the morning to 20 miles per hour with gusts by early afternoon.
Temperatures: it is likely to be in the mid-sixties today, cooler in higher elevations, dropping to the upper 40s overnight; temperatures tomorrow are expected to rise to near 70 by midday but then drop slowly during the afternoon to the high 50s by nightfall.
Precipitation: Light rain, with amounts under a tenth of an inch, will likely fall periodically this afternoon in most locations, ending by dark; cloudy conditions tomorrow morning are expected to give way to clear skies by afternoon.

Note that this format will require software changes in the Interactive Forecast Preparation System (IFPS), in addition to the change in forecaster thought processes. IFPS changes would be needed to focus on parameters, versus time. On the other hand, these changes would be a logical extension of the already anticipated effort to shift forecaster focus to creating graphical fields of parameters, which are then made available in a digital database, as opposed to focusing on creating a text. Note that the language used in forecasts should not include the predictive term "will" as in "X will occur." Qualifiers that imply forecaster judgment have to be used throughout (it is possible," "it is expected," "it is likely"). While such mitigation would make forecasts in isolation seem so weak and over-qualified as to invite distrust, our proposal is that forecasts always be accompanied by a description of the model forecasts that helps consumers understand what the uncertainties in the forecast are based on.

An example of the second part of a forecast text, then, corresponding to what we propose consumers be able to access graphically, is:

AREA FORECAST DISCUSSION:

Today's forecast models of choice are the nested grid and aviation models. The models generally agree on wind speed and direction today and tonight, but they diverge on wind speed tomorrow afternoon, which could range from 15 miles per hour in one model to over 25 in others. The models generally agree on forecast temperatures, but differ in how quickly cooling will begin tomorrow, with one model forecasting a temperature drop to begin before noon and the other forecasting steady temperatures until late afternoon. The models disagree slightly on rain amounts to be expected this afternoon, but both forecast less than one-tenth inch.

The nested grid model has changed since yesterday, and now projects a less intense low-pressure system will be entering the region. It thus now forecasts lower precipitation amounts for this afternoon than it was forecasting yesterday, which brings it more in line with the aviation model. The wind speed and temperature differences tomorrow between the models results from a difference in their predictions of how fast the cool air mass behind this system will spread into the region.

In writing such texts, forecasters will be led to change their perception of the audience they are writing for, and their purpose. First, forecasters will have to become concerned with the way they describe physical processes, to be sure their language will seem clear and direct to consumers. For example, in beginning the second paragraph of the "area forecast discussion" above, we at first produced a sentence that we decided was too convoluted for many consumers, and replaced it:

Sentence 1: "The nested grid model has backed off on the deepening of the low pressure system entering the region."

Sentence 2: "The nested grid model has changed since yesterday, and now projects a less intense low pressure system will be entering the region."

In writing such texts, forecasters will also need to understand that the area forecast discussion is no longer intended as a coordination tool between meteorologists, using jargon and abbreviations not readily understood by consumers, aimed at explaining their forecasts. Instead, their area forecast discussion will need to be informative to consumers about the range of possibilities within which the forecast was produced, describing model differences in such a way as to convey sources of certainty and uncertainty, so as to enable consumers to assess potential risk

In summary, in the event of NWS acceptance of the goal of empowering forecast consumers via the recommendations in this report, we propose a series of steps be taken. Taken in order, with deliberation, these steps will move us from the current emphasis on conclusion-based text forecasts to a suite of products which enable the individual consumer to make informed judgments of certainty and uncertainty regarding particular conditions and events that constitute a risk for that individual.

1. Centrally develop the software to produce the graphics and text products described in this report in a routine, automated fashion. This would include modifying IFPS software to add the recommended text products to the suite of output formats.

2. Provide forecasters with a computer based learning course to acquaint them with what has been learned about how people use weather information, and the ways we can improve our forecast message.

3. Choose a group of offices, perhaps those serving a selected state, willing to field test the concepts we propose, and begin routinely creating these products, in tandem with current formats.

4. At those field test sites, with nationally supported software expertise, develop an online tutorial, working with local weather information consumers, to make accessing and understanding the new products as efficient and user friendly as possible.

5. Deploy the new software across a Weather Service region, and change the area forecast discussion format to reflect this report's recommendation.

6. When consumer/forecaster acceptance reaches an appropriate level, pick a date for the chosen Weather Service region to replace the current zone forecast format with the one recommended in this report.

As noted, these implementation steps internally need to be accompanied by a consumer education initiative. This has two parts, both to begin at the same time new graphic products are put online, and new protocols for forecast text are adopted:

1. Launch a public information campaign through public service announcements on radio and television to alert consumers to the NWS intention to provide them with decision-making tools, not just bottom-line predictions, along with directions for accessing these new products on the Internet.

2. Devise tutorials for using these tools, to be included on the web sites at which these new products are available.

6. SUMMARY AND CONCLUDING DISCUSSION

Empowering consumers has the potential to (a) make NWS and its products more credible, and (b) thereby improve NWS effectiveness in emergency weather situations.

We expect that producing such products will make NWS more credible because the consumers' yardstick for evaluating NWS products will shift from the accuracy and reliability of predictions, to the usefulness and informativeness of forecast information. Currently, conclusion-based NWS forecast predictions cannot avoid seeming more unreliable and inaccurate than they actually are, in that they can never be specific and comprehensive enough to be "correct" for all the diverse plans and decisions diverse consumers make across diverse situations and activities. But with our suggested changes, NWS products can achieve increasingly better usefulness and informativeness for making decisions and plans, even if scientific accuracy remains constant, and on that basis improve the credibility of the NWS.

We expect that investing in such forecast products will make NWS more effective in weather emergencies for two reasons. First, NWS will have become more credible, and consumers will have learned to rely on its products more, if everyday forecast products have been devised to empower consumers to make their own assessments of risk Second, consumers will be more reliant on and more responsive to NWS emergency advisories and warnings if changes in forecast products on an everyday basis carry over to forecast products during emergency weather situations, to help consumers make judgments, plans and decisions, not simply to "tell" them what will happen and what to do. Hurricane strike probability graphics are an excellent prototype.

The practical consideration we are left with is whether there will be a sufficiently large market for these products to justify investing in their development and dissemination. Our data indicate that there will be a market, but not how large it will be. We anticipate that the market for these new products would grow as they became familiar. As in the case of any technological advance, once these products are available, we can expect them to find a niche, and to attract a growing number of users who will become progressively more sophisticated and more proactive in their use of NWS products. Unfortunately, we think it would be difficult to empirically demonstrate this. First, our informants led us to expect wide variation in how frequently they would make use of these products. Second, to estimate market now would be handicapped by consumers' absence of familiarity and experience with these products; to estimate market size once these products are available raises the issue of how often does a consumer have to use these products to be counted. While some index of market size could nonetheless be produced, it would be expensive to do it, and it would delay moving forward. But we think that it is not a question of market, but a question of their relevance to the NWS mission, that should be the key.

On the question of mission, NWS currently makes raw data and model forecasts available in a form that is valuable to professional forecasters and commercial vendors but not suited to the non-technical consumer. NWS products for the non-technical consumer consist of "bottom-line" predictions from which the input complexities and judgments are withheld. This prevents consumers from making use of the much richer information that forecasters have available, and evidently forces proactive consumers who have to find a basis for assessing risk to turn away from official sources of weather information and rely on various non-scientific indicators. At first this does not seem to be a problem. NWS's mission does not apply to supporting consumer needs directly on an everyday basis, but only in the case of emergency weather situations. Our question is whether the two are separable. Learning to utilize nonscientific sources to second-guess forecasts is not a good habit to apply in emergency situations, but neither consumers nor forecasters are prepared on a daily basis for anything else. Hence, while it would require a change in current practice, and in forecaster thinking about audience and goals, we contend that to provide consumers with forecast products that empower them to make their own assessments of risk is consonant with the NWS mission.

REFERENCES

Adams, C. K. (1995). National Weather Service Emergency Management Forum: Building Better Warning Partnerships. Silver Springs, Maryland, U.S. Dept. of Commerce, National Oceanic and Atmospheric Administration, National Weather Service.

Carter, M.T. (1980). Community Warning Systems: The Relationships Among the Broadcast Media, Emergency Service Agencies, and the National Weather Service. In Committee on Disasters and the Mass Media, Disasters and the Mass Media (pp. 214228). Washington, D.C.: National Academy of Sciences.

Lelk, R.K. Carter, T.M. & Clark, J.P. et al. (1981). Community Response to National Hazard Warnings: Final Report. Minneapolis, MN: University of Minnesota.

McLuckie, B.F.(1974). Warning: A Call to Action. Washington, D.C.: National Weather Service.

Mileti, D.S. & Sorenson, J.H. (1990). Communication of Emergency Public Warnings. Technical Report, National Technical Information Service, U.S. Department of Commerce, Springfield, VA.

Sanders, R.E. & Westergard, R. (1996). The Public Relations/Education Function of Meteorological Communication: A Study of Citizens' Understandings and Uses of Weather Information and Forecasts and Recommendations for Improving Them. Final Report of Research Conducted as a Partner's Project, under the COMET Program, University Corporation for Atmospheric Research, Project no. S-9554367

APPENDIX

Examples of Prototype Graphics Depicting Model Forecasts for
Temperature, Precipitation, and Wind in the State of Florida and the City of Miami

Figure 1 - Temperature Contours & Color Fill: AVN (single model) Model Forecast

Figure 2 - Temperature Contours: AVN vs. NGM Model Forecast

Figure 3 - Temperature Line Graph: AVN vs. NGM Models vs. Forecaster for Miami

Figure 4 - Precipitation Amounts: AVN (single model) Model Forecast

Figure 5 - Precipitation Amounts: AVN vs. NGM Model Forecast

Figure 6 - Precipitation Amounts: AVN vs. NGM Models vs. Forecaster for Miami

Figure 7 - Wind Speed Contours & Color Fill and Directional Arrows: AVN (single model) Model Forecast

Figure 8 - Wind Speed Contours and Directional Arrows: AVN vs. NGM Model Forecasts

Figure 9 - Wind Speed Line Graph and Direction: AVN vs. NGM Models vs. Forecaster for Miami

*The graphics in Figures 1-6 were prepared in December 2000 to show to focus groups in January 2001. However, the file of graphics for wind speed that we showed to focus groups was lost: Figures 7-9 are new graphics for purposes of illustration that were created in August 2001 to include in this report.