How to Measure the Progress and Value of NMME? Issues to Address: Need to define how to measure the value of NMME. Besides CPC, who are other users of NMME? What is the process to engage users? Presentation from Colorado Basin River Forecast Center (A. Wood) Need to develop a longer-term strategy to measure the progress and value. How to Measure the Progress and Value of NMME? To answer these questions need to define target audience(s)! I would argue there are three dominant audiences each with its own metrics relative to progress and value of NMME:
1. Science research community: New multi-model ensemble and correction methods Improved global models 2. Forecast skill community: Skill metrics such as RMSE, ACC, Briar-Skill Score, reliability diagrams, ROC, RPSS, etc. 3. Customers who will make use of the forecasts: Do NMME forecasts allow me to make decisions that will ?: Make money Save lives, avoid disasters Build more robust infrastructure Many other examples
How to Measure the Progress and Value of NMME? Experience suggests that the more difficult part of the task for climate scientists will be audience 3, the customers (stakeholders, users). Many reasons for this: 1. Customers are heterogeneous. One size may not fit all. 2. Customers have different backgrounds and aptitudes. 3. Forecast products we might think are great may not be very useful for particular users. How to overcome these barriers: Find out what customers wants/needs are and what knowledge gaps exist for them to use forecasts. Provide educational support and simple, flexible web-based tools. Conduct a regular (annual) survey: NWS already doing this. A survey provides regular feedback from customers on strengths
and weaknesses of product (NMME). Understanding User Needs for Climate Information NWS Customer Survey Excerpted analysis from: "Customer Satisfaction with NOAAs National Weather Service Climate Products and Services, Meyers, J., M. Timofeyeva, M. Hawkins, and S. Dixon. NOAA CPC CDPW, Raleigh, North Caroline, October 2010. 1.Customer perception of accuracy and reliability was found to have the higher impact on their overall rating of CPC. Improving the customers understanding of the accuracy of products by providing verification information, clarification of the information content of products, and when possible, improvements in skill should be priorities.
2. Respondents using NWS climate products for personal reasons are most satisfied; the more technical the user (i.e. those in academia, financial markets) the more critical the response. Understanding User Needs for Climate Information NWS Customer Survey 3. NWS will continue regular climate services customer satisfaction surveys to ensure ongoing improvement in products and services based on customer input. 4. Results and analysis will motivate development of new products and services (more than 85% of respondents want additional local climate products). 5. Survey helped identify critical high-impact areas for priority improvement of climate services.
Understanding User Needs for Climate Information NWS Customer Survey 1433 Responses (6 NWS Regions and international users) Breakdown of users by sector: Public: 47% Academia: 10% Government (non-NOAA): 10% NOAA (non-NWS): 3% NWS: 9% Commercial: 8% Other: 12% Customer Metrics for Progress and Value of NMME
1. Web-analytics Number of views, size of data downloads. 2. Memorandum of understanding, web acknowledgement of use of product, use of NMME/NWS/ NCS logo on products. 3. Written acknowledgement of use of NMME in annual reports, inputs to NWS survey, papers, etc. 4. NWS survey on customer satisfaction indices. Progress can be tracked by relative satisfaction from year to year. These can be evaluated on a regular (annual) basis to assess progress. Groups that currently use CPCs ISI-SI forecasts. This likely includes:
1. Other parts of NOAA: RISAs, River Forecast Centers, NIDIS, CTB. 2. Other federal agencies: DOI, Army Corps of Engineers, USAID. 3. State and local emergency planning agencies. 4. Reservoir managers and farmers. 5. Private companies: agriculture, insurance. 6. Humanitarian Relief Agencies: IFRC, WFP, FEWS-NET. 7. Developing world governments. 8. WMO: Global Framework for Climate Service, GPC. 9. Groups working in the developing world to support sectoral projects that use climate information: IRI, IASCLIP.
Mymaths and Mathswatch. Your child records topics that they need to work on independently. This may be their own view or the view of the teacher. The Revision List. Consider "One Minute Maths" on Mathswatch for a quick recap of...
The Synderesis Rule - this is the key principle of NL and all humans strive for this. Good is to be done and evil is to be avoided. So...according to Aquinas. The universe was created by God. Everything has a...
How have holidays improved since those taken 50 years ago? Listen and tick which of the following are mentioned: C'est plus facile de voyager. Il fallaitavoir beaucoup d'argent pour partir en vacancesavant. On avaitmoins de congéspayés.
Questions to ask Parents. IEP/504: Is the individual on an IEP/504 plan at school?. Independent with ADL's: Is the individual able to dress themselves, do laundry, use the stove, cross the street independently, make a sandwich, manage money or checking...
CJ John Marshall-increases courts power. Chief Justice John Marshallannounced the ruling that Marbury's rights had been violated and that he should have his commission.However Marshall said the Judiciary Act had given the Court more power than the Constitution had allowed.
VSEPR Theory - Valence Shell Electron Pair Repulsion theory. VSEPR is a simple, yet powerful technique to predict the molecular geometry (or shapes) of molecules. ... In CO. 2. the bond dipoles are equal in magnitude, but exactly oppose each...