IUSE Reviewer Orientation Improving Undergraduate STEM Education Program

IUSE Reviewer Orientation Improving Undergraduate STEM Education Program

IUSE Reviewer Orientation Improving Undergraduate STEM Education Program (IUSE: EHR) Program Solicitation: NSF 14-588 Division of Undergraduate Education Directorate of Education and Human Resources National Science Foundation Outline IUSE: EHR Overview

Panel Review Process Expectations Ratings & Panel Summary Criteria Panel Review Process COI/Confidentiality & Implicit Division of Undergraduate Education

Directorate of Education and Human Resources National Science Foundation IUSE: EHR OVERVIEW Division of Undergraduate Education Directorate of Education and Human Resources National Science Foundation IUSE: EHR supports 2 NSF strategic goals: 1. Transform the Frontiers of Science and Engineering: Supporting fundamental, high-risk and potentially transformative research in STEM

Education of the next generation of the STEM workforce to continue this transformation Division of Undergraduate Education Directorate of Education and Human Resources National Science Foundation IUSE: EHR supports 2 NSF strategic goals: 2. Stimulate Innovation and Address Societal Needs through Research and Education: supports research and development on STEM education prepare a diverse, globally competent STEM workforce and a STEM-literate citizenry

Division of Undergraduate Education Directorate of Education and Human Resources National Science Foundation IUSE: EHR Program Goals Improve STEM Learning & Learning Environments Improve the knowledge base for defining, identifying, and innovating effective undergraduate STEM education teaching and learning Broaden Participation & Institutional Capacity for STEM Learning Increase the number and diversity of undergraduate students Build the Professional STEM Workforce

for Tomorrow Improve the preparation of undergraduate students Division of Undergraduate Education Directorate of Education and Human Resources National Science Foundation Educational Practice and Research Cycle Educationa l Practice Which help improve... Answers Insights

STEM Learning Environment Identifies and motivates... Questions Ideas That results in... Educationa l Research Division of Undergraduate Education Directorate of Education and Human Resources National Science Foundation

Which lead to... IUSE: EHR 2 tracks Institutional and Community Transformation Engaged Student Learning Research, Design, and Development studies that involve creation, exploration, and implementation of tools, resources, or models Projects that use innovative approaches to

substantially increase the propagation of highly effective methods of STEM teaching and learning in institutions of higher education Division of Undergraduate Education Directorate of Education and Human Resources National Science Foundation Panel Review Process Expectations Division of Undergraduate Education Directorate of Education and Human Resources National Science Foundation Who reviews - What and

How? In our panels we include reviewers that are Experts in fields of STEM education, STEM content, methodology, cognitive science, and administration Panelists read 10 12 proposals, but panels may include more than 12 proposals and thus you will not read every proposal assigned to a panel IUSE is using a mix of virtual and face to face panels. Often the more complex Exploration proposals are being discussed face to face in panels here in the DC area. Division of Undergraduate Education Directorate of Education and Human Resources National Science Foundation The Proposer Receives: NATIONAL SCIENCE FOUNDATION

4201 Wilson Boulevard Arlington, Virginia 22230 Dear Dr. Doe, The National Science Foundation hereby awards a grant of... NATIONAL SCIENCE FOUNDATION 4201 Wilson Boulevard Arlington, Virginia 22230 $ Reviews Panel Summary (if applicable) anonymous

& Dear Dr. Doe, I regret to inform you that the National Science Foundation is unable to support your proposal referenced above... Context statement & Award/Declination letter Division of Undergraduate Education Directorate of Education and Human Resources National Science Foundation

NSF Merit Review Criteria Guiding Principles All NSF projects should be of the highest quality and have the potential to advance, if not transform, the frontiers of knowledge. Meaningful assessment and evaluation of NSF-funded projects should be based on appropriate metrics, keeping in mind the likely correlation between the effect of broader impacts and the resources provided to implement projects. Division of Undergraduate Education Directorate of Education and Human Resources National Science Foundation NSF Merit Review Criteria Required

Both criteria are to be given full consideration during the review and decision-making processes; each criterion is necessary but neither, by itself, is sufficient. Intellectual Merit: The Intellectual Merit criterion encompasses the potential to advance knowledge Broader Impacts: The Broader Impacts criterion encompasses the potential to benefit society and contribute to the achievement of specific, desired societal outcomes Division of Undergraduate Education http://www.nsf.gov/pubs/policydocs/pappguide/nsf14001/

Directorate of Education and Human Resources National Science Foundation gpg_3.jsp#IIIA Merit Review Considerations The following elements should be considered in the review for both Intellectual Merit & Broader Impacts: What is the potential for the proposed activity to: Advance knowledge and understanding within its own field or across different fields (Intellectual Merit)? and Benefit society or advance desired societal outcomes (Broader Impacts)? To what extent does the proposed activity suggest and

explore creative, original or potentially transformative concepts? Division of Undergraduate Education Directorate of Education and Human Resources National Science Foundation (Merit Review Considerations cont.) The following elements should be considered in the review for both Intellectual Merit & Broader Impacts: Is the plan for carrying out the proposed activities wellreasoned, well-organized, and based on a sound rationale? Does the plan incorporate mechanisms to assess success? How well qualified is the individual, team, or institution to conduct the proposed activities? Are there adequate resources available to the PI (either at the home institution or through collaborations) to carry out the proposed activities?

Division of Undergraduate Education Directorate of Education and Human Resources National Science Foundation Common Guidelines for Education Research & Development: NSF13-126 Offer guidance on building the evidence base in STEM learning Are not a requirement for IUSE: EHR proposals, but may be helpful to proposers and reviewers We encourage you to be familiar with them Division of Undergraduate Education Directorate of Education and Human Resources National Science Foundation

Basic perspectives from the Common Guidelines for IUSE: EHR Projects Should be both knowledge -using and knowledge-producing. Using: Is the project building on prior work? Producing: What will we learn from the planned conduct of this project? *Perspective: It is OK for IUSE: EHR projects to serve the PIs institution, but they should also serve all of us by providing useful knowledge for broad constituencies of educators Division of Undergraduate Education Directorate of Education and Human Resources National Science Foundation

(Basic Perspectives cont.) Applied work (e.g. development) is fine as long as it adheres to the principles of the preceding slide. [NSF budgets are not large enough to support simple local improvement projects.] While we call for innovation, replication is also acceptable and an important part of science. [How do we know that something is effective if has been tested at only one place?] Division of Undergraduate Education Directorate of Education and Human Resources National Science Foundation (Basic Perspectives cont.)

IUSE: EHR accepts projects of different size. They should be judged relative to their funds request. Institutional transformation projects by their very nature seek major local impacts. These also need to provide insights about how to accomplish the transformation, and thus the proposal should deal explicitly with the theory of change that underlies its promise, and provide an applied research design. Division of Undergraduate Education Directorate of Education and Human Resources National Science Foundation The Project Budget & Summary Rating

As you explore the project budget request and compare this with the scope of the project, please note that: Budgets that seem out of line with the proposed project can be negotiated by the program officer if the project itself seems promising. Hence, we request that you dont assign a killer rating due to budget alone. [A review might provide a rating conditional on budget reforms.] Division of Undergraduate Education Directorate of Education and Human Resources National Science Foundation Reviewer Ratings and

Panel Summary Criteria Division of Undergraduate Education Directorate of Education and Human Resources National Science Foundation Use the Entire Proposal to Inform Your Review: Project Summary Project Description Biographical Sketches Budget Letters of Commitment Project Data Form Current & Pending Support Division of Undergraduate Education

Directorate of Education and Human Resources National Science Foundation Individual Review Process: Rating the Proposal Assign only ONE rating: E, V, G, F, P Do NOT assign split ratings, such as E/ V or V/G Your rating MUST be consistent with your written review Division of Undergraduate Education Directorate of Education and Human Resources National Science Foundation Individual Review Process: Writing Your Review

Within FastLane: In your own words, write aspects you find to be compelling and those that you believe will benefit from improvement; Do NOT cut and paste text from the proposal into your review Be sure that your written review is consistent with your rating It is OK to modify reviews during the panel meetings, including a change of rating Be sure any modifications to reviews are recorded in FastLane! - MUST be done BEFORE the panel is closed. Note that many MS Word symbols do not translate in FastLane - check your work after submitting Division of Undergraduate Education Directorate of Education and Human Resources National Science Foundation

Characteristics of Informative Reviews Include a section for IM, BI and summary Use appropriate style (positive tone) Contain adequate details Write understandable, specific, and complete statements Relate strengths and weaknesses, supported by detailed explanation, associated with review criteria If an item would have helped to strengthen the proposal, a clear statement should be included as to why that would be the case. Division of Undergraduate Education Directorate of Education and Human Resources National Science Foundation

Panel Review Process Division of Undergraduate Education Directorate of Education and Human Resources National Science Foundation During the panel: Each proposal will typically have four reviews Each proposal will have a reviewer assigned as the Scribe/Lead Discussant (called the primary panelist) The Scribe/Lead Discussant will write the panel summary as well as write an individual review Panel members assigned to review the proposal are called secondary panelists. Panel members not assigned to review the proposal are simply identified as panelists.

Division of Undergraduate Education Directorate of Education and Human Resources National Science Foundation Proposal Discussion The Scribe/ primary panelist is the lead discussant who presents an overview of the project being proposed. The Scribe then provides the highlights of her/his own review

Each assigned secondary panelist will provide his/her individual highlights while the scribe takes notes Additional discussion will include all panelists providing the scribe with sufficient Division of Undergraduate Education Directorate of Education and Human Resources National Science Foundation Panel Summaries

For each proposal: Initially framed by primary panelist who serves as scribe Should reflect the panel discussion (not just restate individual reviews) Include short, clear comments to help PIs improve their projects Should be written in 3rd-person, as the panel, and will be read and agreed to by all panelists Division of Undergraduate Education Directorate of Education and Human Resources National Science Foundation Characteristics of Informative Panel Summaries Includes a section for IM, BI and summary

Uses appropriate style (positive tone) Contains adequate details Contains understandable, specific, and complete statements Relates strengths and weaknesses, supported by detailed explanation, associated with review criteria If an item would have helped to strengthen the proposal, a clear statement should be included as to why that would be the case Division of Undergraduate Education Directorate of Education and Human Resources National Science Foundation COI/Confidentiality & Implicit Bias

Division of Undergraduate Education Directorate of Education and Human Resources National Science Foundation Conflict of Interest (COI) Primary purpose is to remove or limit the influence (or appearance of influence) of ties to an applicant institution or investigator that could affect reviewer advice Second purpose is to preserve the trust of the scientific community, Congress, and the general public in the integrity, effectiveness, and evenhandedness of NSFs peer review process

Division of Undergraduate Education Directorate of Education and Human Resources National Science Foundation Statement on Confidentiality NSF receives proposals in confidence and protects the confidentiality of their contents Proposals should not be discussed before the review process or after the review process with anyone outside of the panel You must not copy, quote from, or otherwise use or disclose (to anyone) material from any proposal you are asked to review The fact that you are reviewing IUSE proposals now should also be kept confidential

Division of Undergraduate Education Directorate of Education and Human Resources National Science Foundation Implicit Bias Implicit bias toward a group (schemas) Non-conscious hypotheses/stereotypes, often about competence Implicit institutional or geographic bias

Lack of critical mass leads to greater reliance on schemas Fewer women and minorities in sciences Accumulation of disadvantage Accumulation of small biases in the same direction has large effects over time Very small differences in treatment can have major consequences in salary, promotion, and prestige (Valian 1998) Division of Undergraduate Education Directorate of Education and Human Resources National Science Foundation https://implicit.harvard.edu/implicit/

Implicit Bias Schemas are: Widely shared culturally: All people, including members of underrepresented groups, hold schemas about these groups People are often not aware of them Applied more under circumstances of: Lack of information Stress from completing tasks Time pressure Lack of critical mass Fiske (2002). Current Directions in Psychological Science, 11, 123-128. Division of Undergraduate Education Directorate of Education and Human Resources National Science Foundation

Division of Undergraduate Education Improving Undergraduate STEM Education Program (IUSE: EHR) Questions? Comments? Division of Undergraduate Education Directorate of Education and Human Resources National Science Foundation Division of Undergraduate Education Improving Undergraduate STEM Education Program

(IUSE: EHR) Thank You! Division of Undergraduate Education Directorate of Education and Human Resources National Science Foundation

Recently Viewed Presentations

  • W H AT T H E D ATA

    W H AT T H E D ATA

    PE LANDSCAPE IN 2007-10. ACTIVE INVESTORS. Growth Capital. Nexus , Helion, Ventureast, Sequoia, Accel, DFJ, Bessemer, Kalaari, IDG, Inventus Capital, Matrix Partners,
  • Principles of Macroeconomics - MIT - Massachusetts Institute ...

    Principles of Macroeconomics - MIT - Massachusetts Institute ...

    Times New Roman Blank Presentation.pot Principles of Macroeconomics Macroeconomics? Style "In the News" (Bloomberg 09/05) A Tour Around the World Figures/Tables-Chapter 1
  • Lesson 1.1 Building a Case to Prevent Foodborne

    Lesson 1.1 Building a Case to Prevent Foodborne

    Lesson 1.1 Building a Case to Prevent Foodborne Illness Definitions and templates for: Case Notes 1.1 l Case File Checklist, FSI Unit Debriefing Notes l FDA Alert l Homework Assignment 1.1 Definition Physical hazards Foreign objects that can cause injury...
  • Probabilistic Approaches to Video Retrieval The Lowlands Team

    Probabilistic Approaches to Video Retrieval The Lowlands Team

    Probabilistic Approaches to Video Retrieval The Lowlands Team at TRECVID 2004 Tzvetanka ('Tzveta') I. Ianeva Lioudmila ('Mila') Boldareva Thijs Westerveld
  • What's In A Question? - Mediate.com

    What's In A Question? - Mediate.com

    What's In A Question? The Art and Science of Inquiry in Mediation Tracey Cairnie and John Settle "Many questions have assumptions embedded within them that influence meaning, impact & effectiveness" -- Making Questions Work, Dorothy Strachan * * * *...
  • Mini Mall Design Submitted by: Dalya Dawoud Dina

    Mini Mall Design Submitted by: Dalya Dawoud Dina

    Fire resisting . Automatic system centralized was selected in this building, the smoke. Alarm sound throughout the mall . There are two stairs in the project which is considered as emergency exist.
  • Validating LiDAR Stand Metrics with Timber Cruise Dan

    Validating LiDAR Stand Metrics with Timber Cruise Dan

    Data Structure FOI-VEG - Forest Vegetation A unique aspect to the FOI-VEG data standard is a published attribute table where much of the data comes from the M*S related tables.
  • Latin & Greek Roots

    Latin & Greek Roots

    One year after the crash of the stock market, the economy remains stuck in deep . malaise. Synonym: boredom . Words from "mal" ... Antonym: affirm . Words from "Neg" ...