Connecting Research, Policy and Practice First Impressions: How

Connecting Research, Policy and Practice First Impressions: How

Connecting Research, Policy and Practice First Impressions: How to Win Grants and Influence Your Research Career Meredith Larson, Ph.D. National Center for Education Research and Amy Sussman, Ph.D. National Center for Special Education Research ies.ed.gov Goals 1. Identify two things you should have with you at all times. 2. Describe a theory of change and discuss why its essential grant writing. 3. Have the tools necessary for refining your research statement of purpose. 4. Have a sense of what program officers and reviewers want from you and your applications. All of this is to help prepare you to write winning proposals and

articulate your research agenda to the broader (research) community. ies.ed.gov Our Assumptions of You and What You Want We assume that you want to help improve education to carve out your own research agenda to get funding to conduct your research We assume that you have expertise in a wide range of topic areas and research methods/analyses varying levels of familiarity with grant writing ies.ed.gov What Youre Facing Whenever you ask someone to support your research, remember that it all comes down to convincing them that they should care about your topic and that you have the necessary competencies to reach your goals. so how do you do it?

ies.ed.gov What Youre Facing Audience awareness 1. Seek common ground. Highlight core values or concerns that your research addresses. Look for ways to build off of things that are familiar. 2. Be helpful and thorough. Make things easy to understand, remember, and find. Be engaging and helpful. ies.ed.gov What You Need to Do Sell your research idea, promote yourself as the best person to do the research, and build goodwill and trust. How? By demonstrating that you know what the problem is and have a way to address it. ies.ed.gov Explaining Your Research

You should have two things ready at all times: 1. Statement of Purpose (preferably an elevator speech or research pitch) 2. Theory of Change ies.ed.gov Statement of Purpose We will come back to this later, but in general a readymade Statement of Purpose should Be short and attention-getting and Contain the problem statement and your contribution to solving it. Your fellow researchers, friends, and family members should be able to understand it and see its relevance. NOTE: Its not as easy to do as you may think. You have to know a lot about what it is you want to do before you can describe it succinctly. ies.ed.gov Theory of Change (ToC) This is the model underlying your research. It is a roadmap to your grant narrative. It is a source for generating research questions. It is constantly evolving. NOTE: Some fields and scholars use terms like Logic Model or Logical

Framework to mean things similar to what we are calling a Theory of Change here. ies.ed.gov ToC: What is It? Theories of Change help define how and why an intervention (e.g., curriculum, policy, practice) should lead to particular outcomes. Makes assumptions explicit Suggests the causal relations ies.ed.gov ToC: What is it? Some associate them with program evaluation studies, but they are also useful for exploration work, work on cognitive or behavioral processes, etc. They are frequently represented visually, but they can be simply text. ies.ed.gov ToC: Why You Should Care Reviewers will evaluate your research on how clear

your theory and model are. A strong ToC will help make your case for you. Four Criteria Used to Judge ToCs (Connell & Klem, 2000): 1. 2. 3. 4. ies.ed.gov Plausible Doable Testable Meaningful ToC: How to Make One Start with the long-term outcome and work backwards. What preconditions need to exist to lead to the outcome? What is occurring in the context that could hinder or support the outcome? What assumptions are you making? Draw it out. Write it up. Get feedback. Revise. Repeat.

ies.ed.gov Create the General Framework Come up with the broad framework. Initial State Context Strategies/ Activities Intermediate Outcomes Long-term Outcomes Generally, you should start with the outcomes. You can then jump all the way back to thinking about the initial state or hop around. ies.ed.gov The Basic Components Assumptions Initial State (resources)

Strategies/ Activities Outcomes (proximal & distal) Context INITIAL STATE: Whats there before the intervention including things such as resources or student characteristics, etc. (sometimes called input) STRATEGIES/ACTIVITIES: The things that happen (e.g., activities, events, curriculum) to the people who participate or who are the targets of the intervention (sometimes called output) OUTCOMES: The proximal and distal changes for people involved in or that are the targets of the intervention ASSUMPTIONS: What you believe to be true of the resources, the people involved, the people targeted, and the intervention in general CONTEXT: The environment (internal or external) in which the intervention takes place (e.g., other policies or practices occurring simultaneously) ies.ed.gov Example

4-year-old pre-K children Demographics Eligibility/need Exposed to intervention 1-hour lesson Wrap-around services Positive supports Focus on early literacy PD for providers Positive attitudes to school Improved pre-literacy skills Increased school readiness

Learn appropriate school behavior Setting/context Personal and family characteristics Prior experience Teacher/provider experience ies.ed.gov Greater gains in K literacy Detailed Components Now that you have the general aspects, you should specify the details. These will lead into your research design, measures, etc. in your narrative. Indicators: Operationalizing the outcomes (e.g., what indicates success?) Populations: Identifying who should show which change (could be multiple groups) Thresholds: Setting your expectations for change for each outcome (e.g., how much is good enough?)

Timeline: Determining when we should meet the threshold for each outcome (Connel & Klem, 2000) ies.ed.gov Example OUTCOME = Improved pre-literacy skills Indicator: Correctly identifies letters and sounds Population: 4-year-olds, especially those with little print exposure Threshold: 1.5 effect greater than comparison Timeline: After full intervention, 16-weeks ies.ed.gov Another Example (Slightly different layout ) Instructional Context Text and task characteristics Nature of instructional support Reader

Characteristics Decoding and fluency proficiency Verbal knowledge Attention and behavior ies.ed.gov Motivation to Read Decision to engage in reading and task persistence Expectancies for success Self-efficacy Perceived control Value Achievement goals Intrinsic motivation Usefulness Social motivation Text Comprehension Reading Engagement

Physical Cognitive: Building and maintaining coherence Adapted from a model presented by Dr. Sharon Vaughn (R305F100013). Don't Do This 1. Be overly simplistic 2. Overwhelm the reader 3. Use color as a key (audience awareness!) ies.ed.gov Example: Too Simple Students with Disabilities ies.ed.gov The Increasing

School Success Program Increased Academic Achievement [Processes 3-17 repeat twice in Year 2] 16. 16. interview interview coaches, coaches, SLTs, SLTs, and and PLTs PLTs about about PD PD and and WL WL

processes processes 13-15b. 13-15b. Observe SLTs, Observe SLTs, document document implementation implementation 15a.Coaches 15a.Coaches assist assist SLT SLT in in implementing implementing PD PD with with faculty faculty 14a.

14a. SLT SLT and and Coaches Coaches create create PD PD unit unit PLT 8a1. 8a1. Coaches Coaches assist assist PLTs PLTs in in using using all all data data 8a2.

8a2. Coaches Coaches assist assist PLTs PLTs in in weekly weekly WLs WLs and and Debriefing Debriefing through through mid-year mid-year PLT PLT WL WL Debrief Debrief

WL WL Debrief Debrief WL WL Debrief Debrief Begins Begins weekly weekly meetings meetings WL WL Debrief Debrief Implement Implement Prof

Prof Devt Devt approx. 3 wks after school begins PLT 13a. 13a. researchers researchers train train SLT SLT to to research research bestbestpractices practices in in PD PD area area PLT

WL WL Debrief Debrief Profile Profile Analysis Analysis Set Set instructional instructional goals goals & & WL WL focus focus [4 weeks at end of prior year or beg of current year] PLT WL WL

Debrief Debrief Research Research [PLT appoints SLT1 to address PD Topic 1] = Begin Process = Feedback for next process = Delegation of PD PLT = Primary Leadership team SLT = Secondary Leadership team 6a. 6a. Coaches Coaches facilitate facilitate PLT

PLT identification identification of of annual annual goal goal PLT WL WL Debrief Debrief PLT PLT SLT 1 6-8b. 6-8b. observe observe PLTs, documents

PLTs, documents implementation implementation 7a. 7a. Coaches Coaches teach teach PLT PLT to to conduct conduct WL WL PLT PLT SLT 1 WL WL Debrief Debrief

ies.ed.gov 18a. 18a. submit submit Annual Report Annual Report to to schools schools 17. 17. reviews/revise model model based based on findings PLT WL WL

Debrief Debrief PLT PLT PLT 22 4. 4. Coaches Coaches work work with with PLT PLT WL WL Debrief Debrief

11. 10. 11. Coaches Coaches 10. share/ reviews/revises share/ implement model implement model based based revisions on findings revisions with with PLTs PLTs approx. 9 wks after school begins 5. 5. guide guide Coaches

Coaches and and PLTs PLTs in in analyzing analyzing data data collected collected WL WL Debrief Debrief WL WL Debrief Debrief PLT [Processes 4-11 WL WL

repeat to mid-year] Debrief Debrief 12. 12. Coaches Coaches & & PLTs PLTs choose choose 2-4 2-4 teachers teachers (based (based on on WLs) WLs) to to become become Dynamic Dynamic Leadership Leadership Team Team 1] 1]

WL WL Debrief Debrief 9. 9. interviews interviews coaches coaches and and PLTs PLTs about about WL WL process process 1a. 1a. PI PI Recruits Recruits and and Trains

Trains Coaches Coaches 1b. 1b. develops develops data data collection collection tools tools 2a. 2a. PI PI matches matches Coaches Coaches to to schools schools (n=5)

(n=5) 3b. 3b. develops develops electronic electronic tool tool 3a. 3a. Coaches Coaches collect collect 3 3 yrs. yrs. stud. stud. ach. ach. & & demo. demo. data data per

per school school 2b. 2b. trains trains coaches coaches to to use use all all data data tools tools Draw your Theory of change Take a few minutes and write a couple words about the following: 1. What are the initial states (inputs)? 2. What are the strategies/activities (outputs)? 3. What are the outcomes? Proximal, Intermediate, Distal 4. What is in the context?

ies.ed.gov So Now What? Your Theory of Change acts like a roadmap that can help lead you to research questions, a proposal, and so much more. ies.ed.gov ToC & Research Questions Questions about initial states or inputs Were resources provided and used? Questions about strategies/activities or outputs Did the activities occur as planned? Questions about outcomes Were there changes in the proximal/distal outcome relative to the control/comparison? Questions about context Did the context suppress or reinforce effects? ies.ed.gov

(Milanowski & Kimball, 2009) ToC & Your Application(s) In IES applications, there are four major parts of the application: Significance, Research Plan, Personnel, and Resources. The first two of them can draw heavily from the ToC. Significance Draws heavily from the outcomes, context, and assumptions Research Plan Draws heavily from the strategies/activities and the outcomes ies.ed.gov ToC & Your Research Plan In your research plan, you need to specify exactly what it is youre exploring, creating, validating, or testing. You also need to specify how you will do these things. Strategies/Activities What are the pieces that youll be exploring, creating, testing, etc.? Outcomes Indicators: What will you measure, and how you will measure it? Populations: Who and where (both in treatment and control/comparison)? Thresholds: What effect (size) should you expect?

Timeline: When should you be collecting what data? ies.ed.gov ToC & Your Application(s) Other components of your narrative and your application can also be informed by the ToC: Personnel For example, you want someone with experience in each of the outcome areas and with knowledge of how to assess the strategies/activities. Budget For example, how many observations will you need to take (# of researchers and % effort) and when (during what grant year) and what assessments you will need (how many of which one should you buy). ies.ed.gov ToC & Your Statement of Purpose Your Theory of Change also contains the core elements of a good statement of purpose, mission statement, elevator speech, etc. You have the pieces. Put them in a short, digestible format that conveys the problems significance. And now you have your statement. (It actually requires a bit more than that.) ies.ed.gov

An Example Now were going to walk through an example of how you might take a ToC and generate an elevator speech, a statement of purpose, even an opening paragraph. Note: Sometimes people have their statement and then create their ToC. You need both, and they inform one another. You do not need to have a ToC to write a statement. ies.ed.gov From ToC to Statement of Purpose Distal Outcome: Increase student achievement in science Proximal Outcome: Young children (preK K) with greater ability to think scientifically Strategies/Activities: Professional development curricula Four 1-hour long workshops for teachers

Training on science content and early childhood pedagogy Current, practicing educators PreK students Initial State: Early childhood educators who lack science content and pedagogical knowledge, lack of curricula to help teachers Assumption: Teachers are essential to student outcomes. Teachers need both content and pedagogical knowledge and expertise. Teaching teachers will improve student outcomes. ies.ed.gov continued Speech: Being able to think scientifically at a young age predicts long-term academic success in science. However, many children coming into elementary school lack this ability. Even if they are exposed to science in PreK programs, they are still not acquiring the necessary skills. One reason PreK students may be struggling is that early childhood educators often do not have the background knowledge of science content or instruction to help their students. Because teachers are so important to the learning process, we need to help better prepare them for their role. So I am developing an intervention that will increase early childhood educators content and pedagogical knowledge so that they can better prepare their students.

ies.ed.gov Components of a Good Statement Story! Characters, goals, drama, resolution. Facts but not too many. This will vary based on audience and use. Short shoot for a paragraph or < 90 seconds. Make them care. ies.ed.gov Now you try What is the problem? The tension? The drama? What evidence do you have that its important? How can you help solve it? ies.ed.gov Uses of Your Statement Opening paragraph(s) of your applications or your purpose paragraph Note: This is true for all documents, such as dissertation proposals, not just IES applications.

Job talks Communicating with the general public (e.g., when you give your NPR interviews) ies.ed.gov Importance of Opening Paragraph Opening paragraph sets the scene for readers. Identifies the significance of the work to be done and what actually will be done Readers use it to organize information in rest of the application You can lose your readers right off with an unclear opening ies.ed.gov NCER/NCSER Abstracts As an example of an opening/purpose paragraph, wed like to use examples from some of the abstracts we have posted online: http://ies.ed.gov/funding/grantsearch

ies.ed.gov Example of Opening/Purpose Purpose: Teaching children how to think scientifically in the Background preschool years has the potential to address an existing achievement gap in early science and provide children with the skills necessary to continue learning and thinking critically throughout the school years. Unfortunately, early childhood teachers typically lack content and pedagogical knowledge in science and are not prepared to provide developmentally appropriate experiences that support children's early science learning and readiness. In order to address these challenges, the researchers will develop the Cultivating Young Scientists (CYS) intervention, which includes a professional development program for early childhood educators, science curricular content, and a set of formative assessment tools. The intervention is intended to lead to an increase in teachers' use of science instructional content and practices in preschool settings and improvements in young children science content knowledge and scientific thinking skills. ies.ed.gov Problem

Goal: How will you address this problem? Example of Opening/Purpose Core diagnostic features of autism include deficits in social-communicative functioning. Background Two pivotal skills for young children with autism include joint attention and pretend play, which constitute early foundations upon which later social-communicative skills are built. Joint attention (characterized by behaviors such as pointing, showing, and coordinated looking to share attention toward objects or events with another person) and symbolic play (characterized by the ability to pretend), play important roles in language development and social engagement with peers. Children with autism show deficits in these skills, and as a consequence may lag behind their peers academically and socially. Advancing Social-Communication and Play (ASAP) is an intervention that has recently been developed to help preschool children with autism learn and practice these important skills. However, we do not know how effective it is. Problem The purpose of this research is to conduct a cluster randomized trial to evaluate the efficacy of ASAP. The major goals of the project include investigating whether children experiencing the intervention demonstrate greater gains in the proximal child outcomes

of social-communication and play skills as well as the more distal outcomes of language development and engagement with classroom objects and peers. The study will also examine whether child-level and teacher-level (i.e., teacher burnout, general classroom quality) characteristics moderate the impact of the intervention. Goal: How will you address this problem? ies.ed.gov Final Comments: Helpfulness of the Program Officer Share your framework and statement of purpose with the Program Officer Ensure you are submitting to the correct competition/topic Springboard for further discussion ies.ed.gov Final Comments: Importance of Clarity of Writing Readers (e.g., application reviewers) often

complain about lack of clarity. Significance too general Lack of detail regarding intervention, development cycle, or data analysis Use of jargon and assumptions of knowledge Poor writing (e.g., grammar), awkward constructions, etc. ies.ed.gov Final Comments: Importance of Practicing All forms of communication require practice. Things only other people can tell you: Is your description of your theory too long or complex? Is your statement of purpose too long or complex? Is your idea engaging, and does it invite questions that build off of your ideas (rather than those that try to figure out what your idea really is)? ies.ed.gov Pulling It All Together Recall Why Youre Here: You want to write winning grants and build a solid

research career. But How? By getting people interested in your work and the questions you feel are important and by building trust that you have a plan for addressing an important issue. ies.ed.gov Pulling It All Together Have a theory of change Should be as fully explicated as possible Is always being refined Should be made with and informed by others Have a clear statement of purpose at all times Your elevator speech Simple, elegant (hard to do, actually) ies.ed.gov Connecting Research, Policy and Practice Thank you. [email protected]

[email protected] ies.ed.gov More info Remember: Some scholars distinguish between a Theory of Change and a Logic Model. Others do not. We use the term Theory of Change for the purposes of this presentation as a way to discuss how people can present the causal relations among elements. Please use the term most appropriate for your field. (Note that the IES RFAs use Theory of Change.) This presentation was informed in part by the following resources: Connell, J., & Klem, A. (2000). You can get there from here: Using a theory of change approach to plan urban education reform. Journal of Educational and Psychological Consultation, 11(1), 93-120. Center for Theory of Change: http://www.theoryofchange.org/. Retrieved May 1, 2013. Harris, E. (2005). An Introduction to Theory of Change (Issue Topic: Evaluation Methodology, vol. 11). Retrieved May 1, 2013, from http://www.hfrp.org/evaluation/the-evaluation-exchange/issue-archive/evaluation-methodology/an-introduction-to-theory-of-ch ange . Kimball , S. & Tony Milanowski, T. (2009). Establishing a Theory of Action and Logic Model for Your Project Session 1 [PowerPoint slides]. Retrieved on May 1, 2013 from Center for Educator Compensation and Reform website: http://www.cecr.ed.gov/pdfs/september2009meeting/TheoryofAction.pdf. Reisman, J., Gienapp, A., Langley, K., & Stachowiak, S. (2004). Theory of change: A practical tool for action, results and learning. Organizational Research Services for Annie E. Casey Foundation. ies.ed.gov

Recently Viewed Presentations

  • Programming Languages Presentation

    Programming Languages Presentation

    Later recognized as a general purpose programming language. Developed in 1960s by Ole-Johan Dahl and Kristen Nygard. First object oriented programming language. Two versions - Simula 1 and Simula 67. Influenced later generations of programming languages notably C++, Smalltalk. Extension...
  • SURREALISM

    SURREALISM

    Create a scene that could not exist in real life. SURREAL COLLAGE SURREAL COLLAGE SURREALISM a 20th-century avant-garde movement in art that sought to release the creative potential of the unconscious mind, for example by the irrational juxtaposition of images.
  • ARIES Program Peer Review - University of California, San Diego

    ARIES Program Peer Review - University of California, San Diego

    The ARIES Engineering Activity is a National Effort UCSD Power core engineering, CAD, H&CD systems, integration UW Neutronics, shielding, activation & waste, engineering design ANL Materials, blanket engineering, power conversion, tritium MIT Magnet systems PPPL TF magnets Boeing Manufacturing, RAMI,...
  • Chapter 1: Drawings

    Chapter 1: Drawings

    - usually from deciduous trees, which lose their leaves in autumn, although some hardwood (for example, tropical hardwood) comes from other types of tree. softwood - from coniferous trees, which remain green throughout theyear. Basil Hamed
  • Wait a Minute: Slowing Down Criminal Activity Eviction Cases ...

    Wait a Minute: Slowing Down Criminal Activity Eviction Cases ...

    This slide show is based on: Fred Fuchs, Defending Families and Individuals threatened with Eviction from Federally Subsidized Housing, HOME-Funded Properties, § 515 Rural Rental Housing, § 8 Moderate Rehabilitation, Shelter Plus Care Housing, Supportive Housing for the Elderly and...
  • EL REALISMO - WordPress.com

    EL REALISMO - WordPress.com

    Sus novelas Arroz y tartana, La barraca… siguen las técnicas del Naturalismo; siendo el rasgo más característico la impersonalidad. La catedral, El intruso… pertenecen al naturalismo político, caracterizadas por una fuerte tendencia anticlerical y republicana.
  • Ethical Foundations

    Ethical Foundations

    CONSEQUENTIALISM (TELEOLOGIAL) Evaluates the moral status of an act by looking at the consequences The END (consequences) does justify the Means (the act). FORMALISM AKA: deontological or nonconsequentialism Determine morality by whether or not an act is right or wrong...
  • Use of semantic technologies for sharing European Spatial Data

    Use of semantic technologies for sharing European Spatial Data

    Class. Definition: description of a set of objects that share the same attributes, operations, relationships, and semantics. Objects are also called