School of Management Mixed methods research: how to

School of Management Mixed methods research: how to

School of Management Mixed methods research: how to combine quantitative and qualitative research Alan Bryman Professor of Organisational & Social Research University of Leicester School of Management The Intrepid Researcher Series, 18 June 2014 www.le.ac.uk/ulsm www.le.ac.uk Plan of the session Focus of session quality issues in mixed methods research Quality criteria in quantitative and qualitative research

Possibility of mixed methods criteria Possible approaches to mixed methods criteria Evidence from studies of social scientists Examination of examples Lessons and tips Why might we need quality criteria? Internal factors Consistent standards for evaluating research Improves the quality of research External factors Useful to be seen to be self-policing Enhances confidence of funding bodies Useful for policy-makers to know which research to ignore or give less

weight to But may stifle innovation and creativity 3 Quality criteria in quantitative research Well known and widely agreed criteria Reliability - concerned with consistency of a measure of a concept Replicability Validity measurement internal external (generalizability)

Quality criteria in qualitative research Significant development Inappropriateness of traditional criteria (internal, external, construct, etc. validity; replicability) But lack of agreement concerning appropriate quality criteria for qualitative research 5 Encyclopaedic approach to quality criteria for qualitative research? 1. How credible are the findings?

2. Has knowledge/understanding been extended by the research? 3. How well does the evaluation address its original aims and purposes? 4. Scope for drawing wider influenceshow well is this explained? 5. How clear is the basis of the evaluative appraisal?

6. How defensible is the research design? 7. How well defended is the sample design/target selection of cases/documents? 8. Sample composition/case inclusionhow well is the eventual coverage described? 9.

How well was the data collection carried out? 10. How well has the approach to, and formulation of, the analysis been conveyed? 11. Contexts of data sourceshow well are they retained and portrayed? 12. How well has diversity of perspective and content been explored? 13. How well has detail, depth and complexity (richness?) of the data been conveyed? 14. How clear are the links between data, interpretation and conclusionsi.e. how well can the route to any conclusions be seen? 15. How clear and coherent is the reporting? 16. How clear are the assumptions/theoretical perspectives/values that have shaped the form and output of the evaluation? 17. What evidence is there of attention to ethical issues? 18. How adequately has the research process been documented? Spencer, L., et al. (2003), Quality in Qualitative Evaluation: A Framework for Assessing Research Evidence www.civilservice.gov.uk/wp-content/uploads/2011/09/a_quality_framework_tcm6-38740.pdf

Why do we need mixed methods quality criteria? More to mixed methods research than the presence of 2 or more separate components May improve mixed methods research practice May enhance credibility of mixed methods research But May stifle creativity at this relatively early stage in development of mixed methods research If lack of agreement on qualitative research criteria, how can we have mixed methods ones? May divert our attention from criteria common to both quantitative and qualitative components But what is mixed methods research?

Mixed methods research is the type of research in which a researcher or team of researchers combines elements of qualitative and quantitative research approaches (e.g., use of qualitative and quantitative viewpoints, data collection, analysis, inference techniques) for the broad purposes of breadth and depth of understanding and corroboration. A mixed methods study would involve mixing within a single study; a mixed method program would involve mixing within a program of research and the mixing might occur across a closely related set of studies. R.B. Johnson et al. (2007) Toward a definition of mixed methods research, Journal of Mixed Methods Research, 2: 112-33. Definitions on page 123. How prevalent is mixed methods research?

From: Bryman, A. Mission accomplished? Research methods in the first five years of Leadership, Leadership, 7 (1), 2011, pp. 73-83. And in Sociology? Mainstream journals % BSA Conference WES % %

35 27 57 Qualitative research 77 83 60 Mixed methods research

11 17 Quantitative research 12 G. Payne et al. (2004) Methodological pluralism in British sociology. Sociology, 38: 153 - 163. Trends in discussions of mixed methods criteria Quality considerations may be important: May be quality issues over and above the use of both

quantitative and qualitative research (mixed methods criteria often a combination of general and specific criteria) Common belief that mixed methods research is automatically superior to mono-method research Discussions of mixed methods research criteria may be treading a similar path to discussions of qualitative research criteria 11 Mixed methods criteria 1. Teddlie and Tashakkori (2009) 2. OCathain et al. (2008) 3. Editors of Journal of Mixed Methods Research (2007)

4. Bryman et al. (2008) 5. OCathain (2010) Mixed methods criteria 1. Teddlie & Tashakkori (2009) emphasise inference quality which is made up of design quality and interpretive rigour. Design quality: 1. 2. 3. 4. Design suitability appropriate methods?; appropriate mixed methods design? Design fidelity are methods and design implemented

rigorously? Within-design consistency do components fit together (e.g. sampling appropriate to type of research method?) Analytic adequacy data analysis rigorous and appropriate to research questions? Teddlie, C. and Tashakkori, A. (2009) Foundations of Mixed Methods Research. Los Angeles: Sage. Interpretive rigour 1. Interpretive consistency are inferences consistent with data and research methods employed? 2. Theoretical consistency are inferences consistent with theory and what is known? 3. Interpretive agreement would other researchers draw same inferences and are inferences consistent

with those of participants? 4. Interpretive distinctiveness are inferences more plausible than other possible inferences? Interpretive rigour (continued) 5. Integrative efficacy do meta-inferences incorporate inferences made in each component and are any inconsistencies in findings explored? 6. Interpretive correspondence do inferences correspond to the research questions and do the meta-inferences meet the stated need for using an MM design? (i.e. is stated purpose for using MM met?) (p. 302) Mixed methods criteria

2. Review of mixed methods quality criteria A good mixed methods study clearly justifies why mixed methods approach is suitable provides a transparent account of the mixed methods design provides appropriate sampling, data collection and analysis of individual components integrates quantitative and qualitative findings and explains process of integration A. OCathain, E. Murphy & J. Nicholl (2008) The quality of mixed methods studies in health services research, Journal of Health Services Research & Policy, 13: 92-8. Good Reporting of A Mixed Methods Study (GRAMMS) 1.

Describe the justification for using a mixed methods approach to the research question 2. Describe the design in terms of the purpose, priority and sequence of methods 3. Describe each method in terms of sampling, data collection and analysis 4. Describe where integration has occurred, how it has occurred and who

has participated in it 5. Describe any limitation of one method associated with the presence of the other method 6. Describe any insights gained from mixing or integrating methods Mixed methods criteria 3. Good mixed methods manuscripts Founding editors of Journal of Mixed Methods Research:

i. Well-developed quantitative and qualitative components ii. Integration of the quantitative and qualitative strands iii. Inclusion of mixed methods approach that adds to the literature about mixed methods research Creswell, J. & Tashakkori, A. (2007) Editorial: developing publishable mixed methods manuscripts, Journal of Mixed Methods Research, 1(2): 107-11. Mixed methods criteria 4. Study of quality criteria among social policy researchers E-survey: URL sent to 800 researchers 347 logged onto website 251 completed

Qualitative telephone interviewees Purposive sample of 28 from 90 who had agreed to be interviewed in e-survey Sampling based on orientations to research, e.g. quantitative vs. qualitative research; user involvement; research vs. policy and practice 20 Evidence from interviews Interviewees probed on criteria for mixed methods research 3 criteria stood out Relevance to research questions Transparency Integration of quantitative and qualitative findings

Also, having a rationale for mixed methods approach Bryman, A., Becker, S., and Sempik, J. (2008) Quality criteria for quantitative, qualitative and mixed methods research: The view from social policy, International Journal of Social Research Methodology, 11: 261-76. 21 Mixed methods criteria 5.The comprehensive approach (OCathain, 2010) 6 domains + 2 Domain 1: Planning quality 4 Domain 2: Design quality 4 Domain 3: Data quality 5 Domain 4: Interpretive rigour 8 Domain 5: Inference transferability 4 Domain 6: Reporting quality 3

28 + 2 = 30 quality criteria OCathain, A. (2010). Assessing the quality of mixed methods research: toward a comprehensive framework. In A. Tashakkori and C. Teddlie (Eds.), SAGE handbook of mixed methods in social and behavioral research (2nd edition, pp. 531-55). Los Angeles: Sage. Do we expect too much of quality criteria? Two prominent roles: 1. Appraisal role. Consideration of adequacy of research from the point of view of assessing research quality (e.g. for inclusion/exclusion in a systematic review) or as part of the refereeing function for research proposals and journal articles. With qualitative research, has led to long lists partly associated with lack of agreement on criteria (cf. Cassell & Symon, 2011) Difficult to argue for the removal of criteria.

In the case of qualitative research, has led to long lists C. Cassell & G. Symon (2011) Assessing good qualitative research in the work psychology field: a narrative analysis, Journal of Occupational and Organizational Psychology, 84: 633-50. Encyclopaedic approach to quality criteria for qualitative research? 1. How credible are the findings? 2. Has knowledge/understanding been extended by the research? 3.

How well does the evaluation address its original aims and purposes? 4. Scope for drawing wider influenceshow well is this explained? 5. How clear is the basis of the evaluative appraisal? 6. How defensible is the research design?

7. How well defended is the sample design/target selection of cases/documents? 8. Sample composition/case inclusionhow well is the eventual coverage described? 9. How well was the data collection carried out? 10. How well has the approach to, and formulation of, the analysis been conveyed? 11. Contexts of data sourceshow well are they retained and portrayed? 12. How well has diversity of perspective and content been explored?

13. How well has detail, depth and complexity (richness?) of the data been conveyed? 14. How clear are the links between data, interpretation and conclusionsi.e. how well can the route to any conclusions be seen? 15. How clear and coherent is the reporting? 16. How clear are the assumptions/theoretical perspectives/values that have shaped the form and output of the evaluation? 17. What evidence is there of attention to ethical issues? 18. How adequately has the research process been documented? Spencer, L., et al. (2003), Quality in Qualitative Evaluation: A Framework for Assessing Research Evidence www.civilservice.gov.uk/wp-content/uploads/2011/09/a_quality_framework_tcm6-38740.pdf Tracys 8 must have criteria 1. Worthy topicrelevant, interesting, significant, etc. 2. Rich rigourrich data supplied in abundance and appropriately 3. Sinceritythe researcher is reflexive about values and biases and is transparent in

approach 4. Credibilityimplements practices such as thick descriptions, triangulation, and respondent validation 5. Resonancehas an affecting impact on readers 6. Significant contributionmakes an impact in terms of such outcomes as theory, practice, and morality 7. Ethicalconsiders and engages in ethical practices 8. Meaningful coherenceaddresses what it claims to address, uses appropriate methods, and links research questions, literature, findings and interpretations. S. J. Tracy (2010). Qualitative Quality: Eight Big Tent Criteria for Excellent Qualitative Research, Qualitative Inquiry, 16: 83751. Do we expect too much of quality criteria? 2. Teaching role Cohorts of students introduced to quantitative research

quality criteria (internal, external, construct, conclusion, and ecological validity). Advantage of small number of issues around which criteria are grouped. From a teaching point of view, long lists may be unhelpful. Lack of differentiation and therefore guidance concerning importance/significance. Potentially off-putting. Unrealisable. Therefore, for teaching purposes, shorter lists may be more helpful to students (and maybe lecturers). Some recurring themes: Alans must haves Need for quantitative and qualitative components to be appropriately implemented

Need for transparency Use of mixed methods to be linked to research questions Need to be explicit about the mixed methods design and its appropriateness to research questions Importance of a rationale for the use of mixed methods research Importance of integration Bryman, A. (2014) June 1989 and beyond: Julia Brannens contribution to mixed methods research, International Journal of Social Research Methodology, 17(2): 121-31. Some recurring themes: Alans must haves Need for quantitative and qualitative components to be appropriately implemented Need for transparency Use of mixed methods to be linked to research questions

Need to be explicit about the mixed methods design and its appropriateness to research questions Importance of a rationale for the use of mixed methods research Importance of integration 28 Some recurring themes: Alans must haves Need for quantitative and qualitative components to be appropriately implemented Need for transparency Use of mixed methods to be linked to research questions Need to be explicit about the mixed methods design and its appropriateness to research questions

Importance of a rationale for the use of mixed methods research Importance of integration Recent reviews of health services research Key methodological components that cut across qualitative and quantitative methodologies were often missing from mixed methods publications. Descriptions of sample selection and sampling procedures, the study context, and data-gathering procedures are essential aspects of interpreting study findings, and mixed methods studies should not be exempt from these basic research requirements. Many mixed methods studies did not include the level of detail that would likely be required for a qualitative or quantitative paper to be accepted in these high-ranking journals.

J.P. Wisdom et al. (2012) Methodological reporting in qualitative, quantitative and mixed methods health services research articles, Health Services Research, 47(2): 721-45. The main quality issue identified was a lack of transparency of the mixed methods aspects of the studies and the individual components. The qualitative components were more likely to be poorly described than the quantitative ones. A. OCathain, E. Murphy & J. Nicholl (2008) The quality of mixed methods studies in health services research, Journal of Health Services Research & Policy, 13: 92-8. 30

Some recurring themes: Alans must haves Need for quantitative and qualitative components to be appropriately implemented Need for transparency Use of mixed methods to be linked to research questions Need to be explicit about the mixed methods design and its appropriateness to research questions Importance of a rationale for the use of mixed methods research Importance of integration Two common approaches to mixed methods research questions 1. Separate quantitative and qualitative research questions.

2. Overarching mixed research question; then expanded or broken down into quantitative and qualitative sub-questions. From: Teddlie, C. and Tashakkori, A. Foundations of Mixed Methods Research: Integrating Quantitative and Qualitative Approaches in the Social and Behavioral Sciences, Los Angeles: Sage, 2009, p. 133. Example of first approach: separate quantitative and qualitative research questions Study of relationship between leadership and effectiveness in conveying strategic organizational goals. H1. Direct reports ratings of their leaders transformational style will be related to those leaders perceptions and articulation of the goals of the organization in prospector strategy terms. Questionnaire

Research Question 1: Are the goals of the organization, as articulated by managers who report to transformational executives, more similar to the executives own goal statements than those who report to executives who are significantly less transformational? Semi-structured interview Example of first approach: separate quantitative and qualitative research questions H2. Transformational leadership will be associated with higher ratings on being open, a careful listener, and a careful transmitter. Questionnaire Research Question 2: To what extent are direct reports who perceive their leaders as superior communicators more aware of the organizations goals as compared with individuals who perceive their leaders as poor

communicators? Semi-structured interview Berson, Y. and Avolio, B.J. (2004) Transformational leadership and the dissemination of organizational goals: a case study of a telecommunications firm, The Leadership Quarterly, 15(5): 625-46. Example of second approach: study of political advertising in US Overarching research question: to explain how and why the political ads of the 2004 presidential candidates failed to engage young adults Three sub-questions: Example of second approach: study of political advertising in US Three sub-questions:

How does the interaction between audience-level and media-based framing contribute to college students interpretations of the messages found in political advertising? To what extent do those interpretations match the framing found in the ads from the 2004 U.S. presidential election? How can political ads be framed to better engage college students? Parmelee, J.H. et al. (2007) What about people our age?: Applying qualitative and quantitative methods to uncover how political ads alienate college students, Journal of Mixed methods research, 1(2): 183-99. Example of second approach: study of political advertising in US Three sub-questions: How does the interaction between audience-level and

media-based framing contribute to college students interpretations of the messages found in political advertising? Focus groups To what extent do those interpretations match the framing found in the ads from the 2004 U.S. presidential election? Content analysis and focus groups How can political ads be framed to better engage college students? Focus groups Some recurring themes: Alans must haves Need for quantitative and qualitative components to be appropriately implemented Need for transparency Use of mixed methods to be linked to research questions Need to be explicit about the mixed methods design and its

appropriateness to research questions Importance of a rationale for the use of mixed methods research Importance of integration Types of mixed methods design: Convergent Parallel Design QUAN QUAN Compare and contrast Findings

QUAL This and the next 3 slides outlining types of mixed methods design based on: J.W. Creswell & V. Plano Clark, Designing and Conducting Mixed Methods Research, 2nd edition, Thousand Oaks, CA: Sage, 2011 Convergent parallel design destination branding in Edinburgh Is the strength of the destination brand greater when stakeholders coordinate their brand activities? Qualitative component Documents and reports on the Destination Edinburgh Marketing Alliance 12 semi-structured interviews with individuals representing diverse subsectors of the tourism industry Quantitative component

Online questionnaire to stakeholders in tourism organizations (55 usable questionnaires returned) Results were merged by comparing them and were interpreted and discussed by stating the degree to which they converged, diverged, or related. I. Bregoli (2013) Effects of DMO coordination on destination brand identity: a mixed method study on the city of Edinburgh, Journal of Travel Research, 52: 212-24. Types of mixed methods design: Explanatory Sequential Design QUAN explained/ elaborated by

Qual Findings 42 Explanatory sequential design: childhood obesity in rural Canadian communities Meaning of health among children in rural Saskatchewan Questionnaire completed by children & parents + height & weight measurements Semi-structured interviews with children at school on 2 occasions

H. Bilinski et al. (2013) Lessons learned in designing and conducting a mixed methods study to explore the health of rural children, International Journal of Health Promotion and Education, 51: 1-10. Explanatory sequential design: childhood obesity in rural Canadian communities although the quantitative data identified that many children (34%) were an unhealthy weight with a significantly higher prevalence of unhealthy weights in boys, the qualitative data discovered that neither weight status nor gender influenced childrens beliefs about health. That is, both boys and girls, and children of healthy and unhealthy weight status described themselves as healthy, had similar beliefs about health, and emphasized that happiness was the most important dimension to their health.

qualitative data allowed an elaboration of the quantitative findings Types of mixed methods design: Exploratory Sequential Design Qual acts as preparation for QUAN Findings

An example Develop the concept of organizational assimilation the process of integration into an organizational culture Semi-structured interviews with a variety of organizational participants several organizations and levels Interviews revealed six dimensions (aspects) of assimilation: familiarity with others acculturation recognition involvement with the organization job competency adaptation and role negotiation 46

Development of questionnaire Questions developed for each dimension (61 items) Dimension Sample questionnaire item Familiarity with others I feel like I know my supervisor pretty well Acculturation I know the values of my organization

Recognition My boss listens to my ideas Involvement with the organization I feel involved in the organization Job competency I can do others jobs, if I am needed Adaptation and role negotiation

I helped to change the duties of my position Response to each item in terms of level of agreement from Strongly Agree to Strongly Disagree 47 Findings Hypothesized that Organizational Assimilation will be: positively correlated with job satisfaction positively correlated with organizational identification negatively correlated with propensity to leave Hypotheses confirmed!

Classic use of a mixed methods approach To develop and improve a quantitative research approach Qualitative research pressed into service for a quantitative approach Myers, K.K. and Oetzel, J.G. (2003)Exploring the dimensions of organizational assimilation: creating and validating a measure. Communication Quarterly, 51: 438-57. 48 Types of mixed methods design: Embedded Design QUAN or QUAL Findings

Qual or Quan Arises from a sense that one set of data wont be enough to answer all aspects of the phenomena of interest (e.g. research questions require quantitative and qualitative data). One set of data often subsidiary to the other Embedded design: Sustainability perspectives on Oulanka National Park, Finland Sustainability of tourism from perspective of local stakeholders A mixed methods approach involving a concurrent embedded strategy with qualitative and quantitative data collection techniques was used to gain a holistic

understanding of local stakeholders ideas and perceptions of tourism and park development in ONP 40 semi-structured interviews followed by selfadministered questionnaire with 33 interviewees Embedded design: Sustainability perspectives on Oulanka National Park, Finland Main method qualitative. Interviewees classified into 1 or 4 groups according to dominant sustainability discourse in their interviews For quantitative component sustainability operationalized and related to other variables the results of qualitative and quantitative analysis supplemented each other and led to a deeper understanding of local perceptions of tourism development pertinent to PAN [Protected Area Network]

Parks Embedded design: Sustainability perspectives on Oulanka National Park, Finland The stakeholder interviews complemented the quantitative analysis since the interviewees discussed issues, especially critical aspects, which were not asked in the survey. More holistic account R. Puhakka et al. (2013) Sustainability perspectives on Oulanka National Park, Finland: mixed methods in tourism research, Journal of Sustainable Tourism, Some recurring themes: Alans must haves Need for quantitative and qualitative components to be

appropriately implemented Need for transparency Use of mixed methods to be linked to research questions Need to be explicit about the mixed methods design and its appropriateness to research questions Importance of a rationale for the use of mixed methods research Importance of integration Mixed methods research rationales

Triangulation Offset Completeness Process Different research questions Explanation Unexpected results Instrument development

Sampling Credibility Context Illustration Utility Confirm & discover Diversity of views Enhancement Other/unclear/not stated

A. Bryman (2006) Integrating quantitative and qualitative research: how is it done?, Qualitative Research, 6: 97-113 The highlighted rationales Triangulation comparing quantitative and qualitative findings for corroboration Completeness using both quantitative and qualitative research for a more comprehensive account Sampling using either quantitative or qualitative research to facilitate selection of respondents Illustration using qualitative data to illustrate quantitative findings Enhancement supplementing to or adding to one set of findings by gathering further data Explanation one is used to help explain findings uncovered by the other

Different research questions explicit link between research questions and mixed methods Instrument development qualitative data used to develop a questionnaire measure Example Political ads & US college students (Parmelee et al., 2007) Mixed methods research entailed Focus groups discussed specific political ads on TV during 2004 presidential race (between Bush and Kerry) and 3 general questions Traditional quantitative content analysis of images and issues in TV ads to establish whether young people and issues in which they are interested are ignored Very explicit rationale. Methods closely tied to specific

research questions. Different research questions Quantitative content analysis to confirm as well as elaborate on the qualitative findings. Triangulation and enhancement Some recurring themes: Alans must haves Need for quantitative and qualitative components to be technically competent Need for transparency Use of mixed methods to be linked to research questions Need to be explicit about the mixed methods design and its appropriateness to research questions Importance of a rationale for the use of mixed methods research Importance of integration

Findings on integration Study of health sciences research: Research reports were assessed to identify the extent to which the potential for integration had been exploited. A fifth of reports were categorized as exploiting their potential for integration. A. OCathain et al. (2007) Integration and publications as indicators of yield from mixed methods studies Journal of Mixed Methods Research, 1: 147-63. Tips on integration 1. Rationale 2. When should integration occur? 3. Dont ignore data clashes 4. Try to see bigger picture as well as fragments of

data 5. Skills 6. Themes Framework Interviewee 1 Interviewee 2 Interviewee 3 Interviewee n Theme 1 Theme 2

Theme 3 Theme 4 Theme n Based on: Ritchie, J., Spencer, L., and OConnor, W. (2003), Carrying out Qualitative Analysis, in J. Ritchie and J. Lewis (eds.), Qualitative Research Practice: A Guide for Social Science Students and Researchers (London: Sage). A Framework approach to integration Method 1 (quantitative) Method 2 (qualitative) Theme 1

Data Data Theme 2 Data Data Theme 3 Data Data

Theme n Data Data Study of destination branding in Edinburgh by Bregoli Putting the two together both sets of results show that there is mixed commitment of stakeholders to the Edinburgh Inspiring Capital brand because both positive and negative answers coexisted. Thanks to the interviews it was possible to identify some of the possible reasons of both the mixed commitment and the

low level of brand citizenship behavior. Several factors identified, e.g. high turnover in industry makes it difficult for newcomers to become familiar with the brand, brand regarded as council brand rather than city brand. Tips on integration 7. At the very least, dont have separate chapters/sections 8. Examine some mixed methods exemplars Structure of the article: Parmelee et al. Introduction Young people, voting, cynicism, and advertising Theoretical and mixed methods perspectives Method

Focus groups Content analysis Results Short introduction Media-based frames and the failure to engage college-age voters Negative ads, audience-based frames, and cynical voters Students recommendations for how to build more engaging political ads Discussion

Summary and implications Limitations and future research Is mixed methods research superior? Universalistic vs. particularistic discourses Particularistic discourse - mixed-methods research should only be used when it is appropriate to an investigations research questions Universalistic discourse views mixed methods research as providing better outcomes more or less regardless of the aims of the research Bryman, A. (2007) The research question in social research: what is its role?, International Journal of Social Research Methodology, 10, 5-20; Bryman et al. (2008) Quality criteria for quantitative, qualitative and mixed methods

research: a view from social policy, International Journal of Social Research Methodology, 11: 1-16. Strategic management research All articles in SMJ 1980-2006. Citations up to January 2009 Quantitative 77%; Qualitative 7.9%; Mixed methods 15.2% (percentages are of all empirical articles) the comparison group consisted of randomly selected monomethod articles that were matched by year and issue to the 165 mixed methods studies. Thus, for each mixed methods study, a monomethod article was selected from the same year and issue of SMJ p. 38 J. Molina-Azorn (2012) Mixed methods research in strategic management : impact and applications, Organizational Research Methods, 15: 33-56.

Highlights of Findings: Practice Main categories in terms of practice: Enhancement52% (rationale 32%) Triangulation 35% (rationale 13%) Completeness 29% (rationale 13%) Illustration 23% (rationale 2%) Explanation 14% (rationale 6%) For example, 80 articles used a triangulation rationale but only 29 of them gave it as a rationale, i.e. majority

of articles using triangulation didnt cite it as a rationale 70 Example: Foot and Mouth Crisis study I W. Poortinga et al. The British 2001 Foot and Mouth crisis: a comparative study of public risk perceptions, trust and beliefs about government policy in two communities, Journal of Risk Research 7 (1), 73-90 (January 2004) Quantitative survey during epidemic + qualitative focus groups with survey participants Sampling Provides more comprehensive view on risk issues Completeness 71

Example: Foot and Mouth Crisis study II Researchers interested in: Public risk perceptions of FMD, who to blame for outbreak, beliefs about government handling, & trust in information about FMD Differences in perception in 2 communities Bude (affected by FMD) + Norwich (not very affected by FMD) 244 + 229 questionnaire respondents 3 + 3 focus groups (questions based in questionnaire) 72 Example: Foot and Mouth Crisis study III Public risk perceptions of FMD Focus group findings reinforce questionnaire (few worried

about health impacts on people) focus groups revealed concern about government policies in handling of FMD, rather than disease itself Blame Questionnaires blame mainly with unregulated meat imports no community differences; focus groups did find differences - reflects community closeness to FMD (reflected greater sympathy for farmers in Bude) 73 Example: Foot and Mouth Crisis study IV Government handling of FMD crisis Support in survey quite high; focus group participants critical (especially in Bude) especially

concern about no government contingency plans Trust in information about FMD Questionnaire data show variations in trust in different groups (vets, farmers, media, MAFF, etc.); focus groups show scepticism about any information sources 74 Example: Foot and Mouth Crisis study V Discussion section Focus group data used to illustrate findings from the questionnaire Illustration Focus groups provided valuable additional information, especially on the reasons, rationalizations and arguments behind peoples

understanding of the FMD issue Completeness 75 Example: Foot and Mouth Crisis study VI perceived causes of the FMD outbreak in the focus groups overlapped largely the 3 factors identified in the questionnaire survey Triangulation + Enhancement Suggest that in Bude high trust ratings of local sources of information + low trust ratings of government may well be a judgment of where these sources are thought to stand in this debate.the focus groups suggested that trust judgments might reflect the extent to which sources are believed to protect people and their interests Explanation

76 Example: Foot and Mouth Crisis study VII Rationale completeness sampling Practice completeness sampling triangulation enhancement illustration explanation 77

Lessons Provide a rationale (or rationales) for doing mixed methods research. Dont assume rationale self-evident Dont forget the importance of describing clearly how the quantitative and qualitative components were conducted Show how research methods proposed/used relate to your research questions Be explicit about the mixed methods design youve employed Demonstrate what is gained by using mixed methods Show how quantitative and qualitative findings are mutually informative (integration) If possible, think about quantitative-qualitative integration right at the outset & at as many stages as possible Try not to think of mixed methods purely in terms of triangulation very

limiting

Recently Viewed Presentations

  • The Introductory Paragraph

    The Introductory Paragraph

    The Parts of the Introductory Paragraph The Hook - Designed to grab attention immediately and give some indication about the essay's topic The Transition - Moves the reader from the hook to the driving force of the essay…. The Thesis...
  • Infection Control Progress Report Trust Board Nizam Damani

    Infection Control Progress Report Trust Board Nizam Damani

    Infection Control Progress Report Trust Board Nizam Damani Clinical Director: Infection Prevention & Control 26th November 2009 * MRSA CDSC Data September 2009 MSSA CDSC Data September 2009 Preventable Infections MSSA/MRSA bacteraemia reduction action plan Introduce standardized IV line pack...
  • Cellular Neuroscience (207) Ian Parker

    Cellular Neuroscience (207) Ian Parker

    Also, another gating particle (h - only one per channel) was introduced to account for the inactivation of the Na channel The Hodgkin Huxley Equation Cellular Neuroscience (207) Ian Parker Lecture # 4 - The Hodgkin-Huxley Axon The Action Potential...
  • healthiersfexcel.org

    healthiersfexcel.org

    Diana Browning Wright, M.S., L.E.P. & Clayton R. Cook, Ph.D. Many people feel that relationship building is simply a natural process that will happen over time, but it needs to be intentional. sometimes even if building relationships is easy for...
  • Introducing The Honeyberry By Bernis Ingvaldson The Honeyberry

    Introducing The Honeyberry By Bernis Ingvaldson The Honeyberry

    Haskap Market Development -The Japanese Opportunity- Feasibility Study August 2007 Éric B. Lefol, PhD, PAg 120 tons of in 2005. Growing Haskap / Blue honeysuckle in Canada Dr. Bob Bors, Sightings reported on Iron Range in Minnesota What is it?...
  • Spectrum Economics By: TRA, UAE at the ITU

    Spectrum Economics By: TRA, UAE at the ITU

    TETRA. FSS. Decreasing Range. Increasing Bandwidth. ... (e.g. height of a station antenna or number of mobile stations in a private network). Fee system should be based on a consensus among all the players through public consultation.
  • Provider/Associate Care Team (PACT): Care for the Caregiver

    Provider/Associate Care Team (PACT): Care for the Caregiver

    Provider/Associate Care Team (PACT) Program. PACT is a standardized program designed to provide emotional support to providers and associates immediately following a significant work-related event.. Provides: Help for the "Second Victim" Immediate peer-to-peer support. Trained peer supporters
  • Chapter 1: Reading Fiction Chapter 1: Reading Fiction

    Chapter 1: Reading Fiction Chapter 1: Reading Fiction

    * "Tell me a story." Some of the earliest forms of storytelling were myths about the origins of the world and legends about the heroic adventures of demigods told in an oral tradition. Why do you think we study stories...