Class 2 Measurement Terminology and Locating Specific Measures

Class 2 Measurement Terminology and Locating Specific Measures

Class 2 Measurement Terminology and Locating Specific Measures October 1, 2009 Anita L. Stewart Institute for Health & Aging University of California, San Francisco 1 Overview of Class 2 Measurement terminology Confusion, variation, and overlap Items and response scales

Things to pay attention to in items Locating measures Finding literature on measures and measurement properties 2 Measurement Assignment of numbers to: aspects of objects/events according to a rule an unobservable characteristic of individuals (construct/latent variable)

Numbers (measures) are proxies for the unobservable latent variable 3 Measurement of Latent Variables Variable A Measure A 4 Measurement of Latent Variables Variable A

Item A1 Item A2 Item A3 Item A4 5 Measure of Energy/fatigue (Vitality) Energy/fatigue Feel tired Feel full of pep

Have a lot of energy Feel worn out 6 Measure, Scale, Index, Item Measure - single- or multi-item scale or index (umbrella term) Scale - aggregation of items from one concept,

scored using accepted scaling method Index - aggregation of 2 or more scales into a summary score Item - a single question or statement including its response scale 7 Types of Measurement Scales Categorical Classification Numbers are simply labels for categories Continuous Ordinal

Interval Ratio 8 Categorical Measures: Numbers Have No Inherent Meaning Race/ethnicity: 1 2 3 4 5 African American Asian Latino

White Other Health insurance: 1 2 3 4 None Private insurance Medicare MediCal 9

Continuous vs. Classification Scores Center for Epidemiological Studies Depression Scale (CES-D) Continuous score (0-60) 20 items on 0-3 scale are summed Classification score (0-1) Those scoring 0-15 = 0 16-60 = 1 (likely to have depression) 10

Ordinal Scales: Numbers Reflect Increasing Level or Amount Severity of pain: 1 2 3 4 5 None Very mild Mild Moderate Severe

Income: 1 2 3 4 < $10,000 $10,000 - <$20,000 $20,000 - <$30,000 >$30,000 Numbers have no inherent meaning but indicate more or less 11 Ordinal vs. Categorical Scales In general, how would you rate your health?

1 poor 2 fair 3 good 4 very good 5 excellent 0 = fair or poor 1 = good, very good, or excellent 12 Measure, Instrument, Battery, Questionnaire, Survey, Tool, Inventory

Measure - single- or multi-item scale or index (umbrella term) Instrument - a published, named measure or set of measures Battery - collection of measures from diverse sources 13 Questionnaire, Survey, Tool, Inventory

Questionnaire - any of the above (instrument, profile, battery) formatted into a set of measures Survey - same as questionnaire Tool = Questionnaire = Survey? Inventory? 14 Examples of Names of Instruments SF-36 Short Form Health Survey Beck Depression Inventory Health Assessment Questionnaire

Sickness Impact Profile Center for Epidemiological Studies Depression Scale 15 Example of a Battery A questionnaire for a study containing the following measures/instruments:

SF-36 (8 domains) Perceived stress scale (1 domain) Social support survey (4 domains) Pain measure Each measure serves purpose (outcome, predictor, mediator, covariate) 16 Multidimensional and Unidimensional A multidimensional measure has scores for each subdomain Unidimensional measure Only one score is obtained

Dimensionality must be empirically tested e.g., factor analysis identifies number of factors or dimensions 17 Example of Unidimensional Scale Perceived Stress Scale (PSS) Single score from all items 14 items, subjective experiences of stress felt confident could handle lifes problems able to control irritations in your life

difficulties piling up so high, could not overcome them Cohen, S, J Health Soc Behav 24:385-396, 1983 18 Example of Multidimensional Questionnaire Patient Satisfaction Questionnaire (PSQ) 55 items, 18 subscales, e.g.

Access to care Technical quality Interpersonal manner Explanations Continuity of care Marshall GN et al., Psychol Assess, 5:477-483, 1993 19 Example of a Multidimensional Instrument SF-36 Measures 8 health concepts (36 items):

Physical functioning Role limitations due to physical health Role limitations due to emotional problems Social functioning Pain Vitality (energy/fatigue) Mental health Health perceptions Also two summary indexes Mental health component Physical health component

20 Overview of Class 2 Measurement terminology Confusion, variation, and overlap Items and response scales Locating measures 21 Composition of an Item During the past month, how

Item stem much of the time have you felt tired? 1 Never Response 2 A little of the time scale 3 Some of the time 4 Most of the time 5 All of the time 22 Things to Pay Attention to in Items Item stem Time frame

Complexity Response scales Type Number of choices Specific choices, distance between choices Format Match of response choices to item stem

23 Time Frame: Part of Concept Definition Past: Average experience over some previous time period (6 months, 3 months, 4 weeks, etc) Present: Current status - today, in general

No time frame: Time frame not specified 24 Response Scale Choices: Vague, Imprecise Quantifiers How often? Very often, pretty often, not too often Sometimes, often, never

How much? Too little, about right, too much Below average, average, above average Bradburn NM. Public Opinion Quart 1979, 92-101. 25 Example of Vague Quantifier For people speaking a language other than English at home: How well does this person speak English? Very well

Well Not well Not at all 26 Example of Vague Quanitifier For people speaking a language other than English at home: How well does this person speak English? Very well Well

Not well Not at all U.S. Census 2000! http://www.census.gov/dmd/www/pdf/07f_or.pdf 27 Types of Response Scales State/level Level or state of behavior, feeling, symptom - Frequency, intensity, severity of pain Evaluative Value attached to level or state - Satisfaction with health

- Bothersomeness of pain Comparative Comparison to prior time, other people - Pain compared to before treatment Agreement Level of agreement with statement - My health is excellent 28 Types of Response Scales for Physical Functioning Items (e.g., walk, climb stairs) State/level

Able/unable Extent of limitation Amount of difficulty Need for help Speed of completing defined task Evaluative Satisfaction with level Comparative Level compared to others your age

Change compared to one year ago Change compared to before treatment 29 Types of Response Scales for Symptoms Items (e.g., nausea, fatigue) Level/state

Evaluative Comparative Had symptom (yes/no) Frequency Intensity/severity (usually, at its worst) Amount of time, # of days had it Extent to which bothered by Sx

Extent of distress due to Sx Extent of change (e.g. frequency) since start of treatment 30 Variation Within State/Level Response Scales Worry about health Intensity (severity) of worry Frequency of worry Amount of time worried 31 Intensity Response Scale Choices

In the past 4 weeks, how worried have you been? 1 Not at all 1 Not at all 2 - Slightly 2 - A little 3 - Moderately 3 - A fair amount 4 - Quite a bit 4 - Much 5 - Extremely 5 - Very much 32 More Intensity Response Scale Choices In the past 4 weeks, how worried have you been? 1 Not at all

1 Not at all 2 - A little 2 - A little 3 - Fairly 3 - Somewhat 4 - Quite 4 - Very much 5 - Extremely 33 Frequency Response Scale Choices How often in the past 4 weeks have you been worried.... 1 - Never 2 - Once or twice 3 - A few times 4 - Often

1 Never 2 - Hardly ever 3 - Some days 4 - Most days 5 - Almost every day 6 - Always, every day 34 More Frequency Response Scales How often in the past 4 weeks have you been worried.... 1 - Never 2 - Almost never 3 - Sometimes 4 - Fairly often 5 - Very often 6 - Always

1 - Once or twice 2 - A few times 3 - Fairly often 4 - Very often 5 - Almost every day 6 - Every day 35 Amount of Time Response Scales How much of the time in the past 4 weeks have you been worried.... 1 None of the time 2 A little of the time 3 Some of the time 4 Most of the time 5 All of the time 36

Feature of Ordinal Scales Distances between numbers unknown, usually vary some closer together in meaning than others 37 Interpretability of Numbers in Single Item Ordinal Scale In general, how would you rate your health? 1 poor 2 fair 3 good 4 very good 5 excellent

Is very good (4) twice as good as fair (2)? 38 Distance Between Levels: In general, how would you rate your health? Self-rated health Raw 1-5 Transf. 0-100 1 poor 2 fair 3 good

4 very good 5 excellent 0 25 50 75 100 Mean 7-item current health scale Screening N=~11,000 10.8 30.0 57.6 75.5 87.9

Stewart and Ware, Measuring Functioning and Well-Being 1992, p. 299. Baseline N=3,054 10.8 30.6 55.9 75.4 86.9 39 Distance Between Levels: In general, how would you rate your health? Self-rated health item:

largest diff 1 poor 2 fair 3 good 4 very good 5 excellent Mean 7-item current health scale Screening Baseline N=~11,000 N=3,054 10.8 30.0 57.6 75.5

87.9 -- 20 --- 26 --- 18 --- 11 -- 10.8 30.6 55.9 75.4 86.9 40 Distance Between Levels: In general, how would you rate your health? Self-rated health item: 1 poor 2 fair

3 good 4 very good 5 excellent Mean: current health scale Screening Baseline N=~11,000 N=3,054 10.8 30.0 57.6 smallest diff 75.5 87.9 -- 20 --- 26 --- 18 --- 11 --

10.8 30.6 55.9 75.4 86.9 41 Response Choice Formats Numeric and non-numeric scales verbal descriptors of all levels verbal descriptors of endpoints only verbal descriptors of endpoints plus midpoint Visual analogue scale

10cm line with endpoints labeled 42 Format: Numeric Scale with Verbal Descriptors of All Levels 1 - All of the time 2 - Most of the time 3 - Some of the time 4 - A little of the time 5 - None of the time 43 Format: Non-Numeric Scale with Verbal Descriptors of All Levels All of the time Most of the time

Some of the time A little of the time None of the time 44 Format: Is This Numeric or NonNumeric? 1 2 3 4 5

All of the time Most of the time Some of the time A little of the time None of the time 45 Format: Bidirectional Numeric Scale, Endpoints and Midpoint Labeled -3 Very much worse -2

-1 0 No change 1 2 3 Very much better 46

Visual Analogue Scale (VAS) 10 cm Line: A Non-Numeric Scale No pain Pain as bad as you can imagine 47 How is a VAS measured? Measure the distance from the no pain end in cm with a ruler

have to measure each survey by hand Precision: measure to the nearest cm, rounding to achieve a whole number from 0-10 or to nearest mm 48 Format: Numeric Scale with Verbal Descriptors of Endpoints Only 0 1 No 2

3 4 5 6 7 8 9 10 Pain

pain as bad as you can imagine 49 Another Numeric Scale with Verbal Descriptors of Endpoints Only 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 No pain Pain as bad

as you can imagine 50 Optimal Number of Response Choices Are 100 levels better than 5? 5-points is optimal for items Especially if items combined into multi-item scales For single-item measures 7 points preferred

Allows distribution over 5 points since people seldom use endpoints 51 Magic Number 7 + 2 (George Miller) Seven chunks of information is the most that can be retained in short-term memory Plus or minus two (5 to 9 chunks) Seven (+ 2) may be the most that people can discriminate on a continuum

Principle has held up well in measurement Miller, GM Psychol Bull, 63:81-97, 1956 52 Can You Find Labels for a 7-point Intensity Scale? How physically active were you this week? 1 - Not at all active 2 - A little bit active 3 - Moderately active 4 - Quite active 5 - Extremely active 53 Can You Find Labels for a 7-point Intensity Scale? How physically active

were you this week? 1 - Not at all active 2 - A little bit active 3 - Moderately active 4 - Quite active 5 - Extremely active 1 - Not at all active 2 - A little active 3 - Fairly active 4 - Quite active 5 - Very active 6 - Extremely active 54 Overview of Class 2

Measurement terminology Confusion, variation, and overlap Items and response scales Locating measures 55 Process of Selecting Good Measures for Your Studies Define concept (variable) Identify potential measures Review measures properties --conceptual adequacy --psychometric adequacy

Pretest best 1-2 measures Select final measure 56 Process of Selecting Good Measures for Your Studies Define concept (variable) Identify potential measures Review measures for --conceptual adequacy --psychometric adequacy Pretest best 1-2 measures Select final measure 57 Locating Potential Measures

Multi-item measures with known psychometric properties Most good measures have been published Original development and testing information Information on application of measure in subsequent publications 58 Locating Potential Measures

Electronic databases Compendia (reviews of measures) Organizations and research centers Government agencies National and state surveys Universities and individual researchers Wikipedia (new) 59 Why Not Just Search on PubMed? Can search PubMed using your concept term and MeSH terms questionnaires, measures, etc. Terminology is not always intuitive

Articles are indexed by most specific MeSH term available Cumbersome! 60 CINAHL (Nursing and Allied Health Literature) CINAHL - on UCSF library home page To find several potential measures Use questionnaires, scales, self-report, or surveys in Major Subject Heading field AND

your topic area in another field (e.g., Abstract) To find articles using a specific measure Use name of measure and Instrumentation field 61 Health and Psychosocial Instruments (HaPI) Evaluation and measurement tools in health and psychosocial studies UCSF Library home page

Databases (shown on bar across top of page) Browse by subject: Social and Behavioral Sciences HaPI Use multifield search e.g. measure and stress (I use the ABSTRACT) 62 Google Scholar http://scholar.google.com/

More practical for searching on vague terms Follow up on PubMed 63 Consider Alternative Labels to Search Search on your term see what other terms come up Housing instability Homelessness

Clinical assertiveness Interpersonal behavior 64 Locating Published Measures: Compendia Specific measures of various concepts are compiled, reviewed, listed, or otherwise provided Many books and some special journal issues review and critique various measures Class 2 Handout Measures compilations and reviews annotated

bibliography 65 Best Compendium McDowell I and Newell C, Measuring Health: A Guide to Rating Scales and Questionnaires. Third Edition, Oxford University Press, 2006. Reviews measures of physical disability, social health, psychological well-being, anxiety, depression, mental status, pain, and general health status and quality of life Includes actual instruments where possible and selected items in some 66

Reviews of Measures: Examples Structured reviews: AC Mui et al. Cross-cultural assessment of geriatric depression: A review of the CES-D and the GDS. J Ment Health Aging, 2001;7:137-164. SM Martinez et al. A review of physical activity measures used among U.S. Latinos: guidelines for developing culturally appropriate meausres. Ann Beh Med, 2008;36:195-207. Narrative reviews: JH Skinner. Acculturation: measures of ethnic accommodation to the dominant American culture. J Ment

Health Aging, 2001;7:41-52. 67 Compendia by Web National Cancer Institute Health behavior constructs: theory, measurement, and research Reviews concepts and measures of constructs such as perceived control, self-efficacy, social support, and perceived vulnerability https://cancercontrol.cancer.gov/brp/research/constructs/index.html 68

Locating Measures: Organizations and Research Centers Some organizations and centers specialize in measurement development RAND Michigan Diabetes Research and Training center MacArthur Research Network on Socioeconomic Status and Health 69 RAND Health Program Actual measures, scoring manuals, and publications for measures of:

Health-related quality of life Quality of care, patient satisfaction Mental health Maternal, child, and adolescent health Medical Outcomes Study (MOS) measures http://www.rand.org/health/surveys_tools.html 70 RAND Measures of Maternal, Child, and Adolescent Health

Pediatric Asthma Symptom Scale Pediatric Quality of Life Inventory (PedsQL Measurement Model) Trauma tool-kit for schools 71 Society of General Internal Medicine Dataset compendium has links to numerous datasets National Survey of Childrens Health (NSCH)

Each link has section for questionnaires http://www.sgim.org/communities/research/dataset-compendium 72 Michigan Diabetes Research and Training Center: Survey Instruments Diabetes Care Profile Diabetes History Diabetes Knowledge Test Diabetes Attitude Scale Diabetes Empowerment Scale Risk perception survey for developing diabetes

http://diabetesresearch.med.umich.edu/Tools_SurveyInstruments.php 73 MacArthur Research Network on Socioeconomic Status and Health Reviews measures in several domains: Psychosocial Social and physical environment Socioeconomic status (SES) SES across the lifecourse http://www.macses.ucsf.edu/ 74 Examples of MacArthur Network

Psychosocial Measures Anxiety Coping strategies Discrimination Hostility Optimism/pessimism

Personal control Psychological stress Self-esteem Social support Vitality and vigor http://www.macses.ucsf.edu/research/psychosocial/ default.php 75 Ottawa Health Decision Centre Patient and physician decision aids http://www.ohri.ca/centres/DecisionAids/default.asp

Patient measures Decisional conflict, decisional regret, stage of decision making, decision self-efficacy, and others http://decisionaid.ohri.ca/eval.html 76 Commonwealth Fund: Surveys and Data Health insurance, Medicare Health system performance Health care quality, patient centered care,

underserved populations Child health, care of the elderly http://www.commonwealthfund.org/ 77 Locating Measures: Government Agencies Several federal and state government agencies provide measures Information about measures (e.g. clearinghouse) Actual measures 78 Centers for Disease Control and Prevention

Behavioral Risk Factor Surveillance System (BRFSS) Surveys of health behaviors www.cdc.gov/brfss/questionnaires/index.htm 79 BRFSS Some published studies of BRFSS measures Fruit and vegetable score and fat score Validity tested in minority populations Correlated equally well with diet records in African

American and White populations M Serdula, Epidemiology. 1993;4(5):455-63. Did not perform as well in lower-income Hispanic women as in other groups RJ Coates et al. Journal of Nutrition. 1995;125:1473-83. 80 CDC: National School-Based Youth Risk Behavior Survey (YRBS) Survey conducted every other year

random national sample of youth in grades 9-12 Most states conduct survey Measures of substance use, risky sexual behaviors, diet, physical activity, overweight http://www.cdc.gov/HealthyYouth/yrbs/index.htm 81 California Health Interview Survey (CHIS) A behavioral risk surveillance survey modeled after the CDC BRFSS

Adults, adolescents, and children Numerous languages http://healthpolicy.ucla.edu/chis/design/Pages/questionnairesEnglish.aspx 82 U.S. Dept of Veterans Affairs National Center for Post Traumatic Stress Disorder http://www.ptsd.va.gov/ Assessment/Instruments Trauma Exposure Measures

PTSD Screens Adult Self Report 83 Agency for Healthcare Research and Quality (AHRQ) National Quality Measures Clearinghouse Consumer Assessment of Health Plans Survey (CAHPS) http://www.ahrq.gov/qual/ 84

National Cancer Institute Special initiatives on measures Health Information National Trends Survey (HINTS) Measures of cancer knowledge, cancer communication, risk perception http://hints.cancer.gov/ 85 CDC: National School-Based Youth Risk Behavior Survey (YRBS) Survey conducted every other year random national sample of youth in grades 9-12

Most states conduct survey Measures of substance use, risky sexual behaviors, diet, physical activity, overweight http://www.cdc.gov/HealthyYouth/yrbs/index.htm 86 National Center for Health Statistics: Population Surveys National Health Interview Survey National Health and Nutrition

Examination Survey National Maternal and Infant Health Survey Longitudinal Studies of Aging (LSOA) https://www.cdc.gov/nchs/surveys.htm#tabs-2-1 87 Locating Measures: Universities and Individual Researchers Individual investigators who specialize in measurement and some universities post measures Some are large research studies on university websites

88 Tool Kit of Instruments to Measure End of Life Care (TIME) Annotated bibliographies of instruments that measure outcomes of care at the end of life Examples: Quality of life

Pain and other symptoms Spirituality Caregiver well-being Grief and bereavement http://www.chcr.brown.edu/pcoc/bibliographies.htm 89 Dr. James Sallis, Ph.D., San Diego State University Expert in physical activity and diet interventions in children Provides nearly 50 relevant measures English and Spanish Parent and child measures Influences on physical activity, environment

Need to search literature to find publications on the measures http://sallis.ucsd.edu/measures.html 90 Locating Measures: Finding Authors of Measures Published research using measure you are interested in Unpublished measures often described in methods Authors may provide measures

91 Wikipedia Some wikis are being developed around specific measures http://en.wikipedia.org/wiki/Hamilton_Depression_Rating_Scale http://en.wikipedia.org/wiki/Beck_Depression_Inventory 92 Homework Due by Class 3 Locate at least 3 measures/instruments that

assess your concept For each one list: Name of instrument/measure Citation for original publication Where you found it (process and location) Send by email to [email protected] Put your last name in the file name 93

Recently Viewed Presentations

  • A Short History of Beer - Brewing &amp; Distilling Center

    A Short History of Beer - Brewing & Distilling Center

    With the waning prominence of Greece, Rome became the center of the "known world". Rome followed suit with Greece in that they saw wine as a high status drink and brewed beer as low class and barbaric. However, there were...
  • Protists and Fungi 2 Diverse Kingdoms of Life

    Protists and Fungi 2 Diverse Kingdoms of Life

    1) Heterotrophs with no permanent locomotor apparatus Mostly unicellular and ameoba-like: Phylum Rhizopoda (amoebas) Phylum Actinopoda (radiolarians) Phylum Foraminifera (forams). Phylum Rhizopoda (amoebas) Move by pseudopods (flowing extensions of cytoplasm) Lack sexual reproduction, cell walls, flagella Reproduce asexually only Phylum...
  • Social Psychology

    Social Psychology

    Social Assumptions, People Perception & Attribution Theory-Some guy. Definition. Social Psychology: Is the study of how we think about, influence, and relate to one another.
  • -NEOPLASIA-212 (2012-2013) Dr.H.M.Zahawi,FRC.Path Definitions :  Neoplasm = New

    -NEOPLASIA-212 (2012-2013) Dr.H.M.Zahawi,FRC.Path Definitions : Neoplasm = New

    -NEOPLASIA- -212 - (2012-2013) Dr.H.M.Zahawi,FRC.Path Multi-Step Theory of Cancer : Sometimes : Normal→Hyperplasia→Metaplasia→Dysplasia→ Neoplasia → Metastases Transformed Cell CARCINOGENIC AGENTS 1- CHEMICAL Carcinogens 2- PHYSICAL Carcinogens 3- VIRAL Carcinogens 4- MICROBIAL Carcinogen Direct Carcinogens - Directly produce damage without prior...
  • Nellcor™ Bedside SpO2 Patient Monitoring System, PM100N

    Nellcor™ Bedside SpO2 Patient Monitoring System, PM100N

    Activate Nellcor™ SatSeconds Alarm Management . Nellcor™ SatSeconds alarm management is designed to help reduce clinically insignificant alarms that occur with your patient. Activate Nellcor™ SatSeconds alarm management and set to 100. How would you respond to a Nellcor™ SatSeconds...
  • Reading / Beginning Reading Skills / Phonics. Students

    Reading / Beginning Reading Skills / Phonics. Students

    decode multisyllabic words in context and independent of context by applying common spelling patterns including dropping the final "e" and add endings such as -ing, -ed, or -able (e.g., use, using, used, usable).[3.1Ai]
  • Citizens Insurance Co. v. Parsons, 1881

    Citizens Insurance Co. v. Parsons, 1881

    Citizens Insurance Co. v. Parsons, 1881 Impugned: Ontario Fire Insurance Policy Act. Fire in Parsons' warehouse. Parsons wanted insurance payment Ins Co: you didn't observe the fine print. Parsons: the fine print didn't conform to the Act. Ins Co: The...
  • Feasibility Study - STEM

    Feasibility Study - STEM

    The South Florida Industry Partners Education Mousetrap Car Design Seminar Presented by: Glenn N. Cunningham, P.E. Senior Associate Hazen and Sawyer, P.C.