How to write a multiple choice Question

How to write a multiple choice Question

How to write a Multiple Choice Question Matt Kelleher Brad Mathis paraphrase.. Doing multiple choice questions is nothing like taking care of patients but writing a multiple choice question is very similar Table 7.1 Constructed-Response Selected-Response

Strengths Non-cued Easy to create Logic, reasoning, steps in problem solving Ease of partial credit scoring In-depth assessment Broad representative content Accurate, objective and

reproducible scores Defensiblity Accurate, timely feedback Reusable Efficient (time, cost, info) Weakness Difficult to write well

Bad public relations (guessing, memorable) Subjective human scoring Limited breath of content Reproducibility issues Inefficient (scoring, testing, information) Limited psychometrics Limited quality control Downing and Yudkowsky. Assessment in Health Professionals Education page 150 Selected Response Examples

Traditional MCQs True-False Alternate-Choice Multiple True-False Traditional Matching Extended Matching Complex Multiple-Choice Testlets or Context-Dependent Item Sets Multiple Choice Questions (MCQs) There are 4 main components to every MCQ. Objective important part and best place to start Stem and Lead-in Options

Explanation Best Evidence in practice for creating effective and defensible MCQs Content/STEM 1. Focused, items should test specific content/objective and single specific mental behavior 2. Based on important content; avoid trivial content 3. Avoid using exact textbook language buzz words (recall/recognition) 4. Content of each item should be independent 5. Avoid over-specific or over-general language (usually, frequently, commonly, often, always, never) 6. No opinion based items Table 7.3 Taxonomy of Multiple-Choice Item Writing Guidelines. Adapted summary from Haldyna, Downing, and Rodriquez 2002

Best Evidence in practice for creating effective and defensible MCQs - STEM 7. Should NOT be tricky or unnecessarily complex (thyroid ex.) 8. Keep vocabulary simple (avoid all abbreviations/acronyms) 9. Be very clear and direct in the stem question (i.e. which of the following is the MOST likely diagnosis) best next vs next best 10. Central idea should be in the stem, not the options 11. Avoid window dressing (excessive verbiage, as short as possible, all sentences should be relevant) 12. Word the stem positively (avoid NOT, EXCEPT, etc.) Table 7.3 Taxonomy of Multiple-Choice Item Writing Guidelines. Adapted summary from Haldyna, Downing, and Rodriquez 2002 A few more of my rules

1. Be as realistic as possible (case-based) 2. Should be answerable without options OR options must be 100% true or false 3. Patients in MCQ dont lie 4. Try to test the application of knowledge (avoid recall of facts) ..if while writing the question you think this is a straightforward concept how can I make it complicated, this is a reason to pause! Best Evidence in practice for creating effective and defensible MCQs - OPTIONS As many effective choices as you can (research says three is adequate)

One indisputably correct option (doesnt mean other options are completely wrong) Randomize choices Choices should be related but independent, not overlapping Homogenous choices

in content, grammar, structure Keep length of choices equal Avoid none or all of the above Phrase choices positively (avoid negatives) All distractors should

be plausible but clearly incorrect Use typical errors of learners to write distractors Avoid clues always, never, completely, absolutely clang associations (choices are identical or resemble words in stem) grammatical inconsistencies conspicuous correct choice pairs, triplets of options

absurd or ridiculous options Table 7.3 Taxonomy of Multiple-Choice Item Writing Guidelines. Adapted summary from Haldyna, Downing, and Rodriquez 2002 Explanation Be very clear and concise Explain why the correct option was correct first (Main Explanation) For each distractor/incorrect answer briefly explain why it was specifically incorrect When possible attach a reference for your source and where someone can go to learn more Whats wrong?

Which of the following is true about pseudogout? A. It occurs frequently in women. B. It is seldom associated with acute pain in a joint. C. It may be associated with a finding of chondrocalcinosis. D. It is clearly hereditary in most cases. E. It responds well to treatment with allopurinol. Whats wrong? Too Vague? Not focused Which of the following is true about pseudogout?

A. It occurs frequently in women. B. It is seldom associated with acute pain in a joint. C. It may be associated with a finding of chondrocalcinosis. D. It is clearly hereditary in most cases. E. It responds well to treatment with allopurinol. Whats wrong? A 60-year-old man is brought to the emergency department by the police, who found him lying unconscious on the sidewalk. After ascertaining that the airway is open, the first step in management should be intravenous administration of A. examination of cerebrospinal fluid

B. glucose with vitamin B1 (thiamine) C. CT scan of the head D. phenytoin E. diazepam Whats wrong? A 60-year-old man is brought to the emergency department by the police, who found him lying unconscious on the sidewalk. After ascertaining that the airway is open, the first step in management should be intravenous administration of A. examination of cerebrospinal fluid B. glucose with vitamin B1 (thiamine) C. CT scan of the head D. phenytoin

E. Diazepam Example: the first step should be the intravenous administration of which medication?.(several options and maybe two meds would work but one comes in IV preparation and one does not) Testing the wrong construct! Ms. James is a 57 yo AAF with PMH DM-2 (a1c 8.2%), HLD, HTN, and obesity. She presents to your office for follow-up. She has no complaints. She takes ASA 81 daily, atorvastatin 40 daily recently increased, and metoprolol tartrate 12.5 bid but sometimes forgets to take metoprolol. BP is 154/82, HR 87 today. Previous blood pressures the last three visits over the year have been 148/79, 158/86, and 156/90. She has trace swelling of the lower extremities and PE is otherwise unremarkable. What is the next best step in managing Ms. James BP? A. Encourage diet and lifestyle modification and switch metoprolol 12.5 bid to HCTZ 25 daily. B. Encourage diet and lifestyle modification and increase metoprolol from 12.5

bid to 25 bid. C. Encourage diet and lifestyle modification and switch metoprolol 12.5 bid to HCTZ 25 daily and amlodipine 10 daily. D. Switch metoprolol 12.5 bid to Lisinopril 40 daily. E. Start amlodipine 2.5 daily. Ms. James is a 57 yo AAF with PMH DM-2 (a1c 8.2%), HLD, HTN, and obesity. She presents to your office for follow-up. She has no complaints. She takes ASA 81 mg daily, atorvastatin 40 daily recently increased, and metoprolol tartrate 12.5 bid but sometimes forgets to take metoprolol. BP is 154/82, HR 87 today. Previous blood pressures the last three visits over the year have been 148/79, 158/86, and 156/90. She has trace swelling of the lower extremities and PE is otherwise unremarkable. What is the next best step in managing Ms. James BP? A. Encourage diet and lifestyle modification and switch metoprolol 12.5 bid to

HCTZ 25 daily. B. Encourage diet and lifestyle modification and increase metoprolol from 12.5 bid to 25 bid. C. Encourage diet and lifestyle modification and switch metoprolol 12.5 bid to HCTZ 25 daily and amlodipine 10 daily. D. Switch metoprolol 12.5 bid to Lisinopril 40 daily. E. Start amlodipine 2.5 daily. Whats wrong? In patients with advanced dementia, Alzheimers type, the memory defect A. can be treated adequately with phosphatidylcholine (lecithin) B. could be a sequela of early parkinsonism C. is never seen in patients with neurofibrillary tangles at autopsy

D. is never severe E. possibly involves the cholinergic system Whats wrong? In patients with advanced dementia, Alzheimers type, the memory defect A. can be treated adequately with phosphatidylcholine (lecithin) B. could be a sequela of early parkinsonism C. is never seen in patients with neurofibrillary tangles at autopsy D. is never severe E. possibly involves the cholinergic system Main problem with this question is the lack of focus in the stem! The verb should be in the lead in of the stem not the options.

Whats wrong? A 58-year-old man with a history of heavy alcohol use and previous psychiatric hospitalization is confused and agitated. He speaks of experiencing the world as unreal. This symptom is called A. depersonalization B. derailment C. derealization D. focal memory deficit E. signal anxiety Whats wrong? A 58-year-old man with a history of heavy alcohol use and previous psychiatric hospitalization is confused and agitated. He speaks of experiencing the world as unreal. This symptom is called

A. depersonalization B. derailment C. derealization D. focal memory deficit E. signal anxiety Whats wrong with these options? Secondary gain is 1. synonymous with malingering 2. a frequent problem in obsessive-compulsive disorder 3. a complication of a variety of illnesses and tends to prolong many of them 4. never seen in organic brain damage

Whats wrong with these options? Secondary gain is 1. synonymous with malingering 2. frequent problem in obsessive-compulsive disorder 3. a complication of a variety of illnesses and tends to prolong many of them 4. never seen in organic brain damage Longer, more specific, more complete one is correct It is also the only double option Stem critique A 60 yo man with pmhx of htn, dm, hld and seasonal allergies presents with difficulty swallowing. He has

difficulty with solids only. He only has trouble swallowing meat and potatoes but not juice or water. He states, this has been going on for about 2 years. He also has recurrent food impaction and occasional burning. He takes hydrochlorothiazide, lantus, Crestor and Claritin. His great-grandfather had lung cancer. He claims to not smoke or drink. What is the most likely diagnosis? 30 year old man with a history of seasonal allergies and recurrent food impaction

presents to clinic with dysphagia to solids. He denies a history of heartburn. He does not smoke or drink alcohol. What is the most likely diagnosis? Objective: Diagnose Eos Esophagitis Answer options: A. GERD induced peptic stricture B. Eosinophilic esophagitis C. Achalasia D. Esophageal cancer Whats wrong? Which of the following is correct regarding when to initiate antibiotic prophylaxis in an HIV infected patient?

A. CD4 conuts <50 starts Bactrim B. CD4 counts <50 start Azithromycin C. Initiate azithromycin when the CD4 count is less than 150 D. CD4 count <100 start isoniazid E. CD4 count <200 start azithromycin F. None of the above correct answer is the option that has the most in common with the other options Whats wrong with these options? 1. Admit to general medicine floor for hypertensive emergency/crisis and restart home medications HCTZ 25 daily and Lisinopril 40 daily, lower DBP to <160/90 mmHg in 16 hours. 2. Admit to ICU for hypertensive emergency/crisis and start nicardipine

5mg/hour and titrate to lower DBP by 10-15% (or 110mmHg) in 30-60 minutes. 3. Admit to ICU for hypertensive emergency/crisis and bolus with Lasix 80mg IV x 1 and start Lasix gtt at 10mg/h. 4. Admit to general medicine floor for hypertensive urgency and use hydralazine 10mg iv every 4-6 hours PRN for SBP >180 or DBP >105 mmHg. 5. Admit to ICU for hypertensive emergency/crisis and start nicardipine 5mg/hour and titrate to lower DBP by 25% over 16 hours. Whats wrong? Following a second episode of PID, what is the likelihood that a woman will become infertile? A. Less than 20% B. 20 to 30%

C. Greater than 50% D. 90% E. 75% Whats wrong? Severe obesity in early adolescence A. usually responds dramatically to dietary regimens B. often is related to endocrine disorders C. has a 75% chance of clearing spontaneously D. shows a poor prognosis E. usually responds to pharmacotherapy and intensive psychotherapy Research has shown that vague frequency terms are not consistently defined or interpreted, even by experts.

A 51 yo Caucasian male presents to your office for follow-up of slightly elevated aminotransferases. His past medical history includes obesity (BMI 40), hyperlipidemia, and type 2 diabetes mellitus. He drinks 4-5, 12 oz. beers on Sundays. Medications include metformin 1000 mg twice daily, atorvastatin 40 mg daily, HCTZ 12.5 mg daily, and aspirin 81 mg daily. Vital signs: afebrile, HR 79, BP 138/94. Physical exam reveals normal cardiac and pulmonary exam, notable central adiposity. Labs: WBC 4.5, Hgb 13.7, Platelets 129,000, AST 68, ALT 32, Total protein 7.2, Total bilirubin 1.4, Alkaline phosphatase: 145, Albumin 3.5, INR 1.2, Hgb A1c 7.9%, Na 139, K 3.7, Cl 105, HCO3 25, BUN 18, SCr 0.9, Ca 8.2, Mg 1.9 Right upper quadrant ultrasound is pending. What would you tell this patient regarding the diagnosis of non-alcoholic fatty liver disease (NAFLD)? A. Recommend lifestyle modification including weight loss of 5-10%, reduce alcohol intake, continuation of statin and metformin for hyperlipidemia and diabetes mellitus, respectively, and start vitamin E.

B. Recommend lifestyle modification including weight loss of 5-10%, reduce alcohol intake, continuation of statin and metformin for hyperlipidemia and diabetes mellitus, respectively, and start pioglitazone. C. Recommend lifestyle modification including weight loss of 5-10%, reduce alcohol intake, continuation of metformin for diabetes mellitus and stop atorvastatin. D. Recommend lifestyle modification including weight loss of 5-10%, reduce alcohol intake, continuation of

statin and metformin for hyperlipidemia and diabetes mellitus, respectively, and he may need a liver biopsy. E. NAFLD is a less common cause of liver disease in the United States and he should not be worried about this. Questions? What do you think about the stem? Diagnose multiple myeloma 66 year black man is seen in clinic complaining of back pain that is made worse with movement. He has no signs of sensory loss, bowel/bladder incontinence, radiculopathy or neuropathy. He has had a 6lb weight loss, generalized weakness, and fatigue. Spinal Xrays reveal

a lytic lesion. Which of the following diagnostic tests would be best to order next?

Recently Viewed Presentations

  • Iowa State University Extension Dr. Robert Wisner, University

    Iowa State University Extension Dr. Robert Wisner, University

    Total short-fall: 1,850 to 1,960 mil. Bu. Potential carryover decrease, '07-08: 0.0 Needed extra harv. acres: 11.6-12.3 Mil. A. @ 160 Bu./A. on all acres (U.S. +13% to +14%) Still Leaves Potential Very Tight Supply Implications of Emerging Energy Market...
  • Session 2 - The Spirit World In this

    Session 2 - The Spirit World In this

    Because of Lucifer's pride he fell, same thing for humans! The Bible says that 1/3rd of the angels . fell with Lucifer and became fallen . angels, or demons. Proverbs 8:13 "The fear of the Lord is hatred of evil....
  • Definition of Mixing Zone

    Definition of Mixing Zone

    Complete Mix. Far field mixing. Concepts can be difficult - e.g., mixing in 1D, 2D, 3D. Some tricky and inconsistent language out there. Ask for clarification! Making sense of it . What is a Mixing Zone? EPA's TSD for Water...
  • Tools for identifying duplicate files and known software files

    Tools for identifying duplicate files and known software files

    NSRL Reference Data Set (RDS) Hashsets and metadata used in file identification. Data can be used in third-party digital forensics tools. 2017 BitCurator User Forum - Tools for identifying duplicate files and known software files
  • WebQuest Template - taylor.k12.ky.us

    WebQuest Template - taylor.k12.ky.us

    Task 2: Alliteration, Refrain, and Rhyme Scheme Alliteration Refrain Rhyme Scheme The repetition of an initial (beginning) consonant sound in two or more words near one another The repetition of one or more phrases or lines usually at the end...
  • Title Slide - OCWD

    Title Slide - OCWD

    PLANT PRODUCTION With the GWRS initial expansion production is 103,000 acre feet per year Production being maximized through OCSD's operation of the Steve Anderson Lift Station (SALS) and the Secondary Effluent Flow Equalization GWRS continues to operate with a high...
  • Africas Economic Systems South Africa, Nigeria, and Kenya

    Africas Economic Systems South Africa, Nigeria, and Kenya

    Economic decisions are made based on the changes in prices that occur as buyers & sellers interact in the market place. The government has no control over the economy; private citizens answer all economic questions. In a truly free market...
  • Biology 1110 Principles of Biology

    Biology 1110 Principles of Biology

    Euglena sp. Organelles/structures to identify. Flagellum. Chloroplasts. Nucleus. Stigma. Cytostome. Contractile vacuole. Pellicle. In the listing and on the diagram you will find the structures/organelles we will be observing in the micrographs of stained and living amoebas. Clicking the forward...