SemanticFind: Locating What You Want in a Patient Record, Not ...

SemanticFind: Locating What You Want in a Patient Record, Not ...

SemanticFind: Locating What You Want in a Patient Record, Not Just What You Ask For John M. Prager, Jennifer J. Liang, Murthy V. Devarakonda IBM T.J. Watson Research Center AMIA Joint Summits, San Francisco CA, March 31 2017 Overview What is SemanticFind? What it is not How is SemanticFind different from Crtl/F? 13 match types 4 technologies Prototype User Interface Evaluation Conclusions What is SemanticFind? An application that aids the physician search a patients medical record for matches to terms of interest Extension of the familiar Ctrl/F find capability in document creation and reading applications Not limited to matching solely the entered

search term Can find matching content along a variety of different dimensions What it is Not Not an application designed for finding matching records (patients) amongst a large collection, i.e. clinical trial matching Can be done, just not optimized for this Not a presentation of a new user interface Prototype GUI is used in subsequent slides for demonstration of functionality Not a Question-Answering System So how is it different from Ctrl/F Search terms represent information needs, but 1. Information needs cannot usually be answered fully just by locating instances of them in text 2. Ambiguous intent behind the search term E.g. if the search term is a disease, user might be wondering

When/how it was first diagnosed What indicative labs for it were over time Are there counterindications Are there complications What treatments were prescribed for it Approach is to perform a variety of different searches simultaneously, and present the results organized by search type EHR systems have both structured information (e.g. tables of orders, diagnoses, lab results) and unstructured (e.g. progress and other notes, free text) Both kinds are searched Search terms are unrestricted: symptoms, diseases, medications, anatomical structures, any sequence of characters. Search is mediated by UMLS, so search terms that correspond to concepts in UMLS can be more fully explored Sample Screenshot: Pain in the abdomen SemanticFind Search Types (1) Trad. Search Search Type Explanation

Comments Literal Traditional find Ctrl-F Includes plurals Semantic Hypothetical Returns terms with same semantic meaning as search term Returns terms in negative context, incompatible with or opposite of the search term Returns terms that are not being asserted to apply to patient More general Returns terms that are broader than search term More specific

Returns terms that are narrower than search term Assoc. Procedures Returns procedures associated with search term Assoc. Tests Returns tests or findings associated with search term Assoc. Treatments Returns treatments associated with search term Assoc. Medications Returns medications related with this search Contradicted

Conceptual Search Associative Search Enables you to search for concepts without regard to clinical terminology represented in record or level of specificity Returns terms that are often related to the search term in the medical literature. IMPORTANT: Not necessarily related in this patient 7 SemanticFind Search Types (2) Search Type Ordered Medications Possible Med

Concerns Inferential Search Ordered Procedures Explanation Comments Returns medications from medication orders logically connected to search term. The medication often treats or prevents the condition (if searching for disease) Returns medications from medication orders that are known to be contraindicated with the search term Returns procedures from the procedure orders logically connected to the search term. The procedure often treats or prevents the condition (search term) Returns labs from lab orders that are logically connected to the search term. Searches through the Orders in

the patient record (structured data) to infer potential relations to the search term, based on known relationships in the medical literature. The explanation field shows the logical relation between the search term and the matched item IMPORTANT: Not necessarily related in this patient Ordered Labs The labs often aid in diagnosing or monitoring a condition (search term) 8 Technologies Used Literal Match As traditional search, but case-insensitive and disregards singular/plural Conceptual Match UMLS concepts and relations for Semantic, More General/Specific Our own Medical Concept Annotator, conceptually similar to cTAKES and MetaMap, but higher accuracy (paper in preparation)

Lab values and vitals mapped to indicated conditions K 6.1 gets annotated as hyperkalemia Parse- and Linguistic-principle-based transformations to catch semantically matching concepts/variants in UMLS Pain in the abdomen not a variant of abdominal pain NLP for Negation and Hypothetical Patient denies discomfort with the rash Ordered urine test to rule out arsenic poisoning Associative Match Uses Latent Semantic Analysis Finds terms in the record that occur in the same contexts in the literature as the search terms Useful for finding terms correlated with search term, but no named relation, e.g. sob wheezing Inferential Match Finds terms in the record that are related through curated relation chains to the search term Most useful for , , relations, e.g. Infection Lower respiratory tract infection Amoxicillin Augmentin 875 mg-125 mg tablet Semantic Search Examples of Search Types Search Term Example Matches

Type of Match Uterus Uterus, uteri Literal Shortness of breath Difficulty breathing, SOB, dyspnea Semantic High blood pressure hypertension, HTN, BP 150/100, elevated BP Semantic Normal blood pressure Hypertension, HTN, BP 90/60, low BP Contradicted Match

Tobacco Patient denies smoking Contradicted Match DM Risk of diabetes Hypothetical Match Hyperlipidemia Hypercholesterolemia More Specific PTSD Anxiety disorder More General Leg pain Pain in the lower limb

More Specific Pain in the lower limb Leg pain More General Insulin Sugar, Novolin, Metformin, Glucose Assoc. Medications Sertraline Asthma Risperidone, cyproheptadine, Zoloft, Antipsychotics, Assoc. Medications Depakote Cardiac Surgery, Coronary Artery Bypass, Percutaneous Assoc. Treatments Coronary Intervention, PCI Albuterol Ordered Meds

Chest Pain Electrocardiogram Heart attack Ordered Procedures 10 Evaluation 3 types suitable for evaluation Semantic Match More Specific Contradicted 10 records selected at random Average of 250 clinical notes per record MD developed list of (13-32) search terms for each Total of 169 terms, 134 unique 4th-yr medical students used as assessors Assessors generated a list of paraphrases for each search term 0-13 per term. Total of 652. Based on medical knowledge and/or lookup, not seeing medical records. Assessment task, per search term

SemanticFind used interactively to locate matches Precision: GUI enhanced with evaluation widgets for assessors to enter judgments of GOOD or BAD #GOOD = True Positives (TP) #BAD = False Positives (FP) Precision = TP/(TP+FP) Recall: System automatically searched for user-generated paraphrases (via Literal Match), and counted how many of these did not correspond to GOOD in Precision task. This count = False Negatives (FN) Recall = TP/(TP+FN) F-Measure F = 2PR/(P+R) Evaluation Interface Results (1) Precision Precision Batch Overall True Positives

11851 False Positives 1728 #matches judged 13579 Precision 0.87 Error analysis shows most FPs due to ambiguity of abbreviations negation detection error Recall: 2 modes evaluated Unconstrained = all supplied paraphrases Constrained = only those paraphrases that matched UMLS concepts Recall Mode Unconstrained Constrained

True Positives 11851 11851 False Negatives 1704 297 Recall 0.87 0.98 F-Measure Unconstrained/Constrained = 0.87/0.92 Results (2) Progressive analysis of GOOD matches: Relative to Literal Match as a baseline Semantic Match Corresponds (very roughly) to Ctrl/F + Synonym Expansion

Semantic Match + More Specific Semantic Match, More Specific + Contradicted Search types considered % GOOD matches as compared to Literal Match alone Literal Match alone 100% Semantic Match, relative to LM 121% SM + More Specific, relative to LM 190% SM + MS + Contradicted Match, relative to LM 203% 103% Dark Matter Interesting Negation Detection Error Due to somewhat informal formatting/writing of clinical notes, e.g.:

alcohol use : no smoking : yes Implicit sentence-end clear to humans, but not to computer, giving rise to recognition of no smoking On fixing problem, reduced false positives by 30% Conclusions SemanticFind is application to search within a patient record 13 searches performed simultaneously using a variety of NLP technologies Organised in a tabbed interface High accuracy F = 0.87 or 0.92 Est. 2 points higher when sentence-end problem fixed Dark Matter calculation shows that Ctrl/F misses as many desirable matches as it finds THANK YOU

Recently Viewed Presentations

  • How Elements Form Compounds - Ms. Anderson's Room 280

    How Elements Form Compounds - Ms. Anderson's Room 280

    How Elements Form Compounds. ... This compound's name is. Calcium has two extra electrons that it will share with oxygen. Ca+2 O-2 ... Build the following molecules with your ion fit kit and then give them a name in your...
  • Treatment of Stiff-Person Syndrome, Myoclonus & Painful-Legs ...

    Treatment of Stiff-Person Syndrome, Myoclonus & Painful-Legs ...

    Baclofen. Oral: 5 mg 3 times/day, may increase 5 mg/dose every 3 days to a maximum of 80 mg/day . Intrathecal: Test dose: 50-100 mcg, doses >50 mcg should be given in 25 mcg increments, separated by 24 hours. A...
  • Year 9 GCSE History Lesson 10 Sunday, February

    Year 9 GCSE History Lesson 10 Sunday, February

    Roger de Breteuil, Earl of Hereford. Roger was the son of William . FitzOsbern, lord of Breteuil in Normandy and the same man that William had rewarded so lavishly in 1066: making him Earl of Hereford, a Marcher earldom with...
  • Ethics and Deviance in Sport part 1 - A Level PE

    Ethics and Deviance in Sport part 1 - A Level PE

    Deviance. Deviance in sport is often a result of the drive to win. Ethics rules that dictate an individual conduct. Gambling in sport has become more sophisticated.. Violence . behaviour of players and spectators a feature of modern sports.
  • Equilibrium shape of hydrogen terminated silicon

    Equilibrium shape of hydrogen terminated silicon

    Equilibrium shape of hydrogen terminated silicon 2 Hypothesis Surface diffusion Directional attachment Surface Diffusion Surface Diffusion Program: HcParcas Classical potential: Tersoff III Equilibrium shape of hydrogen terminated silicon Surface Diffusion Surface Diffusion Program: HcParcas Classical potential: Tersoff III Equilibrium shape...
  • Students Wellbeing and Social Connections Tom Strong, Michelle

    Students Wellbeing and Social Connections Tom Strong, Michelle

    Examples. FRIENDS - Anxiety prevention. The Fourth R -Bullying and Substance Use (Healthy, Safe, Connected) School Mental Health (Dr. Stan Kutcher) - Mental Health awareness. Gatehouse Project (Australia) Emotional Well-being. Lions . Quest - School Climate . Skills for Growing,...
  • Snap-Shot

    Snap-Shot

    Sensory Details Lets the reader SEE what is happening! Zoom in close to physical detail Listen to sounds and speech Paint a feeling without saying a word Paint a setting or description of a place Shows a characters movements and...
  • Introducing The Honeyberry By Bernis Ingvaldson The Honeyberry

    Introducing The Honeyberry By Bernis Ingvaldson The Honeyberry

    Haskap Market Development -The Japanese Opportunity- Feasibility Study August 2007 √Čric B. Lefol, PhD, PAg 120 tons of in 2005. Growing Haskap / Blue honeysuckle in Canada Dr. Bob Bors, Sightings reported on Iron Range in Minnesota What is it?...