Focused Entailment Graphs for Open IE Propositions

Focused Entailment Graphs for Open IE Propositions

Focused Entailment Graphs for Open IE Propositions Omer Levy Ido Dagan Jacob Goldberger Bar-Ilan University, Israel Open IE Extracts propositions from text which makes aspirin relieve headaches. No supervision No pre-defined schema Whats missing in Open IE? Structure

Open IE does not consolidate natural language expressions relieve headache treat headache Adding Structure to Open IE Which structure? Build a graph of Open IE propositions and their semantic relations Adding Structure to Open IE Which structure? Build a graph of Open IE propositions and their entailment relations Why entailment? Merges paraphrases into mutual entailment cliques aspirin relieves headache aspirin treats headache

Organizes information hierarchically from specific to general aspirin relieves headache painkiller relieves headache aspirin, eliminate, headache aspirin, cure, headache coffee, help, headache drug, relieve, headache headache, control with, aspirin drug, treat, headache tea, soothe, headache analgesic, banish, headache headache, treat with, caffeine

headache, respond to, painkiller Original Open IE Output drug, relieve, headache drug, treat, headache headache, respond to, painkiller headache, treat with, caffeine analgesic, banish, headache headache, control with, aspirin tea, soothe, headache

aspirin, cure, headache aspirin, eliminate, headache coffee, help, headache Consolidated Open IE Output Semantic Applications Example: Structured Queries What relieves headaches? Semantic Applications Example: Structured Queries What relieves headaches?

drug, relieve, headache drug, treat, headache headache, respond to, painkiller headache, treat with, caffeine analgesic, banish, headache headache, control with, aspirin tea, soothe, headache aspirin, cure, headache aspirin, eliminate, headache

coffee, help, headache Structured Query: drug, relieve, headache drug, treat, headache headache, respond to, painkiller headache, treat with, caffeine analgesic, banish, headache headache, control with, aspirin

tea, soothe, headache aspirin, cure, headache aspirin, eliminate, headache coffee, help, headache Structured Query: drug painkiller caffeine

analgesic tea aspirin coffee Structured Query: Our Contributions Structuring Open IE with Proposition Entailment Graphs Dataset: 30 gold-standard graphs, 1.5 million entailment annotations Algorithm for constructing Focused Proposition Entailment Graphs Analysis: Predicate entailment is not quite what we thought Proposition Entailment Graphs

Related Work: Predicate Entailment Graphs Berant et al. (2010,2011,2012) We extend Berant et al.s work from predicates to propositions Focused Proposition Entailment Graphs Nodes: Open IE propositions Edges: Textual Entailment Focused Proposition Entailment Graphs Assumptions: Binary Propositions and Common Topic Binary Propositions

Focused on a common topic Focused Proposition Entailment Graphs Assumptions: Binary Propositions and Common Topic Binary Propositions Focused on a common topic drug, relieve, headache drug, treat, headache headache, respond to, painkiller headache, treat with, caffeine

analgesic, banish, headache headache, control with, aspirin tea, soothe, headache aspirin, cure, headache aspirin, eliminate, headache coffee, help, headache drug, relieve, headache drug, treat, headache headache, respond to, painkiller

headache, treat with, caffeine analgesic, banish, headache headache, control with, aspirin tea, soothe, headache aspirin, cure, headache aspirin, eliminate, headache coffee, help, headache Focused Proposition Entailment Graphs Edges: Textual Entailment

Proposition Entailment Simpler than sentence-level entailment More complicated than lexical entailment Enables investigation of inference phenomena in an isolated manner Constructing Proposition Entailment Graphs Task Definition: Given a set of propositions , find all their entailment edges. Dataset Dataset: High-Quality Open IE

Propositions Googles Syntactic N-grams Based on millions of books Filter for subject-verb-object Including prepositional objects and passive Result: 68 million high-quality propositions Dataset: Annotating Entailment Graphs Select 30 healthcare topics antibiotic, caffeine, insomnia, scurvy, Collect a set of propositions focused on each topic Manually clean noisy extractions

Retaining 200 propositions per graph (average) Efficiently annotate entailment 1.5 million entailment judgments Algorithm How do we recognize proposition entailment? . ? How do we recognize proposition entailment?

. Observation: propositions entail their lexical components entail How do we recognize proposition entailment? . Observation: propositions entail their lexical components entail How do we recognize proposition entailment? .

Proposition entailment is reduced to lexical entailment in context Lexical Entailment Lexical Entailment Features 1 Lexical Entailment (Logistic) 2

3 = ( ) Lexical Entailment Features WordNet Relations UMLS Distributional Similarity String Edit Distance Lexical Entailment Features

1 Lexical Entailment (Logistic) 2

3 = ( ) Supervision From Lexical to Proposition Entailment Lexical Entailment Features 1 Lexical Entailment (Logistic)

2 3 = ( ) Supervision From Lexical to Proposition

Entailment Predicate Entailment Features 1 Predicate Entailment (Logistic) 2

3 = ( ) Supervision Argument Entailment Features 1

Argument Entailment (Logistic) 2 3

= ( ) Supervision From Lexical to Proposition Entailment Predicate Entailment Features 1

Predicate Entailment (Logistic) 2 3 = ( ) Supervision

Proposition Entailment (Conjunction) Argument Entailment Features 1 Argument Entailment (Logistic) =

2 3 = ( ) Supervision Distant Supervision (WordNet)?

Predicate Entailment Features 1 Predicate Entailment (Logistic) 2

3 = ( ) WordNet Proposition Entailment (Conjunction) Following Snow (2005), Berant (2012) Argument Entailment Features

1 Argument Entailment (Logistic) = 2

3 = ( ) WordNet Direct Supervision (30 Annotated Graphs) Predicate Entailment Features

1 Predicate Entailment (Logistic) 2 Argument Entailment Features 3

= ( ) Proposition Entailment (Conjunction) 1 Argument Entailment (Logistic)

= Annotated Graphs 2 3

= ( ) Direct Supervision (30 Annotated Graphs) Predicate Entailment Features 1 2

Argument Entailment Features 3 1 Hidden Layer

Proposition Entailment (Conjunction) = Annotated Graphs 2

3 Flat Model Predicate Entailment Features 1 2

Argument Entailment Features 3 Proposition Entailment (Logistic) = ( Annotated Graphs

1 2 + ) 3

Compared Methods Component-Level Distant Supervision (WordNet) Predicates & Arguments Predicates Only Arguments Only Proposition-Level Direct Supervision (30 Annotated Graphs) Hierarchical (our method) Flat All methods used Berant et al.s Global Optimization method Results

Direct Supervision: Flat vs Hierarchical Hierarchal model performs better than flat model 70% Better to model predicate and argument entailment separately Performance (F1) 65% 60% 63.7%

55% 50% 61.6% Distant vs Direct Supervision 70% Although WordNet provides more training examples 65% Performance (F1) Direct supervision is better

60% 63.7% 55% 50% 61.6% 59.7% Predicate Entailment with Distant Supervision Ignoring predicates improves distant supervision baselines

70% 60% Performance (F1) 50% 40% 30% 59.7% 20% 10% 8.0% 0%

7.2% Are WordNet relations capturing real-world predicate entailments? Predicate Entailment vs WordNet Relations Over a predicate inference subset, how many predicate entailments are covered by WordNet? Positive 11.64% Negative 14.65%

Positive indicators synonyms, hypernyms, entailment None 73.70% Predicate Entailment vs WordNet Relations Over a predicate inference subset, how many predicate entailments are covered by WordNet? Positive 11.64% Negative 14.65%

Positive indicators synonyms, hypernyms, entailment Negative Indicators None 73.70% antonyms, hyponyms, cohyponyms Why isnt WordNet capturing predicate entailment? Predicate Entailment is ContextSensitive The words do not necessarily entail,

but the situations do. Predicate Entailment is ContextSensitive The words do not necessarily entail, but the situations do. Investigating Context-Sensitive Entailment Recent work on context-sensitive lexical inference e.g. (Melamud et al., 2013) Previous datasets Lexical substitution (McCarthy and Navigli, 2007) Predicate inference (Zeichner et al., 2012)

We offer a new dataset of real-world lexical entailments in context! Sample: synthetic vs naturally occurring Size: several thousands vs 1.5 million Conclusion Conclusion Structuring Open IE with Proposition Entailment Graphs Algorithm for constructing Focused Proposition Entailment Graphs Analysis: Predicate entailment is extremely context-sensitive Dataset: 1.5 million proposition entailment decisions Thank you for listening!

Recently Viewed Presentations

  • CS 620 Advanced Operating Systems - Cleveland State University

    CS 620 Advanced Operating Systems - Cleveland State University

    The default gateway contains info about the network Ids of the other networks in the internetwork, so it can forward the packet to other routers until the packet is eventually delivered to a router connected to the specified destination. This...
  • ADM Guidance vs. POM10-15 Funding for Milestone B ... - qmdns.org

    ADM Guidance vs. POM10-15 Funding for Milestone B ... - qmdns.org

    MDAP Portfolio Tornado Chart. We have a bad behaving population of MDAPs; the magnitude of the JSF program makes portfolio-wide statistics meaningless. Measuring portfolio cost growth and other similar statistics is Bad Math. This is the Tornado Chart. It provides...
  • Parenthetical Citation

    Parenthetical Citation

    Other Notes and Tricky Tips Typically the citation is the author's last name and page number. (Jones 18) Notice there is no comma after the name and there is no p. for page Parenthetical citation goes at the end of...
  • Fronts and Air Masses

    Fronts and Air Masses

    Fronts and Air Masses By Brian Fontaine 1. An air mass is a large body of air that has similar temperature and moisture throughout. Air Masses are named for their place of origin and source of moisture.
  • The Lesson of Salem - White Plains Middle School

    The Lesson of Salem - White Plains Middle School

    The article, "The Lesson of Salem", by Laura Shapiro and the play The Crucible, by Arthur Miller, connect because they both tell a story of accused witchcraft. Like in The Crucible, the article "The Lesson of Salem", demonstrates the numerous...
  • Image-based rendering Michael F. Cohen Microsoft Research Computer

    Image-based rendering Michael F. Cohen Microsoft Research Computer

    Video view interpolation Fewer cameras Smooth Motion Automatic and Real-time rendering Prior work: IBR (static) The Lumigraph Gortler et al., SIGGRAPH '96 Concentric Mosaics Shum & He, SIGGRAPH '99 Plenoptic Modeling McMillan & Bishop, SIGGRAPH '95 Light Field Rendering Levoy...
  • Resources by State The American Recovery and Reinvestment Act ...

    Resources by State The American Recovery and Reinvestment Act ...

    Prioritize Now decide on a strategy using these prospects that will come closest to your goals. It could be led by renewals of existing foundation grants, or applications for regional development funds or private donors, or online and email campaigns....
  • (J Clin Endocrinol Metab 104: 128, 2019) Introduction

    (J Clin Endocrinol Metab 104: 128, 2019) Introduction

    rebound . in bone turnover and to decrease the risk of . rapid . BMD loss and an increased risk of fracture. ... benefit-risk ratio even for long-term . treatment, particularly . in older women at highest . risk ....