Temporal information extraction Reasoning with events based on

Temporal information extraction: Reasoning with events based on their descriptions Leon Derczynski University of Sheffield

Introduction Background Anchoring events Reasoning about events Representing temporal data Evaluating annotations

Background Why bother? Temporal information affects everything described by language. The world is in a state that changes with time. Not all assertions made in written text are true together. Temporal information shows which sets of data can concurrently be true.

Tense and temporal models Zeno Vendler (1957) “Verbs and Times” Hans Reichenbach (1947) “The tenses of verbs” James Allen (1983) “Maintaining knowledge about temporal intervals”

Vendler verb classification: Verb instances fall into one of four groups: Stative: a persistent state (“John sits”) Activity: lasts for a finite period (“Bob ran for an hour”) Accomplishment: takes a finite period, and culminates (“Kate climbed the hill in five minutes”) Achievement: Instantaneous finishing events (“Lucy reached the top of Everest”) Tests are provided to see which group a verb sense fits in.

Reichenbach model of verb tenses: Speech time: when the words were uttered. Event time: when the described event occurred. Reference time: like a viewpoint. “The cat will break the door” – ST=RT, ET in the future “The cat will have broken the door” – ST = present, ET in the future, RT looks back onto ET Allows simplistic description of any phrase. Tracking reference time is sometimes very helpful: “When John comes home, I will have gone” In this case, when describes a reference time for the whole sentence.

Allen Interval logic: All events are described as intervals, with start and end points. Interval relation types are defined (before, includes, starts…). A table for inferring about interval relations is given. E. g. : A before B, A includes D: Before stipulates that A’s endpoint is before B’s start. We can infer D before B.

Anchoring events Introduction to event anchoring Dealing with named weekdays TEA – an implemented anchoring system Problems

Anchoring events Fixing information from text to a timeline. Calendrical time is a common reference, given a calendar. Expressions describing a time are sometimes referred to as TIMEXs. Once identified, a TIMEX may be normalised to a fully specified date or interval. Named entity recognition, finite state grammars and machine learning have all been used to identify these expressions. Appropriate granularity should be chosen.

Weekday references The English week has seven day names. A single day name is often deemed sufficient reference for a human: “I’ll see you next Tuesday” “Monday, and the markets are buzzing” To anchor a weekday, given ST and a day name, we need to choose direction from ST, and optionally distance. Baldwin: an inclusive 7 -day sliding window, centred on today. Mani & Wilson: find controlling verb’s tense and use this to determine direction. Tense estimation: check Po. S of sentence tokens for VBD; if found, assume backwards. Dependency-based: use Stanford parser to find controlling verb. Mazur & Dale - “What’s the Date? ” (2008)

Generic vs. specific Some expressions, that look like TIMEXs, should not be normalised. “Today” can mean: the 24 -hour period containing ST and bounded by midnights. Modern times, or a change of frame of reference: “In Victorian times, ladies wore long dresses. Today, modern fashions do not dictate a single length. ” This second idea is not restricted to the period from 00: 00 to 23: 59 GMT on Thursday 7 th May 2009! As 90% of uses in some texts are specific 1, some systems choose to accept a 10% error rate. Features based on local words can help distinguish generic from specific, but below this baseline accuracy. 2 1: Han, Gates & Levin – “From Language to Time: A Temporal Expression Anchorer” (2006) 2: Mani & Wilson – “Robust Temporal Processing of News” (2000)

TEA Temporal Expression Anchorer: Han, Gates & Levin at CMU. Calendar used as time ontology, dealing with various levels of granularity. Processes TCNL (Time Calculus for Natural Language). Identifies temporal expressions in input, and associates TIMEXs with their textually nearest verb. Absolute and relative expressions are evaluated using TCNL: “Friday last week” is split, into “Friday” and “last week” {fri} + {now - |1 week|} = {fri, {now - |1 week|}} = {now |1 fri|} Constraint satisfaction based on a calendar model narrows the possible set of absolute dates.

Determining event durations Given some normalised expressions, knowing event durations can greatly increase our reasoning ability. Data can be taken from human annotators. Determining a typical event duration is difficult: “The dog ran up the hill” “Linda had finished her cleaning” This results in low inter-annotator agreement. A simplified approach would allocate durations into two classes: shorter or longer than a day. Possible to classify events this simply with 76% accuracy, using hypernym and local word Po. S features. Pan, Mulkar & Hobbs – “Learning Event Durations from Event Descriptions” (2006)

Reasoning about events Introduction Temporal closure Minimal notations and temporal inference Help from linguistic models

Reasoning about events Annotations often only describe a subset of a document’s temporal information, perhaps as a number of labelled events and times. An annotation may also include some links between pieces of temporal information. It is possible to infer data about relations between points, given a set of rules or logic, and some existing relations. It is also possible to add detail and boundaries to an annotation based on linguistic features of the source text. This ability to reason about events saves human annotators work, and allows us to maximise the available descriptions from their efforts.

Temporal closure A temporal closure can be thought of as a graph: Times and events are node; relations are edges. Every time and event is connected to every other. E. g. t 1 is Tuesday 5 th May 2009 e 1 is hearing this talk We can say: t 1 before e 1, thus giving a type to this relation. A temporal closure includes relations that have been inferred between every possible node in the graph. This can lead to very large amounts of data for only a moderate-sized document.

Minimal annotations It is rare for every relation (graph edge) to be annotated. We can infer some relations: (t 1 before e 1) ^ (e 1 before e 2) => (t 1 before e 2) Inference can be used to complete a closure without specifying every relation’s type. When this applies, and no more relations can be removed, we have a minimal annotation. For example: Three nodes: e 4, e 5, e 6 Closure has 3 possible relations A minimal graph may just say: (e 4 after e 5) (e 5 simultaneous e 6) To infer the closure, we simply need to add: (e 4 after e 6), or (e 6 before e 4)

Relation inference Allen’s interval logic describes 13 relationships, and provides a transitivity table for inferring a relation given two related ones. Some inconsistent labellings are possible. Backtracking over the initial graph should detect these cases. A set of ten inference rules can be used: Allen’s 13 relations are reduced to just 3, including some reversal of parameters. Only before, simultaneous and includes are used e 9 after e 10 => e 10 before e 9 These rules can be iteratively added to an agenda and used to reason with a database of approved relations. For small graphs (< 2000 edges) we can assign types to around 10% of relations, given a human annotation.

Applying Reichenbach Reference time can provide a boundary on an event. “John had eaten all the pies” “John had eaten all the pies when Annika arrived” ET – RT = ET 2 – ST Because we know that RT is after ET and equal to ET 2, we can specify three temporal relations: Event 2 = arriving Reference time is the same across the sentence. Event 1 = eating ET – RT – ST e 1 before e 2 e 1 before ST e 2 before ST Having a model for tenses allows us to confidently add relations to a temporal graph of a discourse.

Representing temporal data Introduction TIMEX and Time. ML TCNL T-BOX

Representing temporal data Once we can identify temporal information, we need to store this information. Temporal information is rich, and favours a format that can capture it well. Aspect, polarity, tense, part of speech Event class, event frequency Hints about reference, speech and event time Notation languages are available both for storing and working with this data. These languages are new (under a decade old), and possibly not yet mature.

TIMEX Standard for describing a time-specific expression. Evolved through the MUC conferences and TERN, through TIMEX, TIMEX 2 and TIMEX 3 is currently used as the means of describing absolute times in Time. ML. <TIMEX 3 tid="t 43" type="DATE" value="1989 -10 -30" temporal. Function="false" function. In. Document="CREATION_TIME"> 10/30/89 </TIMEX 3>

Time. ML SGML-based language for temporal annotation. Allows identification of events and times. Thorough provision of links between events and times: TLINK: temporal, possibly including a SIGNAL tag to a linking word SLINK: subordinate ALINK: aspectual ISO standard.

Time. ML - Time. Bank Corpus of 181 newswire texts. Temporal information annotated in Time. ML: 6383 TLINKs, 7940 EVENTs, 3004 k. B in size. Tiny compared to some other types of corpus. Involved a large human annotator effort and a few different versions. Biggest temporally annotated corpus.

TCNL Developed at CMU with L. Levin. Useful for reasoning between events. Captures intensional meanings of expressions. “Yesterday” becomes {now-|1 day|} instead of something like 20090506 A set of operators are used to reason between operands: +/- forward/reverse shifting @ for in; {|2 sun| @ {may}} is “the second Sunday in May” & for distribution; {15 hour} & [{wed}: {fri}]} is “ 3 pm from Wednesday to Friday”

T-BOX Reading solid SGML is inconvenient for humans; a visual representation of events may be preferable. Presenting events on a timeline may lead to unintentional over-specification. Suggests a distance. Many intervals are left with one end open Plotting parts of a sentence in temporal order will destroy word order, making it hard to read Annotating documents can be done more easily when events are grouped locally and visually connected. T-BOX 1 from Brandeis specifies a set of rules for rendering events and their relations. 1: Verhagen – “Drawing Time. ML relations with T-BOX” (2007)

T-BOX Relations only exist between nodes that are directly connected or contained. This suggests: - X contains Y - Y is before Z Drawing a temporal closure could provide a very cluttered and messy graph. A set of guidelines are provided for reducing graphs to something more visually appealing. Equivalence classes for some events. Break cycles in graphs. Remove derivable relations.

Evaluating annotations Typical annotation evaluation. Graph-based evaluation.

Evaluating annotations Annotations can be compared in different ways. When evaluating automated TIMEX or relation identification against a gold standard, we can measure precision and recall. Time. Bank is often used as a gold standard for training and evaluation or systems working in Time. ML. Evaluating TIMEX normalisation needs a different measure, as there are varying degrees of correctness available.

Graph based evaluation Based on the use of minimal temporal graphs. Graphs between events (intervals) are converted into graphs between points: Smaller set of relations, needing only = and < Simpler algebra Simultaneous points are grouped into nodes. Graphs over the same set of points can then be compared, based on the number of node splits and merges needed to reach one from the other.

Summary Background and models useful for temporal information extraction. Technical approaches to temporal IE. How to reason about events. Temporal closure & minimal annotations. Notations for temporal information. Evaluating temporal graphs & annotations.

Questions