Text Summarization using Lexical Chains Summarization What is

Text Summarization using Lexical Chains

Summarization? • What is Summarization? • Advantages… • Challenges… Summarization using Lexical Chains

Project Goals • Aim: Summary of an original text without requiring full semantic interpretation • Tools: Word. Net thesaurus, shallow parser, POS & Brill’s tagger, Segmentation algorithm. Summarization using Lexical Chains

Description • Input Domain: Technical_Article. txt • Processing: Algorithm by Regina Barzilay and Michael Elhadad • Output: Lexical Chains & Extract Summarization using Lexical Chains

Design steps… • Segment the original text • Construct lexical chains • Identify strong chains • Extract significant sentences Summarization using Lexical Chains

Step One… • Segment the original text • Construct lexical chains • Identify strong chains • Extract significant sentences Summarization using Lexical Chains

Seg-ment-ation Algorithm • Form a Token Sequence • Parameter: w • Form a Block • Parameter: b • Computation of Similarity • Parameter: sim(b 1, b 2) Summarization using Lexical Chains

…. • Plot Graph. • Parameter: depth score • Sort • Segment boundary Summarization using Lexical Chains

Step Two… • Segment the original text • Construct lexical chains • Identify strong chains • Extract significant sentences Summarization using Lexical Chains

Construction of Lexical Chains Procedure: 1. Select a set of candidate words. 2. Find appropriate chain. 3. Insert the word in the chain. Summarization using Lexical Chains

Step Three… • Segment the original text • Construct lexical chains • Identify strong chains • Extract significant sentences Summarization using Lexical Chains

Strong chains? Good predictors of strength of a chain • Length = number of occurrences of members in a chain • Homogeneity index = 1 – (number of distinct occurrences / length) Summarization using Lexical Chains

Chain Score(Chain) = Length * Homogeneity_Index Strength Criterion : Score(Chain) > Average(Scores) + 2* Standard. Deviation(Scores) Summarization using Lexical Chains

Step Four… • Segment the original text • Construct lexical chains • Identify strong chains • Extract significant sentences Summarization using Lexical Chains

Extraction A Heuristic • Select representative words • Extract sentence with first appearance of representative Word. Summarization using Lexical Chains

Thank You