Vision Scene Labelling Structure in the world allows

Vision: Scene Labelling Structure in the world allows us to interpret complex situations by propagating constraints. This is conveniently demonstrated in computer vision, using a technique known as Guzman labelling. CSE 415 -- (c) S. Tanimoto, 2002 Scene Labelling 1

Blocks-World Vision Problem Given: A 2 -D line drawing representing a collection of polyhedral blocks on a table. Determine: Which faces (bounded regions in the drawing) go together as parts of the same objects. Which edges (line segments) are internal to objects and which are “occluding” edges. CSE 415 -- (c) S. Tanimoto, 2002 Scene Labelling 2

Example Scene Analysis Problem CSE 415 -- (c) S. Tanimoto, 2002 Scene Labelling 3

Guzman’s Vertex Labels Ell Arrow Fork Tee Arrow: There is an angle of more than 180 deg. Fork: Each of the 3 angles is of less than 180 deg. Tee: There is one angle of exactly 180 deg. , and 2 smaller ones. CSE 415 -- (c) S. Tanimoto, 2002 Scene Labelling 4

Guzman’s Labelling Technique 1. Classify each vertex into one of the categories: ell, arrow, fork, tee, or other. 2. At each vertex, mark each incoming edge according to the role it plays at the vertex (e. g. , shank of an arrow, stem of a tee). 3. Create single links between neighboring regions each time: a. they are divided by a line segment at a fork, b. they are divided by the shank of an arrow, c. they are in corresponding positions of a configuration of two opposing tees with colinear stems. 4. Create a “node” for each region. Whenever two regions are doubly linked, connect their nodes with a “same object” link. CSE 415 -- (c) S. Tanimoto, 2002 Scene Labelling 5

Example with Vertex Labels A L L L A A T F T T T L A F A L A CSE 415 -- (c) S. Tanimoto, 2002 Scene Labelling 6

Example with Region Links CSE 415 -- (c) S. Tanimoto, 2002 Scene Labelling 7

Example with Object Links CSE 415 -- (c) S. Tanimoto, 2002 Scene Labelling 8

Example with Internal and Occluding Edges Identified internal occluding CSE 415 -- (c) S. Tanimoto, 2002 Scene Labelling 9

Discussion The work of Guzman was refined by D. Huffman, M. Clowes, and D. Waltz to take into account additional kinds of edges, including crack edges, oriented occluding edges, and concave and convex internal edges. A key point of this research was to show that constraints imposed by the real world could make seemingly intractable combinatorial interpretation problems actually solvable. CSE 415 -- (c) S. Tanimoto, 2002 Scene Labelling 10