Shape Predicates 15 494 Cognitive Robotics David S

Shape Predicates 15 -494 Cognitive Robotics David S. Touretzky & Ethan Tira-Thompson Carnegie Mellon Spring 2008 03/06/2021 15 -494 Cognitive Robotics 1

The World is Full of Shapes When we extract shapes from camera images, we may get a lot of objects. We needs ways of selecting and comparing shapes. “Find all the orange things. ” “Find all the lines longer than this line. ” Tekkotsu provides shape predicates for testing shapes. These can be composed to form complex tests. To use these, you need to understand C++ functors. 03/06/2021 15 -494 Cognitive Robotics 2

Function Objects (Functors) #include <iostream> using namespace std; class My. Functor { public: void operator() () const { cout << "Foo!" << endl; } }; int main() { My. Functor fluffy; fluffy(); } 03/06/2021 15 -494 Cognitive Robotics 3

Functors Can Store Values Private comparison value class Bigger. Than { private: int value; Constructor initializes the private value public: Bigger. Than(int val) : value(val) {} bool operator() (int x) const { return x > value; } }; Function call operator compares x against the value 03/06/2021 15 -494 Cognitive Robotics 4

Testing Bigger. Than int main() { Bigger. Than bigtest(5); for (int i = 3; i < 8; i++ ) cout << i << (bigtest(i) ? " passes" : " fails" ) << endl; } 3 fails 4 fails 5 fails 6 passes 7 passes 03/06/2021 15 -494 Cognitive Robotics 5

Function Conjunction class And. Big. Small { private: Bigger. Than bigtest; Smaller. Than smalltest; public: And. Big. Small(Bigger. Than b, Smaller. Than s) : bigtest(b), smalltest(s) {} bool operator() (int x) { return bigtest(x) && smalltest(x); } }; int main() { And. Big. Small myconj(Bigger. Than(0), Smaller. Than(100)); for ( int i = -10; i < 150; i+=40 ) cout << i << " gives " << myconj(i) << endl; } 03/06/2021 15 -494 Cognitive Robotics -10 gives 0 30 gives 1 70 gives 1 110 gives 0 6

STL functional. h The STL (Standard Template Library) provides classes called unary_function and binary_function from which functors can be composed. class Bigger. Than : unary_function<int, bool> { private: int value; public: Bigger. Than(int val) : value(val) {} bool operator() (int x) { return x > value; } }; These user-defined functor classes can then be used with STL functions for searching, etc. But they're kind of awkward. 03/06/2021 15 -494 Cognitive Robotics 7

Shape Predicates The Shape classes provide their own functor mechanism for defining shape predicates. Easier to use than the generic STL. Some predicates for common shape tests are built in, e. g. , Comparing the positions of two shapes (left/right or above/below) Comparing the lengths of two lines Comparing line orientations New predicates are easy to define. 03/06/2021 15 -494 Cognitive Robotics 8

Shape<Line. Data> Functors Compare the lengths of all the pink lines in the image against that of the third line. NEW_SKETCH(cam. Frame, uchar, sketch. From. Seg()); NEW_SKETCH(pink_stuff, bool, visops: : colormask(cam. Frame, "pink")); NEW_SHAPEVEC(lines, Line. Data: : extract. Lines(pink_stuff)); SHAPEVEC_ITERATE(lines, Line. Data, ln) if ( Line. Data: : Length. Less. Than()(ln, lines[2]) ) cout << "Shorter: " << ln->get. Id() << endl; else cout << "Longer: " << ln->get. Id() << endl; END_ITERATE; 03/06/2021 15 -494 Cognitive Robotics 9

Line. Data: : Length. Less. Than Class-specific shape predicates are defined with the respective shape, e. g. , in Line. Data. h and Line. Data. cc. In Line. Data. h: class Length. Less. Than : public Binary. Shape. Pred<Line. Data> { public: bool operator() (const Shape<Line. Data> &ln 1, const Shape<Line. Data> &ln 2) const; }; In Line. Data. cc: void Line. Data: : Length. Less. Than: : operator() (const Shape<Line. Data> &line 1, const Shape<Line. Data> &line 2) const { return line 1 ->get. Length() < line 2 ->get. Length(); } 03/06/2021 15 -494 Cognitive Robotics 10

Generic Shape Predicates Some predicates work for shapes of any type. They are defined on class Shape. Root. Example: Is. Color. NEW_SHAPEVEC(blobs, Blob. Data: : extract. Blobs(cam. Frame, 50)); Is. Color orangetest("orange"); SHAPEVEC_ITERATE(blobs, Blob. Data, b) if ( orangetest(b) ) cout << "Orange: " << b->get. Id() << endl; else cout << "Not orange: " << b->get. Id() << endl; END_ITERATE; 03/06/2021 15 -494 Cognitive Robotics 11

Subclasses of Base. Data: Subclasses of Shape. Root: 03/06/2021 15 -494 Cognitive Robotics 12

Generic Is. Color Predicate class Is. Color : public Unary. Shape. Root. Pred { private: rgb color; public: Is. Color(rgb col) : Unary. Shape. Root. Pred(), color(col) {} Is. Color(std: : string const &colorname) : Unary. Shape. Root. Pred(), color(Project. Interface: : get. Color. RGB(colorname)) {} bool operator() (const Shape. Root &shape) const { return shape->get. Color() == color; } }; 03/06/2021 Note: the colorname string is looked up once, by the constructor, and the result is stored in the private variable color. When the functor is invoked on a Shape. Root, no lookup is necessary. 15 -494 Cognitive Robotics 13

Is. Left. Of / Is. Left. Of. This Is. Left. Of() This is a Binary. Shape. Root. Pred that requires two arguments, and compares their centroids: Is. Left. Of() (line 2, blob 6) Is. Left. Of. This(x) This is a Unary. Shape. Root. Pred that requires one argument: Is. Leftof. This(line 2) (blob 6) constructor 03/06/2021 argument 15 -494 Cognitive Robotics 14

Using Is. Left. Of. This An instance of Is. Left. Of. This stores a Shape. Root inside it, and uses it for comparison tests. Is. Left. Of. This mytest(lines[4]); SHAPEVEC_ITERATE(lines, Line. Data, ln) if ( mytest(ln) ) cout << "This is left of me: " << ln->get. Id() << endl; END_ITERATE; 03/06/2021 15 -494 Cognitive Robotics 15

Built-In Shape Predicates Shape. Root: Is. Color Is. Type Is. Name Is. Left. Of / Is. Right. Of Is. Above / Is. Below Is. Left. Of. This. . . Is. Above. This. . . 03/06/2021 Shape<Line. Data>: Length. Less. Than Is. Horizontal Is. Vertical Parallel. Test Perpendicular. Test Colinear. Test 15 -494 Cognitive Robotics 16

And. Pred / Or. Pred Because shape predicates are classes, we can compose them using the functors And. Pred and Or. Pred. SHAPEVEC_ITERATE(lines, Line. Data, ln) if ( And. Pred(Is. Color("pink"), Is. Left. Of. This(lines[3])) (ln) ) cout << "winner: " << ln->get. Id() << endl; else cout << "loser: " << ln->get. Id() << endl; END_ITERATE; We are composing two unary predicates, so the result is also a unary predicate: it takes one argument. 03/06/2021 15 -494 Cognitive Robotics 17

Vectors of Shape. Roots cam. Sh. S. all. Shapes() returns all the shapes in the shape space, as a vector<Shape. Root>. cam. Sh. S will be automatically coerced to vector<Shape. Root> by an implicit call to all. Shapes() Use SHAPEROOTVEC_ITERATE(vec, var) to iterate: SHAPEROOTVEC_ITERATE(cam. Sh. S, s) if ( Or. Pred(Is. Type(blob. Data. Type), Is. Type(line. Data. Type)) (s) ) cout << "Is blob or line: " << s->get. Id() << endl; END_ITERATE; Shape type constants like blob. Data. Type are defined in Shape. Types. h 03/06/2021 15 -494 Cognitive Robotics 18

Mirroring STL Search Functions The STL provides a collection of functions for searching through a vector using either a binary comparison predicate or a unary test predicate. Tekkotsu provides similar functions for shape predicates: find_if, subset, max_element, stable_sort, remove_copy_if There also some new functions unique to shapes: 03/06/2021 find_shape, select_type 15 -494 Cognitive Robotics 19

Filtering Shapes Find the first blob: NEW_SHAPE(blob 0, Blob. Data, find_if<Blob. Data>(cam. Sh. S)); cam. Sh. S is treated as shorthand for cam. Sh. S. all. Shapes() If no blobs found, an invalid Shape is returned Find all the blobs: NEW_SHAPE_VEC(all_blobs, Blob. Data, select_type<Blob. Data>(cam. Sh. S)); 03/06/2021 15 -494 Cognitive Robotics 20

More Filtering and Searching Find all the orange blobs: NEW_SHAPEVEC(orange_blobs, Blob. Data, subset(all_blobs, Is. Color(“orange”))) Find the longest line: NEW_SHAPE(longest, Line. Data, max_element(lines, Line. Data: : Length. Less. Than())) Test is “less than”, but max_element returns longest. 03/06/2021 15 -494 Cognitive Robotics 21

Implementing max_element // from Dual. Coding/Shape. Funs. h template<class T, typename Comparison. Type> Shape<T> max_element(const vector<Shape<T> > &vec, Comparison. Type comp) { typename vector<Shape<T> >: : const_iterator result = max_element(vec. begin(), vec. end(), comp); if ( result != vec. end() ) return *result; else return Shape<T>(); } 03/06/2021 T = Line. Data Comparison. Type = Length. Less. Than vec is a SHAPEVEC of Line. Data If no elements, return an invalid shape. comp is an instance of 15 -494 Cognitive Robotics Length. Less. Than 22

Negating a Predicate Use not 1(p) to negate a unary predicate: NEW_SHAPEROOTVEC(non_orange, subset(cam. Sh. S, not 1(Is. Color(“orange”)))); Use not 2(p) to negate a binary (comparison) predicate: NEW_SHAPEVEC(shortlines, Line. Data, stable_sort(lines, not 2(Line. Data: : Length. Less. Than()))); 03/06/2021 15 -494 Cognitive Robotics 23

Inside SHAPEVEC_ITERATE(lines, Line. Data, ln) do_something_with(ln); END_ITERATE; Expands into: for ( vector<Shape<Line. Data> >: : _iterator ln_it = lines. begin(); ln_it != lines. end(); ln_it++ ) { Shape<Line. Data> &ln = *ln_it; do_something_with(ln); }; 03/06/2021 15 -494 Cognitive Robotics 24

Nested Iteration NEW_SHAPEVEC(lines, Line. Data, select_type<Line. Data>(cam. Sh. S)); lines = stable_sort(lines, not 2(Line. Data: : Length. Less. Than())); SHAPEVEC_ITERATE(lines, Line. Data, ln 1) SHAPENEXT_ITERATE(lines, Line. Data, ln 1, ln 2) if ( Line. Data: : Parallel. Test()(ln 1, ln 2) ) cout << ln 1 << " parallel to " << ln 2 << endl; if ( Line. Data: : Perpendicular. Test()(ln 1, ln 2) ) cout << ln 1 << " perpendicular to " << ln 2 << endl; if ( Line. Data: : Colinear. Test()(ln 1, ln 2) ) cout << ln 1 << " colinear with " << ln 2 << endl; END_ITERATE; Shape<Line. Data>(id=10002, indx=1) perpendicular to Shape<Line. Data>(id=10005, indx=4). . . etc. 03/06/2021 15 -494 Cognitive Robotics 25