Binary Morphology Morphology Morphology generally concerned with shape

Binary Morphology

Morphology • Morphology generally concerned with shape and properties of objects. Used for segmentation and feature extraction. • Segmentation = used for cleaning binary objects. • Two basic operations – erosion (opening) – dilation (closing)

Morphological operations and algebras • Different definitions in the textbooks • Different implementations in the image processing programs. • The original definition, based on set theory, is made by J. Serra in 1982. • Defined for binary images - binary operations (boolean, set-theoretical) • Can be used on grayscale images - multiple-valued logic operations

Morphological operations on a PC • • • Various but slightly different implementations in Scion Paint Shop Pro Adope Photoshop Corel Photopaint mm Try them, it is a lot of fun

Morphological operations in Scion • Rank filtering (Grayscale) • Binary operations – based on 3 x 3 neighborhood – based on EDM (Euclidean Distance Map) • Applications – Noise reduction – Segmentation – Edge detection

In general; what is “Morphology”? • The science of form and structure – the science of form, that of the outer form, inner structure, and development of living organisms and their parts – about changing/counting regions/shapes • Among other applications it is used to pre- or post-process images – via filtering, thinning and pruning • Smooth region edges – create line drawing of face • Force shapes onto region edges – curve into a square • Count regions (granules) – number of black regions • Estimate size of regions – area calculations

Mathematical Morphology - Set-theoretic representation for binary shapes

Binary Morphology • Morphological operators are used to prepare binary (thresholded) images for object segmentation/recognition – Binary images often suffer from noise (specifically salt-andpepper noise) – Binary regions also suffer from noise (isolated black pixels in a white region). Can also have cracks, picket fence occlusions, etc. • Dilation and erosion are two binary morphological operations that can assist with these problems.

Goals of morphological operations: Ø 1. Simplifies image data Ø 2. Preserves essential shape characteristics Ø 3. Eliminates noise Ø 4. Permits the underlying shape to be identified and optimally reconstructed from their distorted, noisy forms

What is the mathematical morphology ? Ø 1. An approach for processing digital image based on its shape Ø 2. A mathematical tool for investigating geometric structure in image Ø The language of morphology is set theory. Mathematical morphology is extension to set theory.

Importance of Shape in Processing and Analysis ØShape is a prime carrier of information in machine vision ØFor instance, the following directly correlate with shape: Øidentification of objects Øobject features Øassembly defects

Shape Operators Ø Shapes are usually combined by means of : • Set Union (overlapping objects): • Set Difference based on Set intersection (occluded objects): Set intersection Set difference

Morphological Operations ØThe primary morphological operations are dilation and erosion ØMore complicated morphological operators can be designed by means of combining erosions and dilations We will use combinations of union, complement, intersection, erosion, dilation, translation. . . Let us illustrate them and explain how to combine

Dilation (Minkowski addition) • Erosion shrinks • Dilation expands binary regions • Can be used to fill in gaps or cracks in binary images structuring Element ( SE ) • If the point at the origin of the structuring element is set in the underlying image, then all the points that are set in the SE are also set in the image • Basically, its like OR’ing the SE into the image

Dilation • Fills in holes. • Smoothes object boundaries. • Adds an extra outer ring of pixels onto object boundary, ie, object becomes slightly larger.

Dilation example

Morphology • Erosion and dilation clean image but leave objects either smaller or larger than their original size. • Opening and closing perform same functions as erosion and dilation but object size remains the same.

More Erode and Dilate Examples Input Image Dilated Eroded Made in Paint Shop Pro

Dilation explained pixed by pixel Denotes origin of B i. e. its (0, 0) Denotes origin of A i. e. its (0, 0) B • • • • A • •

Dilation explained by shape of A B Shape of A repeated without shift • • • • A • • Shape of A repeated with shift

Properties of Dilation objects are light (white in binary) Dilation does the following: • 1. fills in valleys between spiky regions • 2. increases geometrical area of object • 3. sets background pixels adjacent to object's contour to object's value • 4. smoothes small negative grey level regions

Structuring Element for Dilation Length 6 Length 5

Structuring Element for Dilation

Structuring Element for Dilation Single point in Image replaced with this in the Result

Structuring Element for Dilation

Definition of Dilation: Mathematically Ø Dilation is the operation that combines two sets using vector addition of set elements. Ø Let A and B are subsets in 2 -D space. A: image undergoing analysis, B: Structuring element, denotes dilation

Mathematics: Dilation versus translation Ø Let A be a Subset of and. Ø The translation of A by x is defined as x is a vector Ø The dilation of A by B can be computed as the union of translation of A by the elements of B

Dilation versus translation, illustrated Element (0, 0) Shift vector (0, 0) • • • Shift vector (0, 1) • B • • • • •

Mathematics of Dilation: using Union Formula Center of the circle This circle will create one point This circle will create no point

Example of Dilation with various sizes of structuring elements Structuring Element Pablo Picasso, Pass with the Cape, 1960

Mathematical Properties of Dilation Ø Commutative Ø Associative Ø Extensivity Ø Dilation is increasing Illustrated in next slide

Illustration of Extensitivity of Dilation By 0 or O we denote the origin point A • • B • • • Replaced with Here 0 does not belong to B and A is not included in A B

More Properties of Dilation Ø Translation Invariance Ø Linearity Ø Containment Ø Decomposition of structuring element

More Formalisms for Dilation • The dilation operator takes two pieces of data as input – A binary image, which is to be dilated – A structuring element (or kernel), which determines the behavior of the morphological operation • Suppose that X is the set of Euclidean coordinates of the input image, and K is the set of coordinates of the structuring element • Let Kx denote the translation of K so that its origin is at x. • The DILATION of X by K is simply the set of all points x such that the intersection of Kx with X is non-empty

Example of Dilation with SE that has an origin in its center Example: Suppose that the structuring element is a 3 x 3 square with the origin at its center K= X= 1 1 1 1 1 { (-1, -1), (0, -1), (1, -1), (-1, 0), (0, 0), (1, 0), ( 1, 1), (0, 1), (1, 1) }

Dilation Example continued Note: Foreground pixels are represented by a color and background pixels are empty

Steo-by-step explanation of Dilation Structuring element First row processing Input Overlap of green and red Explanation of processing output

Dilation Second row processing output

Dilation Third row processing output Fourth row processing nothing

Final result of this Dilation Example

Another example of Dilation that can be analyzed step-by-step by students

Why we analyze Morphology mathematically? 1. Useful when you want to write efficient software 2. Very useful when you want to design a special purpose hardware computer for image processing using FPGAs. 3. Useful when you want to prove something mathematically about images – this is an area of pure mathematics 4. Useful when you want to create new morphological operators. 5. Useful when you want to extend to complex numbers, quaternions, octonions, gray level and color images.

Erosion

Erosion • (Minkowski subtraction) • The contraction of a binary region (aka, region shrinking) • Use a structuring element on binary image data to produce a new binary image • Structuring elements (SE) are simply patterns that are matched in the image • It is useful to explain operation of erosion and dilation in different ways.

Intuition for Erosion • Removes isolated noisy pixels. • Smoothes object boundary. • Removes the outer layer of object pixels, ie, object becomes slightly smaller.

Properties of Erosion: • Erosion removes spiky edges – objects are light (white in binary) – decreases geometrical area of object – sets contour pixels of object to background value – smoothes small positive grey level regions

Erosion Example

Erosion explained pixel by pixel When whole SE included, origin point copied to new image B A • • • •

Structuring Element in Erosion Example

How Erosion Works? • During erosion, a pixel is turned on at the image pixel under the structuring element origin only when the pixels of the structuring element match the pixels in the image • Both ON and OFF pixels should match. • This example erodes regions horizontally from the right.

Structuring Element in Erosion Example

Mathematical Definition of Erosion Ø Erosion is the morphological dual to dilation. Ø It combines two sets using the vector subtraction of set elements. Ø Let denotes the erosion of A by B

Erosion in terms of other operations: translation and intersection Ø Erosion can also be defined in terms of translation Ø In terms of intersection Observe that vector here is negative

Reminder - this was A • •

Erosion illustrated in terms of intersection and negative translation Observe negative translation Because of negative shift the origin is here • • • • • •

Erosion formula and intuitive example Center here will not add a point to the Result Center of B is here and adds a point

Example of Erosions with various sizes of structuring elements Structuring Element Pablo Picasso, Pass with the Cape, 1960

Mathematical Properties of Erosion Ø Erosion is not commutative! Ø Extensivity Ø Erosion is dereasing Ø Chain rule The chain rule is as sharp operator in Cube Calculus used in logic synthesis. There are more similarities of these algebras

Mathematical Properties of Erosion Ø Translation Invariance Ø Linearity Ø Containment Ø Decomposition of structuring element

Duality Relationship between erosion and dilation Ø Dilation and Erosion transformation bear a marked similarity, in that what one does to image foreground and the other does for the image background. Ø , the reflection of B, is defined as Ø Erosion and Dilation Duality Theorem Similar but not identical to De Morgan rule in Boolean Algebra Observe negative value which is mirror image reflection of B

Rules for execution of Erosion • To compute the erosion of a binary input image by the structuring element: 1. For each foreground pixel superimpose the structuring element 2. If for every pixel in the structuring element, the corresponding pixel in the image underneath is a foreground pixel, then the input pixel is left as it is 3. Otherwise, if any of the corresponding pixels in the image are background, however, the input pixel is set to background value

Duality of Dilation and Erosion • Erosion is the dual of dilation i. e. eroding foreground pixels is equivalent to dilating the background pixels.

Duality of Erosion and Dilation • Erosion is the dual of dilation i. e. eroding foreground pixels is equivalent to dilating the background pixels. Result of dilation of background

Erosion through dilation and negation Result of dilation of background negate So finally we have for erosion:

A large example of Erosion

Duality Relationship between erosion and dilation • • Easily visualized on binary image Template created with known origin Another look at duality • Template stepped over entire image – similar to correlation • Dilation – if origin == 1 -> template unioned – resultant image is large than original • Erosion – only if whole template matches image – origin = 1, result is smaller than original

Morphological Operations in terms of more general neighborhoods

Erode and Dilate in terms of more general neighborhoods Yet another loook at Duality Relationship between erosion and dilation

Erode and Binary Contour in Matlab Erosion can be used to find contour Dilation can be also used for it think how?

Edge detection

Edge detection This subtraction is set theoretical Dilate - original Now you need to invert the image There are more methods for edge detection

More Details about Opening & Closing • Opening and Closing are two important operators from mathematical morphology • They are both derived from the fundamental operations of erosion and dilation • They are normally applied to binary images

Close = Dilate next Erode Open = Erode next Dilate Original image Open and Close eroded dilated eroded Close Open

Open

Open • An erosion followed by a dilation • It serves to eliminate noise • Does not significantly change an object’s size

Opening • What combination of erosion and dilation gives: – cleaned binary image – object is the same size as in original Original

Opening • Erode original image. • Dilate eroded image. • Smooths object boundaries, eliminates noise (isolated pixels) and maintains object size. Original Erode Dilate

Opening • • • Opening is defined as an erosion followed by a dilation using the same structuring element The basic effect of an opening is similar to erosion Tends to remove some of the foreground pixels from the edges of regions of foreground pixels Less destructive than erosion The exact operation is determined by a structuring element.

Opening • • • Erosion can be used to eliminate small clumps of undesirable foreground pixels, e. g. “salt noise” However, it affects all regions of foreground pixels indiscriminately Opening gets around this by performing both an erosion and a dilation on the image

Close

Close • Dilation followed by erosion • Serves to close up cracks in objects and holes due to pepper noise • Does not significantly change object size

Closing • What combination of erosion and dilation gives: – cleaned binary image – object is the same size as in original Original

Closing • Dilate original image. • Erode dilated image. • Smooths object boundaries, eliminates noise (holes) and maintains object size. Original Dilate Erode

Closing • • • Closing, like its dual operator opening, is derived from the fundamental operations of erosion and dilation. Normally applied to binary images Tends to enlarge the boundaries of foreground regions Less destructive of the original boundary shape The exact operation is determined by a structuring element.

Closing • Closing is opening performed in reverse. It is defined simply as a dilation followed by an erosion using the same

Closing

Mathematical Definitions of Opening and Closing Ø Opening and closing are iteratively applied dilation and erosion Opening Closing

Opening and Closing Difference is only in corners

Mathematical Idea = Idempotency: Opening and Closing are idempotent Ø Their reapplication has not further effects to the previously transformed result

Mathematical Properties of Opening and Closing Ø Translation invariance Ø Antiextensivity of opening Ø Extensivity of closing Ø Duality

Example of Openings with various sizes of structuring elements Structuring Element Pablo Picasso, Pass with the Cape, 1960

Example of Closings with various sizes of structuring Example of Closing elements Structuring Element

Morphological Filtering

Morphological Filtering Ø Main idea of Morphological Filtering: Ø Examine the geometrical structure of an image by matching it with small patterns called structuring elements at various locations Ø By varying the size and shape of the matching patterns, we can extract useful information about the shape of the different parts of the image and the interrelations of these shapes. Ø We combine set-theoretical and morphological operations:

Example 1: Morphological filtering Ø Noisy image will break down OCR systems Clean original image Noisy image

Morphological filtering (MF) By applying MF, we increase the OCR accuracy! Restored image

Rank Filter Median Input 1 operation 2 operations

Postprocessing • Opening followed by closing. • Removes noise and smoothes boundaries.

Conclusion on Morphological Approaches 1. Mathematical morphology is an approach for processing digital image based on its shape 2. The language of morphology is set theory 3. The basic morphological operations are erosion and dilation 4. Morphological filtering can be developed to extract useful shape information 5. Methods can be extended to more values and more dimensions 6. Nice mathematics can be formulated - non-linear

Conclusion on Morphological 1. Segmentation separates an image into regions. 2. Segmentation of Image has many applications 3. Morphological methods can be used for Segmentation 4. Use of histograms for brightness based segmentation. 1. Peak corresponds to object. 2. Height of peak corresponds to size of object. 5. If global image histogram is multimodal, local image region histogram may be bimodal. 6. Local thresholds can give better segmentation.

Conclusion • Postprocessing uses morphological operators. • Same as convolution only use Boolean operators instead of multiply and add. – Erosion clears noise, makes smaller. – Dilation fills in holes, makes larger. • Postprocessing – Opening and closing to clean binary images. – Repeated erosion with special rule produces skeleton.

Problems for students for homework 1

Problems 1 - 6 • 1. Write LISP or C++ program for dilation of binary images • 2. Modify it to do erosions (few types) • 3. Modify it to perform shift and exor operation and shift and min operation • 4. Generalize to multi-valued algebra • 5. Create a comprehensive theory of multi-valued morphological algebra and its algorithms (publishable). • 6. Write a program for inspection of Printed Circuit Boards using morphological algebra.

Problem 7. • Electric Outlet Extraction has been done using a combination of Canny Edge Detection and Hough Transforms • Write a LISP program that will use only basic morphological methods for this application.

Image Processing for electric outlet, how? • Currently there are many, many ways to approach this problem – – – – – Segmentation Edge Detection DPC compression FFT IFFT DFT Thinning Growing Haar Transform Hex Rotate Alpha filtering DPC compression Perimeter Fractal Gaussian Filter Band Pass Filter Homomorphic Filtering Contrast Sharper Least Square Restoration Warping Dilation

Image Processing, how? • Create morphological equivalents of other image processing methods. • New, publishable, use outlet problem as example to illustrate

Problem 8. Openings and Closings as examples. • The solution here is to follow up one operation with the other. • An opening is defined as an erosion operation followed by dilation using the same structuring element. • Similarly, a closing is dilation followed by erosion. • Define and implement other combined operations.

Problems 9 - 12. • 9. Generalize binary morphological algebra from 2 dimensional to 3 dimensional images. What are the applications. • 10. Write software for 9. • 11. Generalize your generalized multi-valued morphological algebra to 3 or more dimensions, theoretically, find properties and theorems like those from this lecture. • 12. Write software for 11.

Problem 13 • Mathematical morphology uses the concept of structuring elements to analyze image features. • A structuring element is a set of pixels in some arrangement that can extract shape information from an image. • Typical structuring elements include rectangles, lines, and circles. • Think about other structuring elements and their applications.

Morphological Operations: Matlab BWMORPH Perform morphological operations on binary image. BW 2 = BWMORPH(BW 1, OPERATION) applies a specific morphological operation to the binary image BW 1. BW 2 = BWMORPH(BW 1, OPERATION, N) applies the operation N times. N can be Inf, in which case the operation is repeated until the image no longer changes. OPERATION is a string that can have one of these values: 'close' Perform binary closure (dilation followed by erosion) 'dilate' Perform dilation using the structuring elementones(3) 'erode' 'fill' 'open' Perform erosion using the structuring elementones(3) Fill isolated interior pixels (0's surrounded by 1's) Perform binary opening (erosion followed bydilation) 'skel' With N = Inf, remove pixels on the boundariesof objects without allowing objects to break apart demos/demo 9 morph/

Sources D. A. Forsyth, University of New Mexico, Qigong Zheng, Language and Media Processing Lab Center for Automation Research University of Maryland College Park October 31, 2000 John Miller Matt Roach J. W. V. Miller and K. D. Whitehead The University of Michigan-Dearborn Spencer Lustor Light Works Inc. C. Rössl, L. Kobbelt, H. -P. Seidel, Max-Planck Institute for, Computer Science, Saarbrücken, Germany LBA-PC 4 Howard Schultz Shreekanth Mandayam ECE Department Rowan University D. A. Forsyth, University of New Mexico