EECS 274 Computer Vision Linear Filters and Edges
- Slides: 67
EECS 274 Computer Vision Linear Filters and Edges
Linear filters and edges • • Linear filters Scale space Gaussian pyramid and wavelets Edges • Reading: Chapters 4 and 5 of FP, Chapter 3 of S
Linear filters • General process: – Form new image whose pixels are a weighted sum of original pixel values, using the same set of weights at each point • Properties – Output is a linear function of the input – Output is a shift-invariant function of the input (i. e. shift the input image two pixels to the left, the output is shifted two pixels to the left) • Example: smoothing by averaging – form the average of pixels in a neighborhood • Example: smoothing with a Gaussian – form a weighted average of pixels in a neighborhood • Example: finding a derivative – form a weighted average of pixels in a neighborhood
Convolution • Represent these weights as an image, h • h is usually called the kernel • Operation is called convolution • Notation in textbook: – it’s associative • Notice weird order of indices – all examples can be put in this form – it’s a result of the derivation expressing any shift-invariant linear operator as a convolution
Convolution
Smoothing by averaging
Smoothing with a Gaussian • Smoothing with an average actually doesn’t compare at all well with a defocussed lens – Most obvious difference is that a single point of light viewed in a defocussed lens looks like a fuzzy blob; but the averaging process would give a little square – Narrow vertical and horizontal bards known as ringing effects • A Gaussian gives a good model of a fuzzy blob
An isotropic Gaussian • The picture shows a smoothing kernel proportional to (which is a reasonable model of a circularly symmetric fuzzy blob)
Smoothing with a Gaussian
Differentiation and convolution • Recall • Now this is linear and shift invariant, so must be the result of a convolution • We could approximate this as (which is obviously a convolution; it’s not a very good way to do things, as we shall see)
Finite differences Partial derivative in y axis, respond strongly to horizontal edges Partial derivative in x axis, respond strongly to vertical edges
Spatial filter • Different approximate operators
Roberts operator One of the earliest edge detection algorithm by Lawrence Roberts Two kernels: Symmetric, tuned to diagonal
Sobel operator One of the earliest edge detection algorithm by Irwin Sobel
Noise • Simplest noise model – independent stationary additive Gaussian noise – the noise value at each pixel is given by an independent draw from the same normal probability distribution • Issues – this model allows noise values that could be greater than maximum camera output or less than zero – for small standard deviations, this isn’t too much of a problem - it’s a fairly good model – independence may not be justified (e. g. damage to lens) – may not be stationary (e. g. thermal gradients in the ccd)
Additive Gaussian noise sigma=16
Finite differences and noise • Finite difference filters respond strongly to noise – obvious reason: image noise results in pixels that look very different from their neighbors • Generally, the larger the noise the stronger the response • What is to be done? – intuitively, most pixels in images look quite a lot like their neighbors – this is true even at an edge; along the edge they’re similar, across the edge they’re not – suggests that smoothing the image should help, by forcing pixels different to their neighbors (=noise pixels? ) to look more like neighbors
Finite differences responding to noise σ=0. 03 σ=0. 09 Increasing noise -> (this is zero mean additive Gaussian noise) Difference operation is strongly influenced by noise (the image is increasingly grainy)
Smoothed noise Smoothing stationary additive Gaussian noise results in signals where pixel values tend to increasingly similar to the value of neighboring pixels (as filter kernel causes correlation)
Smoothing reduces noise • Generally expect pixels to “be like” their neighbors – surfaces turn slowly – relatively few reflectance changes • Generally expect noise processes to be independent from pixel to pixel • Implies that smoothing suppresses noise, for appropriate noise models • Scale – the parameter in the symmetric Gaussian – as this parameter goes up, more pixels are involved in the average – and the image gets more blurred – and noise is more effectively suppressed
Effects of smoothing The effects of smoothing • Each row shows smoothing with Gaussians of different width • Each column shows images with different Gaussian noise
Gradients and edges • Points of sharp change in an image are interesting: – – change in reflectance change in object change in illumination noise • Sometimes called edge points • General strategy – determine image gradient – now mark points where gradient magnitude is particularly large wrt neighbors (ideally, curves of such points) In one dimension, the 2 nd derivative of a signal is zero when the derivative magnitude is extremal a good place to look for edge is where the second derivative is zero
Smoothing and differentiation • Issue: noise – smooth before differentiation – two convolutions to smooth, then differentiate? – actually, no - we can use a derivative of Gaussian filter • because differentiation is convolution, and convolution is associative Smoothed derivative of Gaussian filter in the x-direction Smoothed derivative of Gaussian filter in the y-direction
Scale of smoothing 1 pixel 3 pixels 7 pixels The scale of the smoothing filter affects derivative estimates, and also the semantics of the edges recovered
Image gradient • Gradient equation: • Represents direction of most rapid change in intensity • Gradient direction: • The edge strength is given by the gradient magnitude
Theory of edge detection Ideal edge Unit step function: Image intensity (brightness):
Theory of edge detection • Image intensity (brightness): • Partial derivatives (gradients): • Squared gradient: Edge Magnitude: Edge Orientation: (normal of the edge) Rotationally symmetric, non-linear operator
Theory of edge detection • Image intensity (brightness): • Partial derivatives (gradients): • Laplacian: Rotationally symmetric, linear operator zero-crossing
Discrete edge operators • How can we differentiate a discrete image? Finite difference approximations: Convolution masks :
Discrete edge operators • Second order partial derivatives: • Laplacian : Convolution masks : 0 1 -4 1 0 or 1 4 -20 4 1 (more accurate)
Effects of noise • Consider a single row or column of the image – Plotting intensity as a function of position gives a signal Where is the edge?
Solution: Smooth first Where is the edge? Look for peaks in
Derivative theorem of convolution …saves us one operation
Laplacian of Gaussian (Lo. G) By Marr and Hildreth Laplacian of Gaussian operator Where is the edge? Zero-crossings of bottom graph !
2 D Gaussian edge operators Gaussian • Derivative of Gaussian (Do. G) Laplacian of Gaussian Mexican Hat (Sombrero) is the Laplacian operator: Can be substituted with two Do. Gs
threshold sigma=2 scale contrast=1 LOG zero crossings contrast=4 sigma=4 How to find best scale and threshold?
Ringing effects Lo. G is not oriented (isotropic) We still have unfortunate behavior at corners and trihedral areas
σ=1 pixel σ=2 pixel From thick trails to curves (formed by points) Three major issues: 1) The gradient magnitude at different scales is different; which should we choose? 2) The gradient magnitude is large along thick trail; how do we identify the significant points? 3) How do we link the relevant points up into curves?
Gradient-based edge detectors Trail of large gradient magnitude Appropriate cutting direction Peak in this direction • Wish to mark points along the curve where the magnitude is biggest • Looking for a maximum along a slice normal to the curve (non-maximum suppression) • These points should form a curve • Two algorithmic issues: • at which point is the maximum • where is the next one?
Non-maximum suppression • Check if pixel is local maximum along gradient direction – requires checking interpolated pixels p and r
Edge following t Predicting the next edge point Assume the marked point is an edge point. Then we construct the tangent to the edge curve (which is normal to the gradient at that point) and use this to predict the next points (here either s or t) Edge following: edge points occur along curve like chains
Edge points coarse scale, high threshold σ=4 pixels fine scale, high threshold σ=1 pixel coarse scale, low threshold σ=4 pixels
Canny edge operator • Smooth image I with 2 D Gaussian: • Find local edge normal directions for each pixel • Compute edge magnitudes • Locate edges by finding zero-crossings along the edge normal directions (non-maximum suppression)
Canny edge detector Original image Magnitude of gradient
Canny edge detector After non-maximum suppression
Canny edge detector original Canny with • The choice of – large – small Canny with depends on desired behavior detects large scale edges detects fine features
Difference of Gaussians (Do. G) • Laplacian of Gaussian can be approximated by the difference between two different Gaussians
Do. G edge detection (a) (b)-(a)
Unsharp masking – = blurred positive +a increase details in bright area =
Edge thresholding • Standard Thresholding: • Can only select “strong” edges. • Does not guarantee “continuity” (i. e. , lead to broken edge curves) • Hysteresis based Thresholding (use two thresholds) Example: For “maybe” edges, decide on the edge if neighboring pixel is a strong edge
Remaining issues • Check that maximum value of gradient value is sufficiently large – drop-outs? use hysteresis • use a high threshold to start edge curves and a low threshold to continue them
Notes • Something nasty is happening at corners • Scale affects contrast • Edges are not bounding contours
Scale space • Framework for multi-scale signal processing • Define image structure in terms of scale with kernel size, σ • Find scale invariant operations • See Scale-Space Theory in Computer Vision by Tony Lindeberg
Orientation representations • The gradient magnitude is affected by illumination changes – but it’s direction isn’t • We can describe image patches by the swing of the gradient orientation • A variety of specialized corner detector • Important types: – constant window • small gradient magnitudes – edge window • few large gradient magnitudes in one direction – flow window • many large gradient magnitudes in one direction – corner window • large gradient magnitudes that change
Around the corners and in the textured regions, the orientation vector swings sharply
Representing windows Looking at variations • Types – constant • small eigenvalues – edge • one medium, one small – flowf • one large, one small – corner • two large eigenvalues
Plotting in ellipses to understand the matrix (variation of gradient) of 3 × 3 window Orientation field Major and minor axes are along the eigenvectors of H, and the extent corresponds to the size of eigenvalues
Orientation field Plotting in ellipses to understand the matrix (variation of gradient) of 5 × 5 window
Corners • Harris corner detector • Moravec corner detector • SIFT descriptors (more in the next lectures)
Filters are templates • Applying a filter at some point can be seen as taking a dotproduct between the image and some vector • Filtering the image is a set of dot products convolution is equivalent to taking the dot product of the filter with an image patch • Insight – filters look like the effects they are intended to find – filters find effects they look like derivative of Gaussian used as edge detection
Normalized correlation • Think of filters of a dot product – now measure the angle – i. e normalized correlation output is filter output, divided by root sum of squares of values over which filter lies – cheap and efficient method for finding patterns • Tricks: – ensure that filter has a zero response to a constant region (helps reduce response to irrelevant background) – subtract image average when computing the normalizing constant (i. e. subtract the image mean in the neighborhood) – absolute value deals with contrast reversal
Finding hands Controlling TV set based on normalized correlation score of patterns by Freeman et al, 98
Anisotropic scaling • Symmetric Gaussian smoothing tends to blur out edges rather aggressively • Prefer an oriented smoothing operator that smoothes – aggressively perpendicular to the gradient – little along the gradient • Also known as edge preserving smoothing • Formulated with diffusion equation
Diffusion equation for anisotropic filter • PDE that describes fluctuations in a material undergoing diffusion • For isotropic filter • For anisotropic filter
Anisotropic filtering Often used in graphics for textured surface to reduce aliasing effects
Anisotropic diffusion Input image with noise Linear isotropic diffusion (simple Gaussian) Nonlinear anisotropic diffusion Nonlinear isotropic diffusion (edge skipping) Perona and Malik 90
Bilateral filtering • Preserving edges – Replace pixel’s value by a weighted average of its neighbors in both space and intensity – Controlled by range and spatial parameters One iteration Tomasi and Manduchi 98 Multiple iterations
- Difference between linear and nonlinear spatial filters
- Human vision vs computer vision
- Non linear pipeline processors
- Linear algebra for computer vision
- Fresco y mas 274
- Cs 274
- Cmput 274
- Msz en 62305
- Cs 274
- Cse 274
- Rct-274
- Wss 274
- Delibera 274 tim
- Wet etch clean and filter
- Vertical blinds
- Erp bootcamp
- ำ
- Impulse invariant method formula
- Filters in weka
- Lymph nodes: “filters of the blood”
- Lymph nodes: “filters of the blood”
- Thomas kuehl
- Ironport outbreak filters
- Discriminative training of kalman filters
- Columbus industries
- Types of analog filters
- Disc filters galaxy 4 spin klin series
- Hailstop hail guard
- Design of analog filters schaumann
- 273
- Large volume parentrals
- Our personal filters assumptions
- Selective filter
- Argument vs opinion
- Pre set widget filters jlv
- Spectral transformation of iir filters
- Advantage of active filter
- Active vs passive low pass filter
- Bacterial filters ppt
- Perceptual filters
- Emfab filters
- Signal processing filter
- Authentication filters in mvc 5
- Universal xss
- Event list filters packet tracer
- Skimage.filters.gaussian
- Barracuda web filter price
- Computer vision models learning and inference
- Mathematical foundations of computer graphics and vision
- Computer vision models learning and inference
- Computer vision models learning and inference pdf
- Computer and robot vision
- Computer vision: models, learning, and inference
- Simple and multiple linear regression
- Appropriate non linear text
- Non linear plot examples
- Linear and non-linear multimedia
- Linear impulse equation
- Persamaan linear simultan
- Left linear grammar example
- Difference between linear and nonlinear equation
- Eliminasi gauss naif
- Qm for windows
- 16-385 cmu
- Kalman filter computer vision
- T11 computer
- Berkeley computer vision
- Multiple view geometry in computer vision pdf