Computer Vision CSE 803 A brief intro Stockman
Computer Vision: CSE 803 A brief intro Stockman MSU/CSE Fall 2009
Computer Vision n n What are the goals of CV? What are the applications? How do humans perceive the 3 D world via images? Some methods of processing images. What are the major research areas? Stockman MSU/CSE Fall 2009
Goal of computer vision n n Make useful decisions about real physical objects and scenes based on sensed images. Alternative (Aloimonos and Rosenfeld): goal is the construction of scene descriptions from images. How do you find the door to leave? How do you determine if a person is friendly or hostile? . . an elder? . . a possible mate? Stockman MSU/CSE Fall 2009
Critical Issues n n Sensing: how do sensors obtain images of the world? Information/features: how do we obtain color, texture, shape, motion, etc. ? Representations: what representations should/does a computer [or brain] use? Algorithms: what algorithms process image information and construct scene descriptions? Stockman MSU/CSE Fall 2009
Root and soil next to glass Stockman MSU/CSE Fall 2009
Images: 2 D projections of 3 D n n 3 D world has color, texture, surfaces, volumes, light sources, objects, motion, betweeness, adjacency, connections, etc. 2 D image is a projection of a scene from a specific viewpoint; many 3 D features are captured, some are not. Brightness or color = g(x, y) or f(row, column) for a certain instant of time Images indicate familiar people, moving objects or animals, health of people or machines Stockman MSU/CSE Fall 2009
Image receives reflections n n n Light reaches surfaces in 3 D Surfaces reflect Sensor element receives light energy Intensity matters Angles matter Material maters Stockman MSU/CSE Fall 2009
Simple objects: simple image? Stockman MSU/CSE Fall 2009
Where is the sun? Stockman MSU/CSE Fall 2009
CCD Camera has discrete elts n n n Stockman MSU/CSE Fall 2009 Lens collects light rays CCD elts replace chemicals of film Number of elts less than with film (so far)
Camera + Programs = Display n n Camera inputs to frame buffer Program can interpret data Program can add graphics Program can add imagery Stockman MSU/CSE Fall 2009
Some image format issues Spatial resolution; intensity resolution; image file format Stockman MSU/CSE Fall 2009
Resolution is “pixels per unit of length” n n Stockman MSU/CSE Fall 2009 Resolution decreases by one half in cases at left Human faces can be recognized at 64 x 64 pixels per face
Features detected depend on the resolution n Stockman MSU/CSE Fall 2009 Can tell hearts from diamonds Can tell face value Generally need 2 pixels across line or small region (such as eye)
Human eye as a spherical camera n n n n 100 M sensing elts in retina Rods sense intensity Cones sense color Fovea has tightly packed elts, more cones Periphery has more rods Focal length is about 20 mm Pupil/iris controls light entry • Eye scans, or saccades to image details on fovea • 100 M sensing cells funnel to 1 M optic nerve connections to the brain Stockman MSU/CSE Fall 2009
Look at some CV applications Graphics or image retrieval systems; Geographical: GIS; Medical image analysis; manufacturing Stockman MSU/CSE Fall 2009
Aerial images & GIS n n Stockman MSU/CSE Fall 2009 Aerial image of Wenatchie River watershed Can correspond to map; can inventory snow coverage
Medical imaging is critical n n n Stockman MSU/CSE Fall 2009 Visible human project at NLM Atlas for comparison Testbed for methods
Manufacturing case n n n 100 % inspection needed Quality demanded by major buyer Assembly line updated for visual inspection well before today’s powerful computers Stockman MSU/CSE Fall 2009
Simple Hole Counting Alg. n n n Customer needs 100% inspection About 100 holes Big problem if any hole missing Implementation in the 70’s Alg also good for counting objects See auxiliary slides Stockman MSU/CSE Fall 2009
Some hot new applications n n Phototourism: from hundreds of overlapping images, maybe some from cell phones, construct a 3 D textured model of the landmark[s] Photo-GPS: From a few cell phone images “the web” tells you where you are located [perhaps using the data as above] Stockman MSU/CSE Fall 2009
Image processing operations Thresholding; Edge detection; Motion field computation Stockman MSU/CSE Fall 2009
Find regions via thresholding Region has brighter or darker or redder color, etc. If pixel > threshold then pixel = 1 else pixel = 0 Stockman MSU/CSE Fall 2009
Example red blood cell image n n Many blood cells are separate objects Many touch – bad! Salt and pepper noise from thresholding How useable is this data? Stockman MSU/CSE Fall 2009
sign = imread('Images/stop. Sign. jpg', 'jpg'); red = (sign(: , 1)>120) & (sign(: , 2)<100) & (sign(: , 3)<80); out = red*200; imwrite(out, 'Images/stop. Red 120. jpg', 'jpg') Stockman MSU/CSE Fall 2009
sign = imread('Images/stop. Sign. jpg', 'jpg'); red = (sign(: , 1)>120) & (sign(: , 2)<100) & (sign(: , 3)<80); out = red*200; imwrite(out, 'Images/stop. Red 120. jpg', 'jpg') Stockman MSU/CSE Fall 2009
Thresholding is usually not trivial Stockman MSU/CSE Fall 2009
Can cluster pixels by color similarity and by adjacency Original RGB Image Color Clusters by K-Means Stockman MSU/CSE Fall 2009
Detect Motion via Subtraction n n Differences computed over time rather than over space Stockman MSU/CSE Fall 2009 Constant background Moving object Produces pixel differences at boundary Reveals moving object and its shape
Two frames of aerial imagery Video frame N and N+1 shows slight movement: most pixels are same, just in different locations. Stockman MSU/CSE Fall 2009
Best matching blocks between video frames N+1 to N (motion vectors) The bulk of the vectors show the true motion of the airplane taking the pictures. The long vectors are incorrect motion vectors, but they do work well for compression of image I 2! Best matches from 2 nd to first image shown as vectors overlaid on the 2 nd image. (Work by Dina Eldin. ) Stockman MSU/CSE Fall 2009
Gradient from 3 x 3 neighborhood Estimate both magnitude and direction of the edge. Stockman MSU/CSE Fall 2009
2 rows of intensity vs difference Stockman MSU/CSE Fall 2009
Boundaries not always found well Stockman MSU/CSE Fall 2009
Canny edge operator Stockman MSU/CSE Fall 2009
Mach band effect shows human bias Biology consistent with image processing operations Stockman MSU/CSE Fall 2009
Human bias and illusions supports receptive field theory of edge detection Stockman MSU/CSE Fall 2009
Color and shading n n Used heavily in human vision Color is a pixel property, making some recognition problems easy Visible spectrum for humans is 400 nm (blue) to 700 nm (red) Machines can “see” much more; ex. X-rays, infrared, radio waves Stockman MSU/CSE Fall 2009
Imaging Process (review) Stockman MSU/CSE Fall 2009
Factors that Affect Perception • Light: the spectrum of energy that illuminates the object surface • Reflectance: ratio of reflected light to incoming light • Specularity: highly specular (shiny) vs. matte surface • Distance: distance to the light source • Angle: angle between surface normal and light source how sensitive is the sensor • Sensitivity Stockman MSU/CSE Fall 2009
CV: Perceiving 3 D from 2 D Many cues from 2 D images enable interpretation of the structure of the 3 D world producing them Stockman MSU/CSE Fall 2009
Many 3 D cues How can humans and other machines reconstruct the 3 D nature of a scene from 2 D images? What other world knowledge needs to be added in the process? Stockman MSU/CSE Fall 2009
What about models for recognition “recognition” = to know again; How does memory store models of faces, rooms, chairs, etc. ? Stockman MSU/CSE Fall 2009
Some methods: recognize n n Via Via geometric alignment: CAD trained neural net parts of objects and how they join the function/behavior of an object Stockman MSU/CSE Fall 2009
summary n n n Images have many low level features Can detect uniform regions and contrast Can organize regions and boundaries Human vision uses several simultaneous channels: color, edge, motion Use of models/knowledge diverse and difficult Last 2 issues difficult in computer vision Stockman MSU/CSE Fall 2009
- Slides: 45