Introduction to Sensors Computer Vision CSC I 6716

Introduction to Sensors Computer Vision CSC I 6716 Spring 2004 3 D Computer Vision and Video Computing Instructor: Zhigang Zhu Part 1 Topic 2: Sensors http: //www-cs. engr. ccny. cuny. edu/~zhu/Vision. Course-2004. html

Introduction to Computer Vision Acknowledgements The slides in this lecture were adopted and modified from lectures by Professor Allen Hanson University of Massachusetts at Amherst

Introduction to Computer Vision n Static monocular reflectance data (monochromic or color) l l l n n Motion sequences (camcorders) Stereo (2 cameras) Range data (Range finder) Non-visual sensory data l l l n Films Video cameras (with tapes) Digital cameras (with memory) infrared (IR) ultraviolet (UV) microwaves Many more Sensors

Introduction to Computer Vision The Electromagnetic Spectrum Visible Spectrum 700 nm C=fl E f 400 nm

Introduction to Computer Vision The Human Eye

Introduction to The Eye Computer Vision Retina Rods Cones n The Retina: l l n rods (low-level light, night vision) cones (color-vision) synapses optic nerve fibers Sensing and low-level processing layer l l 125 millions rods and cones feed into 1 million nerve fibers Cell arrangement that respond to horizontal and vertical lines

Introduction to Computer Vision n n Film, Video, Digital Cameras Black and White (Reflectance data only) Color (Reflectance data in three bands - red, green, blue)

Introduction to Color Images Computer Vision Blue Spatial Resolution Green ‘Dimensions’ of an Image Spectra Resolution Spatial (x, y) Depth (no. of components) Number of bits/channel Temporal (t) Radiometric Resolution Temporal Resolution Red Pixel

Introduction to Computer Vision Across the EM Spectrum Crab Nebula

Introduction to Computer Vision Across the EM Spectrum Cargo inspection using Gamma Rays Mobile Vehicle and Cargo Inspection System (VACIS®) Gamma rays are typically waves of frequencies greater than 1019 Hz Gamma rays can penetrate nearly all materials and are therefore difficult to detect Courtesy: Science Applications International Corporation (SAIC),

Introduction to Computer Vision Across the EM Spectrum Cargo inspection using Gamma Rays Mobile Vehicle and Cargo Inspection System (VACIS®) Gamma rays are typically waves of frequencies greater than 1019 Hz Gamma rays can penetrate nearly all materials and are therefore difficult to detect Courtesy: Science Applications International Corporation (SAIC),

Introduction to Computer Vision Across the EM Spectrum Cargo inspection using Gamma Rays Mobile Vehicle and Cargo Inspection System (VACIS®) Gamma rays are typically waves of frequencies greater than 1019 Hz Gamma rays can penetrate nearly all materials and are therefore difficult to detect Courtesy: Science Applications International Corporation (SAIC),

Introduction to Computer Vision n Medical X-Rays Across the EM Spectrum

Introduction to Computer Vision n Chandra X-Ray Satellite Across the EM Spectrum

Introduction to Computer Vision n Across the EM Spectrum From X-Ray images to 3 D Models: CT Scans

Introduction to Computer Vision Flower Patterns in Ultraviolet Dandelion - UV Potentilla n Across the EM Spectrum

Introduction to Computer Vision n Messier 101 in Ultraviolet Across the EM Spectrum

Introduction to Computer Vision n Traditional images Across the EM Spectrum

Introduction to Computer Vision n Across the EM Spectrum Non-traditional Use of Visible Light: Range

Introduction to Computer Vision n Across the EM Spectrum Scanning Laser Rangefinder

Introduction to Computer Vision n Across the EM Spectrum IR: Near, Medium, Far (~heat)

Introduction to Computer Vision n Across the EM Spectrum IR: Near, Medium, Far (~heat)

Introduction to Computer Vision n Across the EM Spectrum IR: Finding chlorophyll -the green coloring matter of plants that functions in photosynthesis

Introduction to Computer Vision n Across the EM Spectrum (Un)Common uses of Microwaves CD Movie Exploding Water Movie

Introduction to Computer Vision n Across the EM Spectrum Microwave Imaging: Synthetic Aperture Radar (SAR) San Fernando Valley Tibet: Lhasa River Red: L-band (24 cm) Green: C-band (6 cm) Blue: C/L Athens, Greece Thailand: Phang Hoei Range

Introduction to Computer Vision n Across the EM Spectrum Radar in Depth: Interferometric Synthetic Aperture Radar - IFSAR (elevation)

Introduction to Computer Vision n Across the EM Spectrum Low Altitude IFSAR elevation, automatic, in minutes Elevation from aerial stereo, manually, several days

Introduction to Computer Vision n Across the EM Spectrum Radio Waves (images of cosmos from radio telescopes)

Introduction to Computer Vision n Single Camera (no stereo) Stereo Geometry

Introduction to Stereo Geometry Computer Vision LEFT CAMERA RIGHT CAMERA P(X, Y, Z) Film plane f = focal length Film plane pl(x, y) pr(x, y) Optical Center f = focal length Optical Center B = Baseline

Introduction to Stereo Geometry Computer Vision LEFT IMAGE RIGHT IMAGE P Pl(xl, yl) Pr(xr, yr) Disparity = xr - xl ≈ depth

Introduction to Computer Vision n A Short Digression Stereoscopes Stereo Images

Introduction to Computer Vision Darjeeling Suspension Bridge Stereo Images

Introduction to Computer Vision Picture of you?

Introduction to Computer Vision n Stereograms Stereo

Introduction to Computer Vision Stereo X-Ray

Introduction to Computer Vision n Range Sensors Light Striping David B. Cox, Robyn Owens and Peter Hartmann Department of Biochemistry University of Western Australia http: //mammary. nih. gov/reviews/lactation/Hartmann 001/

Introduction to Computer Vision n A mosaic is created from several images Mosaics

Introduction to Computer Vision n Stabilized Video Mosaics

Introduction to Mosaics Computer Vision n Depth from a Video Sequence (single camera) GPS Height H from Laser Profiler P(X, Y, Z)

Introduction to Computer Vision n Brazilian forest…. . made at UMass CVL Mosaics

Introduction to Computer Vision n Natural Variation in Object Classes: l n Color, texture, size, shape, parts, and relations Variations in the Imaging Process l l l n Why is Vision Difficult? Lighting (highlights, shadows, brightness, contrast) Projective distortion, point of view, occlusion Noise, sensor and optical characteristics Massive Amounts of Data l 1 minute of 1024 x 768 color video = 4. 2 gigabytes (Uncompressed)

Introduction to The Need for Knowledge Computer Vision Variation Knowledge Motion Context Function Shape Purpose Specific Objects Generic Objects Structure Size

Introduction to Computer Vision The Figure Revealed

Introduction to Computer Vision The Effect of Context

Introduction to Computer Vision The Effect of Context - 2

Introduction to Computer Vision n …. a collection of objects: Context, cont.

Introduction to Computer Vision n The objects as hats: Context

Introduction to Computer Vision Context n And as something else…. . n ‘To interpret something is to give it meaning in context. ’

Introduction to Computer Vision n Vision System Components …. . at the low (image) level, we need l l Ways of generating initial descriptions of the image data Method for extracting features of these descriptions Ways of representing these descriptions and features Usually, cannot initially make use of general world knowledge IMAGE (numbers) DESCRIPTION (symbols)

Introduction to Computer Vision n …. at the intermediate level, we need l l n Vision System Components Symbolic representations of the initial descriptions Ways of generating more abstract descriptions from the initial ones (grouping) Ways of accessing relevant portions of the knowledge base Ways of controlling the processing Intermediate level processes should be capable of being used top-down (knowledge-directed) or bottom-up (datadirected) IMAGE IINTERMEDIATE DESCRIPTIONS KNOWLEDGE

Introduction to Computer Vision n Vision System Components …. at the high (interpretation) level, we need l Ways of representing world knowledge n n l Mechanisms for Interferencing n n l l Objects Object parts Expected scenarios (relations) Specializations Beliefs Partial matches Control Information Representations of n n n Partial interpretations Competing interpretations Relationship to the image descriptions

Introduction to Next Computer Vision Anyone who isn't confused really doesn't understand the situation. --Edward R. Murrow Next: Topic 3: Image Formation Reading: Ch 1, Ch 2 - Section 2. 1, 2. 2, 2. 3, 2. 5 Questions: 2. 1. 2. 2, 2. 3, 2. 5 Exercises: 2. 1, 2. 3, 2. 4