CS 598 KN Advanced Multimedia Systems Lecture 1
- Slides: 23
CS 598 KN Advanced Multimedia Systems Lecture 1 – Vision System Klara Nahrstedt Fall 2018 CS 598 kn - Fall 2018
Covered Aspects of Multimedia Image/Video Capture Audio/Video Perception/ Playback Audio/Video Presentation Playback Image/Video Information Representation Transmission Compression Processing Audio Capture Audio Information Representation CS 598 kn - Fall 2018 Media Server Storage A/V Playback
Today we start with Review of Image-based Visual System HUMAN VISUAL SYSTEM AND IMAGE DIGITIZATION CS 598 kn - Fall 2018
Human Visual System n Eyes, optic nerve, parts of the brain n Transforms electromagnetic energy
Human Visual System n Image Formation ¨ cornea, sclera, pupil, iris, lens, retina, fovea n Transduction ¨ retina, rods, and cones ¨ Retina has photosensitive receptors at back of eye n Processing ¨ optic nerve, brain
Rods vs Cones (Responsible for us seeing brightness and color) Cones (color) n n n Rods (brightness) Contain photo-pigment Respond to high energy Enhance perception Concentrated in fovea, exist sparsely in retina Three types, sensitive to different wavelengths n n n Contain photo-pigment Respond to low energy Enhance sensitivity Concentrated in retina, but outside of fovea One type, sensitive to grayscale changes CS 598 kn - Fall 2018
Tri-stimulus Theory n 3 types of cones (6/7 Mil. of them) Red = L cones, Green = M cones, Blue = S cones ¨ Ratio differentiates for each person ¨ E. g. , Red (64%), Green (32%), rest S cones ¨ E. g. , L(50. 6%), M(44. 2%), rest S cones ¨ n n Each type most responsive to a narrow band electro-magnetic waves Light stimulates each set of cones differently, and the ratios produce sensation of color CS 598 kn - Fall 2018
Color and Visual System n Color refers to how we perceive a narrow band of electromagnetic energy ¨ source, object, observer n Visual system transforms light energy into sensory experience of sight
Color Perception (Color Theory) n Hue Scale ¨ Refers to pure colors ¨ dominant wavelength of the light Saturation ¨ Perceived n Brightness (lightness) Source: Wikipedia lightness ¨ perceived Saturation intensity of a specific color ¨ how far color is from a gray of equal intensity Original n intensity CS 598 kn - Fall 2018
Digitalization of Images – Capturing and Processing CS 598 kn - Fall 2018
Image Concepts - Sampling An image is a function of intensity values over a 2 D plane I(r, s) n Sample function at discrete intervals to represent an image in digital form n ¨ matrix of intensity values for each color plane ¨ intensity typically represented with 8 bits n Sample points are called pixels CS 598 kn - Fall 2018 pixel
Digital Image Sampling Sample = pixel n Image Size (in pixels) n Image Size = Height x Width (in pixels) n 320 x 240 pixels n 640 x 480 pixels n 1920 x 1080 pixels - HD n 3840 pixels x 2160 lines (8. 3 megapixel) – 4 K UHD n 7680 pixels x 4320 lines (33. 2 megapixel) – 8 K UHD n CS 598 kn - Fall 2018
Digital Images - Quantization = number of bits per pixel n Example: if we would sample and quantize standard TV picture (525 lines) by using VGA (Video Graphics Array), n ¨ video controller creates matrix 640 x 480 pixels, and ¨ each pixel is represented by 8 bit integer (256 discrete gray levels) CS 598 kn - Fall 2018
Image Representations n Black and white image ¨ single bits n Grey scale image ¨ single bits n color plane with 2 color plane with 8 Color image ¨ three color planes each with 8 bits ¨ RGB, CMY, YIQ, etc. n Indexed color image ¨ single plane that indexes a color table n Compressed images ¨ TIFF, JPEG, BMP, etc. 4 gray levels 2 gray levels
Digital Image Representation (3 Bit Quantization) CS 598 kn - Fall 2018
Color Quantization Example of 24 bit RGB Image 24 -bit Color Monitor CS 598 kn - Fall 2018
Image Representation Example 24 bit RGB Representation (uncompressed) 128 135 166 138 190 132 129 255 105 189 167 190 229 213 134 111 138 187 135 255 213 190 167 138 129 229 138 189 111 Color Planes 166 105 134 132 190 187
Image Properties (Color) CS 598 kn - Fall 2018
Color Histogram CS 598 kn - Fall 2018
Spatial and Frequency Domains n Spatial domain ¨ refers to planar region of intensity values at time t n Frequency domain ¨ think of each color plane as a sinusoidal function of changing intensity values ¨ refers to organizing pixels according to their changing intensity (frequency) CS 598 kn - Fall 2018
After this lecture… n n n Browse the class website and piazza Check-in to compass 2 g Start considering which papers would you like to present Start considering which reviews would you like to post Start thinking about your project Send me your paper and review selections by September 1 CS 598 kn - Fall 2018
ADDITIONAL INFORMATION CS 598 kn - Fall 2018
n n n n n Where’s the Action in Multimedia? New interfaces for organizing, retrieving, and accessing large collections of content (Apple - IPhone, Microsoft – Kinect/Hololens) Capturing and sharing experiences (Facebook, Twitter, Google) Multi-party collaborative systems (Cisco Tele-presence, Skype, Web. Ex, …) Multi-camera: omnidirectional cameras, distributed Video 360 (Netflix, Facebook, Microsoft) 3 D media – 3 D compression, 3 D movies (Microsoft) P 2 P Streaming and CDN Distribution (Netflix, Amazon, Comcast, ESPN, Apple, …) Mobile multimedia (Apple, Samsung, Google, ) Media servers – You. Tube, Amazon VR, AR, HMD, Quality of Experience – all companies
- Cse 598 advanced software analysis and design
- Cs598
- Round the following numbers to the nearest hundreds 625
- Round 673 422 to the nearest hundred
- Evaluate the postfix expression 6523+8*+3+*
- Cse 598
- 598 kpc
- Cse 598
- Pipeline adc ppt
- Ixp 598
- 01:640:244 lecture notes - lecture 15: plat, idah, farad
- Advanced inorganic chemistry lecture notes
- Multimedia becomes interactive multimedia when
- Linear multimedia definition
- Csc 253
- Esa multimedia.esa.int./multimedia/virtual-tour-iss
- Operating systems lecture notes
- Articulators
- Lecture sound systems
- Advanced fusion systems llc
- Advanced fueling systems
- Marine fluid systems
- Tom kilcer advanced ag systems
- Ist 421