Color Image Fidelity Assessor Wencheng Wu Xerox Corporation

  • Slides: 26
Download presentation
Color Image Fidelity Assessor * Wencheng Wu (Xerox Corporation) Zygmunt Pizlo (Purdue University) Jan

Color Image Fidelity Assessor * Wencheng Wu (Xerox Corporation) Zygmunt Pizlo (Purdue University) Jan P. Allebach (Purdue University) * Research supported by HP Company while Wencheng Wu was at Purdue University Page 1

Outline • • • Introduction Spatial color descriptor: chromatic difference Structure of Color Image

Outline • • • Introduction Spatial color descriptor: chromatic difference Structure of Color Image Fidelity Assessor (CIFA) Psychophysical experiment and its results Test examples Conclusion Purdue University Page 2

Introduction (Motivation) • Image fidelity assessment is important in the development of imaging systems

Introduction (Motivation) • Image fidelity assessment is important in the development of imaging systems and image processing algorithms w Create visually lossless reproduction w Allocate efforts on most visible area • Subjective evaluation is expensive and slow. Purdue University Page 3

Introduction (Prior work) • Simple but not working w Root-Mean-Square Error • Consider structure

Introduction (Prior work) • Simple but not working w Root-Mean-Square Error • Consider structure of HVS and perceptual process w Achromatic: Daly’s VDP, Lubin’s VDM, Taylor’s Achromatic IFA (IFA) w Color: Jin’s CVDM (Daly’s VDP + Wandell’s Spatial CIE Lab) Purdue University Page 4

Introduction (CVDM vs. CIFA) • • Both operate along opponent-color coordinates • They differ

Introduction (CVDM vs. CIFA) • • Both operate along opponent-color coordinates • They differ in a similar way as VDP vs. IFA Both incorporate results from electrophysiological and psychophysical exp. w CIFA has closer link between the structure of the model and the psychophysical data used by the model • CIFA normalize the chromatic responses w This discounts luminance effect in chromatic channels w This reduces the dimension of psychometric LUT Purdue University Page 5

Introduction (Overview of CIFA) • • Color extension of Taylor’s achromatic IFA The model

Introduction (Overview of CIFA) • • Color extension of Taylor’s achromatic IFA The model predicts perceived image fidelity w Assesses visible differences in the opponent channels w Explains the nature of visible difference (luminance change vs. color shift) Ideal Rendered Color Image Fidelity Assessor (CIFA) Image maps of predicted visible differences Viewing parameters Purdue University Page 6

Chromatic difference (Definition) • • Objective: evaluate the spatial interaction between colors First transform

Chromatic difference (Definition) • • Objective: evaluate the spatial interaction between colors First transform CIE XYZ to opponent color space (O 2, O 3) * Luminance Red-Green Blue-Yellow • Then normalize to obtain opponent chromaticities (o 2, o 3) • Define chromatic difference (analogous to luminance contrast c 1) * X. Zhang and B. A. Wandell, “A SPATIAL EXTENSION OF CIELAB FOR DIGITAL COLOR IMAGE REPRODUCTION”, SID-97 Purdue University Page 7

Opponent color representation (Y, o 2, o 3) (Y, 0. 24, 0. 17) (13.

Opponent color representation (Y, o 2, o 3) (Y, 0. 24, 0. 17) (13. 3, o 2, 0. 17) (13. 3, 0. 24, o 3) Purdue University Page 8

Chromatic difference (illustration) 0. 1 • • • 0. 05 0. 2 0. 1

Chromatic difference (illustration) 0. 1 • • • 0. 05 0. 2 0. 1 Chromatic difference is the amplitude of the sinusoidal grating Chromatic difference is a measure of chromaticity variation Chromatic difference is a spatial feature derived from opponent chromaticity that has little dependence upon luminance Purdue University Page 9

CIFA Ideal Y Image Rendered Y Image map of predicted visible luminance differences Achromatic*

CIFA Ideal Y Image Rendered Y Image map of predicted visible luminance differences Achromatic* IFA Multi-resolution Y images Ideal O 2 Image Rendered O 2 Image Ideal O 3 Image Rendered O 3 Image Red-green IFA Image map of predicted visible red-green differences Blue-yellow IFA Image map of predicted visible blue-yellow differences Chromatic IFAs (Y, O 2, O 3): Opponent representation of an image * Previous work of Taylor et al Purdue University Page 10

Lum. contrast Chromatic diff. discrimination Red-green IFA Achromatic IFA Psychometric LUT (f, o (f,

Lum. contrast Chromatic diff. discrimination Red-green IFA Achromatic IFA Psychometric LUT (f, o (f, Y, c LUT 2, c 21)) Adaptation level Lowpass Pyramid Psychometric Selector Contrast. Diff. Chromatic Decomposition + S – Lowpass Pyramid Contrast. Diff. Chromatic Decomposition Channel Response Predictor Limited Memory Prob. Sum. Contrast: luminance contrast & chromatic difference Purdue University Page 11

*C. Taylor, Z. Pizlo, and J. P. Allebach, IS&T PICS, May. 1998 * Purdue

*C. Taylor, Z. Pizlo, and J. P. Allebach, IS&T PICS, May. 1998 * Purdue University Page 12

IFA components • Psychometric LUT w Results from psychophysical experiment w Stored in the

IFA components • Psychometric LUT w Results from psychophysical experiment w Stored in the form of Lookup-Table: (f, Y, c 1), (f, o 2, c 1), (f, o 3, c 1) w Time consuming, but it is done off-line • Image processing: w Lowpass pyramid: create 5 multi-resolution images » Lowpass filtering, then down-sample by 2 in horizontal and vertical directions » Normalized by Y images if it is a chromatic IFA w Signal decomposition: create 8 orientation-specific contrast or chromaticdifference images at each resolution w Lowpass pyramid + Signal decomposition: 40 (5 levels, 8 orientations) visual channels for each image pixel Purdue University Page 13

IFA components (cont’d) • Image processing (continued): w Psychometric selector: for each pixel at

IFA components (cont’d) • Image processing (continued): w Psychometric selector: for each pixel at each visual channel, find discrimination threshold by choosing appropriate data from LUT w Channel response predictor: for each pixel at each visual channel, convert chromatic difference to discrimination probability w Limited memory probability summation: for each pixel, combine discrimination probability across all 40 visual channel Purdue University Page 14

Estimating parameters of LUT (Stimulus: Isoluminant Gabor patch) • Red-green (O 2 or o

Estimating parameters of LUT (Stimulus: Isoluminant Gabor patch) • Red-green (O 2 or o 2) stimulus w Keep Y, O 3 (o 3) constant w Let O 2=Yo 2+Yc 2 cos(. )e(. ) or equivalently o 2’ =o 2+c 2 cos(. )e(. ) • (Y, o 2, o 3) specifies the background color, c 2 is the chromatic difference Gabor patch f, o 2, c 2 Purdue University Page 15

Estimating parameters of LUT (Psychophysical method) • Red-green stimulus: (Y, o 2, o 3)

Estimating parameters of LUT (Psychophysical method) • Red-green stimulus: (Y, o 2, o 3) specifies the background color, c 2 is the ref. chromatic difference • Which stimulus has less chromatic difference? This is the probability that subject says that the left side has lower contrast. Purdue University Page 16

Estimating parameters of LUT (Data analysis) Fit subject’s responses to a Normal distribution using

Estimating parameters of LUT (Data analysis) Fit subject’s responses to a Normal distribution using probit analysis Record the standard deviation as the discrimination threshold LUT: rg(f, o 2, c 2) Subject WW’s responses probability • • • This is the probability that subject says that the left side has lower contrast. Purdue University Page 17

Estimating parameters of LUT (List of experimental conditions) = indicate spatial frequency of 1,

Estimating parameters of LUT (List of experimental conditions) = indicate spatial frequency of 1, 2, 4, 8, 16 cpd Purdue University Page 18

Representative results Red-green discrimination at RG 1: (Y, o 2, o 3)=(5, 0. 2,

Representative results Red-green discrimination at RG 1: (Y, o 2, o 3)=(5, 0. 2, -0. 3) Blue-yellow discrimination at BY 1: (Y, o 2, o 3)=(5, 0. 3, 0. 2) Threshold Reference c 2 • • • Reference c 3 Results for f = 16, 8, 4, 2, 1 cycle/deg are drawn in red, green, blue, yellow, and black. Threshold is not affected strongly by the reference chromatic difference Chromatic channels function like low-pass filters Purdue University Page 19

CIFA output for example distortions (Hue change) Luminance R-G B-Y Purdue University Page 20

CIFA output for example distortions (Hue change) Luminance R-G B-Y Purdue University Page 20

CIFA output for example distortions (Blurring) Luminance R-G B-Y Purdue University Page 21

CIFA output for example distortions (Blurring) Luminance R-G B-Y Purdue University Page 21

CIFA output for example distortions (Limited gamut) Luminance R-G B-Y Purdue University Page 22

CIFA output for example distortions (Limited gamut) Luminance R-G B-Y Purdue University Page 22

Conclusion • CIFA provides good assessment of the perceived visible differences over a range

Conclusion • CIFA provides good assessment of the perceived visible differences over a range of image contents and distortion types • Chromatic difference describes the color percept of HVS efficiently • Suggestions on future directions w Add DC component in the LUT in chromatic IFAs w Subjective validation w Improve spatial localization w Take dependency between visual channels into account (in prob. Sum. stage) Purdue University Page 23

CIFA output for example distortions (Limited color quantization) Luminance R-G B-Y Purdue University Page

CIFA output for example distortions (Limited color quantization) Luminance R-G B-Y Purdue University Page 24

CIFA output for example distortions (Limited gamut) Luminance R-G B-Y Purdue University Page 25

CIFA output for example distortions (Limited gamut) Luminance R-G B-Y Purdue University Page 25

CIFA output for example distortions (Increased saturation) Luminance R-G B-Y Purdue University Page 26

CIFA output for example distortions (Increased saturation) Luminance R-G B-Y Purdue University Page 26