Image Processing Models Inspired by Two Kinds of

Image Processing Models Inspired by Two Kinds of Double. Opponent Neurons in the Primary Visual Cortex Yong-Jie Li / 李永杰 School of Life Science and Technology Univ of Electronic Science & Tech of China liyj@uestc. edu. cn � 子科技大学生命科学与技� 学院 / 神� 信息教育部重点�� 室

Outline Introduction n Computational color constancy by double-opponent V 1 cells with concentric center-surround receptive fields n Contour detection by doubleopponent V 1 cells with oriented receptive fields n Summary & Discussion n

Color is an important visual feature …

（Conway, 2009）

Retina

Ganglion cells M cell P cell K cell

M cell P cell K cell

Color-opponent cells L+ M-

Single- & double-opponent cells Single-opponency doubleopponency RG, LGN, V 1

Single-opponent Single- & doubleopponent

Single-opponency double-

Luminance cells (60%) Color-luminance cells (29%) Color cells (11%)

Luminance cells color-luminance cells Color cells 29% (48/167) 11% (19/167) Double-opponent Singleopponent & Doubleopponent Band-pass Low-pass Achromatic boundaries Achromatic & chromatic boundaries Chromatic regions Proportio 60% (100/167) n Opponent Non-opponent Spatial tuning Respons e Orientation. Non-orientation selectivity -selectivity on. Johnson tuning(2001)selectivity Nature Neuroscience; Solomon (2007) Nature Neuroscience Location 4 B, 4 Cα, 5, 6 2/3, 5, 6

Luminance cells (60%) Non-opponent color-luminance cells(29%) Double-opponent Color cells(11%) Single-opponent Simple cell Complex cell few

We are concerned with the functional roles of the two types of doubleopponent V 1 cells

Outline Introduction n Computational color constancy by double-opponent V 1 cells with concentric center-surround receptive fields n Contour detection by doubleopponent V 1 cells with oriented receptive fields n Summary & Discussion n

Color constancy

Color constancy l l The effect that the perceived color of a surface remains constant despite changes in the intensity and spectral composition of the illumination. Color constancy has had a long history of analysis (Monge, 1789; Young, 1807; von Helmholtz, 1867; Hering, 1920; …)

V 4 ?

Single-opponent Input from LGN Double-opponent V 1

(ICCV, 2013(oral); IEEE Trans PAMI, 2015)

(Reinhard et al. 2001)

(Ebner, 2007)

Results on Gehler-Shi dataset (Lower angular error is better).

Median angular errors of various models on the Gehler-Shi dataset with 568 high dynamic range linear images, including a variety of indoor and outdoor scenes

Median angular errors of different algorithms on SFU lab dataset, which contains 321 available images of 31 different objects captured with calibrated camera under 11 different lights in laboratory.

Outline Introduction n Computational color constancy by double-opponent V 1 cells with concentric center-surround receptive fields n Contour detection by doubleopponent V 1 cells with oriented receptive fields n Summary & Discussion n

Oriented double-opponent cells in V 1 (equal cone inputs) (unequal cone inputs)

V 1 Double-Opponent (Output Layer) LGN Single-Opponent (Middle Layer) Retina Cone (Input Layer) CVPR, 2013; IEEE Trans IP, 2015

Input image R G B (Berkley Dataset)

R-G Channel B-Y Channel Luminance edge

Original image Ground-truth

Pb [3] Our model [3] Martin, et al. , IEEE Trans. on PAMI, vol. 26, pp. 530 -549, 2004.

(Computation time (s) per image averaged over 100 images)

Image Robert Shapley, Michael J. Hawken, Vision Research. 2011.

Outline Introduction n Computational color constancy by double-opponent V 1 cells with concentric center-surround receptive fields n Contour detection by double-opponent V 1 cells with oriented receptive fields n Summary & Discussion n

n V 1 cells with different receptive field (RF) properties have different functional roles: DO V 1 cells with concentric center-surround RFs can code the illuminant color, which is used by higher area (e. g. , V 4) to realize color constancy. DO V 1 cells with oriented RFs and unbalanced cone inputs can response to (detect) both the color- and luminance-defined contours.

Related papers [1] Kaifu Yang, Shaobing Gao, Chaoyi Li, Yongjie Li*, Efficient Color Boundary Detection with Coloropponent Mechanisms, CVPR, 2013, pp 2810 -2817. [2] Shaobing Gao, Kaifu Yang, Chaoyi Li, Yongjie Li*, A Color Constancy Model with Double-Opponency Mechanisms, ICCV, 2013, pp 929 -936 (oral paper). [3] Shaobing Gao, Kaifu Yang, Chaoyi Li, Yongjie Li*, Color Constancy Using Double-Opponency, IEEE Transactions on PAMI, 2015 (DOI: 10. 1109/TPAMI. 2015. 2396053) [4] Kai-Fu Yang, Shao-Bing Gao, Ce-Feng Guo, Chao-Yi Li, and Yong-Jie Li*, Boundary Detection Using Double -Opponency and Spatial Sparseness Constraint, IEEE Transactions on Image Processing, 2015, 24(8): 25652578.

谢谢 ! 敬请批评指正！ Code available at: http: //www. neuro. uestc. edu. cn/vccl/ liyj@uestc. edu. cn