Image Repairing Robust Image Synthesis by Adaptive ND
Image Repairing: Robust Image Synthesis by Adaptive ND Tensor Voting IEEE Computer Society Conference on Computer Vision and Pattern Recognition Jiaya Jia, Chi-Keung Tang Computer Science Department The Hong Kong University of Science and Technology
Motivation • Main difficulties to repair a severely damaged image of natural scene – Mixture of texture and colors – Inhomogeneity of patterns – Regular object shapes
Motivation • Given as few as one image without additional knowledge, we address: – How much color and shape information in the existing part is needed to seamlessly fill the hole? – How good can we achieve in order to reduce possible visual artifact when the information available is not sufficient. • Robust Tensor Voting method is adopted
Tensor Voting Review • Tensors: compact representation of information • Tensor encoding: Ball tensor: uncertainty in all directions Stick tensor: certainty along two opposite directions 3 D tensor Plate tensor: certainty of directions in a plate
Tensor Voting Review • Voting process is to propagate local information Osculating circle P
Image repairing system Complete Segmentation Input Damaged Image Texture-based Segmentation Curve Connection Statistical Region Merging Adaptive Scale Selection ND Tensor Voting Image synthesis Output Repaired Image
Segmentation • JSEG [Deng and Manjunath 2001] – color quantization – spatial segmentation • Mean shift [Comanicu and Meer 2002] • Deterministic Annealing Framework [Hofmann et al 1998]
Texture-based Segmentation
Statistical Region Merge • (M + 1)D intensity vector for each region Pi, where M is the maximum color depth in the whole image. if histogram gradient
Why Region Merge? • Decrease the complexity of region topology • Relate separate regions P 1 P 5 P 2 Damaged area P 3 P 4
Curve Connection • 2 D tensor voting method Z P 1 P 5 P 2 P 3 P 4 P 2 P 4 X
Why Tensor Voting? • The parameter of the voting field can be used to control the smoothness of the resulting curve. • Adaptive to various hole shapes t rain t t s n e o l c a tlhl o. Slce u a o m h t S i W Wi. Lthah roglee e Scocnasltera int
Connection Sequence • Topology of surrounding area of the hole can be very complex • Greedy algorithm – Always connect the most similar regions P 1 P 2 and P 4 P 3 and P 5 P 2 Damaged area P 1 P 3 P 4
Complete Segmentation
Image repairing system Complete Segmentation Input Damaged Image Texture-based Segmentation Curve Connection Statistical Region Merging Adaptive Scale Selection ND Tensor Voting Image synthesis Output Repaired Image
ND Tensor Voting • Tensor encoding – Each pixel is encoded as a ND stick tensor 5 5 Scale N=26 Stick tensor
ND Tensor Voting • Voting process in ND space – An osculating circle becomes an osculating hypersphere. – ND stick voting field is uniform sampling of normal directions in the ND space. sample
Adaptive Scaling • texture inhomogeneity in images gives difficulty to assign only one global scale N [Lindeberg et al 1996]. • For each pixel i in images, we calculate: • trace(M) measures the average strength of the square of the gradient magnitude in the window of size Ni
Adaptive Scaling • For each sample seed: – Increase its scale Ni from the lower bound to the upper bound – If trace( ) < trace( ) - α where α is a threshold to avoid small perturbation or noise interference, set Ni - 1 → Ni and return – Otherwise, continue the loop until maxima or upper bound is reached
Results
Results
Results
Results
Results
Limitations • Lack of samples. • Meaningful and semiregular objects.
Conclusion • • • An automatic image repairing system. Region partition and merging. Curve connection by 2 D tensor voting. ND tensor voting based image synthesis. Adaptive scale.
- Slides: 26