ARTcode Preserve Art and Code In Any Image

ARTcode: Preserve Art and Code In Any Image Zhe Yang, Yuting Bao, Chuhao Luo, Xingya Zhao, Siyu Zhu, Chunyi Peng, Yunxin Liu, Xinbing Wang (Shanghai Jiao Tong University, The Ohio State University, Microsoft Research)

Outlines q. Introduction q. ARTcode Architecture q. How to Preserve Art in ARTcode q. Reliable ARTcode Detection and Decoding Ø ARTcode Detection Ø ARTcode Decoding q. Evaluation 2

Outlines q. Introduction q. ARTcode Architecture q. How to Preserve Art in ARTcode q. Reliable ARTcode Detection and Decoding Ø ARTcode Detection Ø ARTcode Decoding q. Evaluation 3

Introduction p In scenarios like electronic payment, state-of-the-art technologies apply barcodes such as QR code, shown on left. p What if we scan the picture of an article directly, as shown in the left image? 4

Introduction p These three barcodes encode the same information: URL to Wikipedia page of Lenna. Which one could help identify the contained information? 5

Introduction p The introduction of ARTcode has two advantages: u Traditional barcodes demand more space to display both human-readable and machine-recognizable information while ARTcode is a two-in-one visualization code. u Traditional barcodes impair user experience while ARTcode takes the form of input image. 6

Introduction p Goals of ARTcode: u Carry the human-friendly information (mostly, artistic images and layouts) and machine-friendly information (coded data bits) together in the same source over the visual channel. u Preserve pleasant userviewing experience (as if the device-oriented code would not exist). 7

Outlines q. Introduction q. ARTcode Architecture q. How to Preserve Art in ARTcode q. Reliable ARTcode Detection and Decoding Ø ARTcode Detection Ø ARTcode Decoding q. Evaluation 8

ARTcode Artchitecture 9

Outlines q. Introduction q. ARTcode Architecture q. How to Preserve Art in ARTcode q. Reliable ARTcode Detection and Decoding Ø ARTcode Detection Ø ARTcode Decoding q. Evaluation 10

ARTcode Encoding p To address this tension between both communications for humans and devices, we tackle the following three technical questions: u How to make changes to an image to largely keep its form and enable data embedment? Ø Colored dot matrix and Shuffling u How to reduce visual distortion in data embedment? Ø Embedding Data with Minor Modifications u How to manipulate colors to encode data? Ø Encoding Color to Bit 11

ARTcode Encoding p Colored dot matrix and Shuffling u ARTCODE uses a pointillism-like blending technique to generate a structure, maintaining original images’ general appearance, called colored dot matrix. 12

ARTcode Encoding u Colored dot matrix generation consists of two phases: • Color palette selection by clustering. We select the colors that represent a majority of the original image to form a color palette. • Revised error diffusion dithering. Dithering is a technique that converts complicated colored or gray-scale images to images with a palette of much fewer colors, retaining original images’ appearance. 13

ARTcode Encoding u Shuffling Algorithm finds embeddable modules: • Shuffling algorithm spreads encoding modules over the dot matrix according to a shuffling table. • The values of shuffling table are randomly generated. • Shuffling algorithm can achieve multiple message transmission. 14

ARTcode Encoding p Embedding Data with Minor Modifications. u In ARTcode, the embedment of machine-readable information in the dot matrix is regarded as noises for human perception. u In order to trade off the capacity and visual quality demands of ARTcode, we integrate a data hiding algorithm and adapt it to the present work. u The key idea is, in a -length vector, by changing only 1 bit, it can embed bits data, for there are in total positions in this vector, plus 1 original unchanged vector. 15

ARTcode Encoding p Encoding Color to Bit. u Two considerations: • To guarantee decoding accuracy, encoding colors should be “sufficiently distant” from each other. • Visual effect preservation demands that selected encoding colors take large proportion in embedding positions. 16

ARTcode Encoding p Encoding Color to Bit. u Four steps: Step 1. In color palette, calculate all n colors’ proportion they occupy in L encoding modules. Step 2. Find all feasible color sets. Colors in feasible sets are “sufficiently distant” from each other. Step 3. Calculate the sum of colors’ proportion in each feasible color set. Select the color set with the largest proportion. Step 4. If, by any chance, the algorithm fails to find a feasible color set, the algorithm decreases the threshold until feasible color sets are found. 17

Outlines q. Introduction q. ARTcode Architecture q. How to Preserve Art in ARTcode q. Reliable ARTcode Detection and Decoding Ø ARTcode Detection Ø ARTcode Decoding q. Evaluation 18

ARTcode Detection p Pre-processing u Local thresholding binarization u Erosion-dilation to eliminate moiré pattern 19

ARTcode Detection p Module Localization u As huge finder patterns affect visual, we do not insert locators in the code area and place an alignment pattern on borders of the barcode for module localization. 20

ARTcode Detection p Module Localization u Module localization detects the following entities in sequence: ARTcode corners ARTcode double deck alignment Each module in code area u The localization of code area modules considers distortion introduced by camera. 21

Outlines q. Introduction q. ARTcode Architecture q. How to Preserve Art in ARTcode q. Reliable ARTcode Detection and Decoding Ø ARTcode Detection Ø ARTcode Decoding q. Evaluation 22

ARTcode Decoding p Color recognition u Color recognition assigns nearest RGB values in encoding color set obtained by double deck alignment pattern. p Comprehensive Error Correction u Both Reed Solomon error correction and Cyclic Redundancy Code error detection. u If CRC algorithm detects an error, another frame will be sent in for detection and decoding. 23

Outlines q. Introduction q. ARTcode Architecture q. How to Preserve Art in ARTcode q. Reliable ARTcode Detection and Decoding Ø ARTcode Detection Ø ARTcode Decoding q. Evaluation 24

Evaluation p Image set. u 197 images from Google Image, under keywords “Logo”, “Gray-scale”, “Portrait”, “Landscape”, “Animal”, and also classical images like “Lenna”. u 25 gray-scale images, 22 logos and 150 colored images. u 48 low contrast, 64 intermediate contrast and 38 high contrast images. 25

Evaluation p Default setting. u 50 bytes, 16 clustered colors, embedding block size 15. u Six examples: (a) Gray (b) Logo 1 (c) Logo 2 (d) Elephant (e) Butterfly (f) Lenna 26

Evaluation p Overall Performance u User study with 50 participants. • • Each participant gives a subjective distortion score δ from 0 to 10. 0 --- distortion not perceptible, 5 --- noticeable but acceptable, 10 --heavily corrupted. 27

Evaluation p Overall Performance. u Information-carrying accuracy. • • PC screen as the sender and each code is 8. 4 × 8. 4 Camera-screen distance 18. . 28

Evaluation p Assessment on Image Visual Quality. 29

Evaluation p Information-Carrying Capability. 30

Thanks！