1 3 Error Diffusion Basic Concepts Advanced Topics

1. 3 Error Diffusion – Basic Concepts Advanced Topics in Digital Halftoning – 17 -19 October 2016

Synopsis • • • Error diffusion architecture Error diffusion textures Edge enhancement effect of error diffusion Spectral analysis of error diffusion Variations on a theme Tone-dependent error diffusion Advanced Topics in Digital Halftoning – 17 -19 October 2016

Error Diffusion Architecture Advanced Topics in Digital Halftoning – 17 -19 October 2016

Description of the Algorithm Advanced Topics in Digital Halftoning – 17 -19 October 2016

2 -D Error Diffusion Weighting Filters Advanced Topics in Digital Halftoning – 17 -19 October 2016

1 -D Example Advanced Topics in Digital Halftoning – 17 -19 October 2016

2 -D Error Diffusion – Floyd-Steinberg Weights Texture Artifacts in Midtones Advanced Topics in Digital Halftoning – 17 -19 October 2016

2 -D Error Diffusion – Floyd-Steinberg Weights Texture Artifacts in Highlights Advanced Topics in Digital Halftoning – 17 -19 October 2016

Error Diffusion Characteristics • At each step, error diffusion preserves local average over part of image that has been binarized and part that is yet to be binarized • No fixed number of quantization levels • Requires more computation than screening • Excellent detail rendition (sharpens image) • Generally good texture with some exceptions: – Texture contouring – Worm-like patterns in highlights and shadows – Texture cliques in midtones – Texture used to render a given gray level may be context-dependent Advanced Topics in Digital Halftoning – 17 -19 October 2016

Two Views of Error Diffusion Advanced Topics in Digital Halftoning – 17 -19 October 2016

Fourier Analysis • Combining (1) and (3), • In Fourier domain, • Rearranging, we get where is a high-pass filter • This shows that the spectrum of the binary image consists of the spectrum of the continuous-tone image plus a high-pass filtered version of the spectrum of the quantization error. Advanced Topics in Digital Halftoning – 17 -19 October 2016

Knox’s Empirical Model for Quantization Error* • We cannot find an analytical expression for • Knox proposed the following model Correlation coefficient – Residual – (generally some image-dependence) ED Weights 1 -D c 0. 0 Floyd and Steinberg 0. 55 Jarvis, Judice, and Ninke 0. 80 *K. T. Knox, “Error Image in Error Diffusion, ” SPIE Vol. 1657 (1992) Advanced Topics in Digital Halftoning – 17 -19 October 2016

Application to Fourier Analysis • Combining (4) and (5), we get High-emphasis filter High-pass noise • The first term shows the origins of the sharpening effect that is characteristic of error diffusion. • To the extent that the residual is uncorrelated with the original image, the second term will look like blue noise. Advanced Topics in Digital Halftoning – 17 -19 October 2016

Knox’s Experimental Results Halftone Image Quantized Error Image The parameter in the upper right-hand corner is the correlation coefficient c 1 -D Advanced Topics in Digital Halftoning – 17 -19 October 2016 Floyd. Steinberg Jarvis, Judice, Ninke

Spectral Characteristics of Error Diffusion continuous tone Bayer dither Advanced Topics in Digital Halftoning – 17 -19 October 2016 clustered dot dither error diffusion

Synopsis • • • Error diffusion architecture Error diffusion textures Edge enhancement effect of error diffusion Spectral analysis of error diffusion Variations on a theme Tone-dependent error diffusion Advanced Topics in Digital Halftoning – 17 -19 October 2016

Variations on a Theme • Non-lexicographic scan raster – Serpentine raster – Peano scan – Block-based scans • Modifications to weights – Randomized weights – Adaptive weights – Tone-dependent weights • Threshold modulation – Control sharpening – Encourage clustering Advanced Topics in Digital Halftoning – 17 -19 October 2016 Serpentine Raster

Early Work • Knox (1993) developed a spectral analysis for serpentine raster, and showed that it eliminates the asymmetry in the error weighting frequency response that is due to the causal nature of the filter. • Ulichney (1986, 1987) experimented with many different combinations of scan rasters, weight sets, and randomization techniques, and concluded that a serpentine raster, Floyd-Steinberg coefficients, and 50% randomization yielded the best results. Advanced Topics in Digital Halftoning – 17 -19 October 2016

Threshold Modulation • Basic equation for threshold modulation • Image-independent threshold modulation where is a periodic screen function – encourages periodic clustered-dot behavior (Billotet-Hoffman and Bryngdahl, 1983) • Image-dependent threshold modulation where is a constant that controls degree of edge enhancement (Eschbach, 1991) Approximately cancels edge enhancement intrinsic to standard error diffusion Yields standard error diffusion Results in more edge enhancement than is provided by standard error diffusion Advanced Topics in Digital Halftoning – 17 -19 October 2016

Tone-Dependent Error Diffusion Introduction • Previous work related to TDED: – R. Eschbach, JEI 1993 – J. Shu, SID 1996 – V. Ostromoukhov, US Patent 1998 – V. Ostromoukhov, SIGGRAPH 2001 • Our approach: – The weights and thresholds are optimized based on a HVS model. – Use variable weight locations to reduce worm like artifacts. Advanced Topics in Digital Halftoning – 17 -19 October 2016

Conventional Error Diffusion f[m, n] + u[m, n] g[m, n] Q( • ) - wk, l Advanced Topics in Digital Halftoning – 17 -19 October 2016 + d[m, n]

Tone Dependent Error Diffusion [Li and Allebach, 2004] Weights and thresholds vary depending on the input f[m, n] + - u[m, n] g[m, n] Q( • ) wk, l(f[m, n]) + d[m, n] X Advanced Topics in Digital Halftoning – 17 -19 October 2016

Tone Dependent Error Diffusion Quantization process: p[m, n; 0. 5] Problem: How to optimize t. U, t. L and wk, l for each gray level. Advanced Topics in Digital Halftoning – 17 -19 October 2016 Midtone halftone pattern generated with direct binary search (DBS)

Optimization of TDED parameters DBS 2 |DFT| Constant Patch (absorptance a) + Normalized MSE TDED 2 |DFT| Update weights and thresholds • Cost function Han Cost Function (Seong-Wook Han Ph. D. Dissertation (2009) Advanced Topics in Digital Halftoning – 17 -19 October 2016 -

Optimal weights and thresholds X Advanced Topics in Digital Halftoning – 17 -19 October 2016

Floyd-Steinberg vs TDED Floyd-Steinberg Advanced Topics in Digital Halftoning – 17 -19 October 2016 TDED

TDED vs DBS TDED Advanced Topics in Digital Halftoning – 17 -19 October 2016 DBS

The First HP Ink. Jet printer to incorporate Tone. Dependent Error Diffusion – The Desk. Jet 970 • The algorithm was built into an ASIC that can be found in every HP Inkjet product sold since the mid-2000 timeframe, including HP’s large format products. HP Desk. Jet 970 CXI HP Design. Jet T 795 (List Price: $4929. 49) Advanced Topics in Digital Halftoning – 17 -19 October 2016

Efficient Implementation of Error Diffusion • Reduced Lookup-table error diffusion – Removal of DBS midtone pattern – direct binary flipping (DBF) – Interpolation of weights and thresholds • Memory-efficient/parallelizable approaches to error diffusion – Block-based error diffusion » Block-interlaced pinwheel error diffusion » Serial block-based error diffusion – Error-quantized error diffusion – Compressible error diffusion Advanced Topics in Digital Halftoning – 17 -19 October 2016

Prototypical System Architecture High Speed Memory Processor . . . High Speed Memory Processor Bus There may be only one of these I/O Advanced Topics in Digital Halftoning – 17 -19 October 2016 Low Speed Memory

Implementation Issues – Parallelizability and Memory Access • Conventional error diffusion is inherently a serial process. • Cannot process any given pixel until all pixels that feed error to it have already been processed. – This limits opportunities for parallelism. • Must have access to the entire previous/next line of continuous-tone error/image data. – This impacts memory and/or bandwidth requirements. • It is a problem when we want to go very cheap or very fast. Advanced Topics in Digital Halftoning – 17 -19 October 2016

Memory Requirements for Raster Implementation storage of errors from previous line and current pixel n-1 n storage of errors from previous line n+1 x m m+1 storage of errors for next line • For low cost or large format printers, on-chip memory is not sufficient to buffer error information for one entire image row – All errors have to be written out to off-chip memory Advanced Topics in Digital Halftoning – 17 -19 October 2016

Strategies for Artifact Reduction: Serpentine Raster Conventional Raster Advanced Topics in Digital Halftoning – 17 -19 October 2016 Serpentine Raster

Block Interlaced Pinwheel Error Diffusion (BIP-ED) with Serpentine Raster • Process outward spiral (black) blocks first. Then process inward spiral (green) blocks. Advanced Topics in Digital Halftoning – 17 -19 October 2016

Application of TDED to BIP-ED • • Adapt tone dependent error diffusion to pinwheel architecture. Use different TDED filters for different block types and different locations within blocks. Randomly initialize outward spiral ED to break up periodic patterns. Initialize inward spiral ED using errors diffused from neighboring outward spiral blocks to eliminate boundary artifacts. Advanced Topics in Digital Halftoning – 17 -19 October 2016

BIP-TDED Block size 8 x 8 Advanced Topics in Digital Halftoning – 17 -19 October 2016 Block size 32 x 32

BIP-TDED vs TDED Pinwheel Error Diffusion Block size 128 x 128 Advanced Topics in Digital Halftoning – 17 -19 October 2016 Conventional Tone Dependent Error Diffusion

Stages for Strip-based Processing 1 3 1 5 2 5 4 7 4 8 6 8 Outward spiral block Advanced Topics in Digital Halftoning – 17 -19 October 2016 Inward spiral block