Mathematical Morphology in Electronic Printing Robert Loce Xerox
Mathematical Morphology in Electronic Printing Robert Loce Xerox Corp. ISMM’ 07 10/11/07
Outline § Electronic printing performs hardcopy representation of several types of image data under various sampling conditions, user preferences and marking processes. § Morphological operations are used to better enable this representation Key operations are forms of dilation, erosion, hit-and-miss transforms, and tophat transforms § We will examine morphological methods used for the following application categories: Line width modification Corner/Serif Sharpening Resampling: Resolution Conversion and Enhancement Halftone structure Color object adjacency: Trapping § We first discuss a few relevant sampling and printing conditions 2/28/2021 2
Real-Time Setting § Processing is often performed at high speed to maintain printer productivity. Ex: 11”x 17” page, 100 pages/min, @ 600 dpi = 1. 1 MPix/sec § Most algorithms are single pass. § High speed scan line memory is expensive, so observation windows are often asymmetrical. (x, y) window requires y 1 scanlines plus x pixels Scanline is typically several thousand pixels in length 2/28/2021 3
Simple Versatile Implementations Consider a Hit-and-Miss Transform Y(A) = i A Ä (Ei, Fi) Geometrically, we view (Ei, Fi) as a template with Ei and Fi, defining 1’s and 0’s respectively. Remainder of pixels in the observation window are “don’t cares. ” Truth tables require more memory, but are easier to modify § Download new y values as requirements changes Example: E 2 E 1 1 1 F 2 0 0 1 1 Logical sum of products: efficient to implement in hardware logic gates, but is difficult to modify y = x 1 x 2 x 3 c + x 1 cx 2 x 3 2/28/2021 4
Imaging Devices Enable Particular Pixel Types Laser Scanner Image Bar Selfoc Lens LED Array Photoreceptor Process Direction 2/28/2021 5
Pixel types PWPM High Addressability Usually converted from isomorphic pixels just before scanning Anti-Aliased Typical source is super resolution Can be written with varying laser power or can be converted to other pixel types 2/28/2021 6
Binary Fine Line Rendition Via HMT Thin black lines might not print. Thin white lines can fill in. These lines require growth in the bit map. § Template Matching: HMT used since 1980’s to identify pixels that neighbor thin binary lines. Neighboring pixel values are modified to increase laser power or introduce additional laser pulses. • Binary input, output is higher quantization or higher addressability § Difficulties: HVS has great sensitivity to shape and overall appearance of text. Minor mistakes can be visible. Many structuring elements needed to recognize desired lines at all angles, tapers, and within complicated surrounds. Images taken from Crawford, J. L. , and C. D. Elzinga, “Improved Output Quality by Modulating Recording Power, ” SPSE 41 st Annual Conference, May 22 -26, 1988, Arlington, Virginia. 2/28/2021 7
Binary Fine Line Rendition Via Top Hat § Designing numerous templates is very time consuming A change in requirements (need to identify different lines) necessitates design of new template sets. Training methods have difficulties in creating suitable training images due to font kerning and stressed fonts. § Top hat methods allowed rapid versatile design L = A – (A o E) From image A create fine line image L B = (L F) – L) Element F creates image of fine line borders B'' = B' ∧ T Control signal T inserts pulses or gray levels into scaled border pixels (T – fine periodic lines) A'' = A' ∨ B'' Modified border is inserted into scaled (quantization, addressability) input image § Adjusting E, F, and T is much easier than designing hundreds of templates. § Multiple different line widths can be selected by multiple applications of the first two steps. Different treatments could be applied to those lines via different T. See US Patent 6, 181, 438 2/28/2021 8
Binary Digital Darkness Control with Fine Feature Protection via Top Hat § Often a need to modify darkness of text. Marking Process Compensation Appearance Matching Appearance Tuning § Darkening/lightening via simple dilation/erosion can ruin fine features Use Erosion/Dilation but protect fine features Input Image A § These isolated features can also be dilated or eroded to preferred widths § Additionally, a series of openings could isolate multiple specific widths and orientations (like a granulometry) and grow each by a preferred amount. Isolate Fine Features AF = A (A o F) Lighten Image AE = A ⊖ E Combine A’ = AF AE Output Image A See US Patent 6, 728, 004 2/28/2021 9
HMT on Halfbits § “Halfbits” (AKA Edge Ticks) are used on fonts to approximate ½ binary pixel. § Marking process blur hides most of the ragged structure § HMT have been used for digital darkness control by filling-in (removing) half bits or adding (subtracting) them to edges. Halfbits added. HMT=1 on checkerboard locations when halfbits not found on edge Darker See US Patent 5, 483, 351 Halfbits filled in filed in by HMT that recognizes halfbits § HMT have also been for replacing halfbits with high addressable pixels to avoid the ragged appearance. See US Patent 5, 539, 866 2/28/2021 10
Challenge Line Width Control on Anti. Aliased Lines § Line art processing typically employs template matching (Boolean logic) on binary pixels. Conventional Line Art Processing § Anti-aliased line art has gray edges Typically starts at high resolution, averaged/subsampled to lower resolution. Conventional Boolean logic cannot identify and modify these line art edges. Example of logic to find inside borders 2400 dpi at the rasterizer 600 dpi sent to printer Anti-alias 600 dpi grid positions 2/28/2021 11
Line Width Control on Anti. Aliased Lines “Loose Gray-Scale Template Matching” (LGSTM) A change may be made to a pixel value based on “loose fit” or “non loose fit” of a gray-scale template to the observed pixels. x 1 → x 1 t 1 �< 1 → x 2 → x 2 t 2 �< 2 → Combine § Fit Decision xn → xn tn �< n → Before After § Considerable art required in designing templates for a given application. § Looseness can be implemented by varying bit depth. See US Patent 6, 944, 341 See US Patent 6, 816, 621 for an efficient implementation 2/28/2021 12
Corner Sharpening via HMT Marking processes can round off corners and shorten serifs. Compensation structures can be formed in the bitmap prior to printing. Compensate Bitmap “Ink Trap” See US Patent 6, 775, 410, Patent app. 20050129328 2/28/2021 13
Resolution Enhancement § Mid 1980’s through the 90’s had great activity in resolution conversion and enhancement. § Many resolution and sampling conditions existed Scanning Gray scale 200, 300, 400, 600 dpi … gray scale. Often anamorphic, e. g. 400 x 600, due to sensor array resolution and stepper resolution Office Printers Binary 240, 300, 400, 600 spi Display Devices Gray scale 75 spi Fax Binary 204 x 98, 204 x 196 § Today, resolutions are high relative to HVS and complicated resolution conversion/enhancement is less important. 2/28/2021 14
Resolution Enhancement Example Walsh, a sociologist on contract to Xerox, described many fundamental issues. For example § Line buffering for sequential raster device. § Window location with respect to output signal § Look-up table structure for signal conversion § Timing controllers to handle input/output data rates § Template symmetry § Don’t-care pixels From Walsh, Halpert working with display resolution (100 dpi) and printer resolution (300 dpi) See US Patent 4, 437, 122, Mar. 13, 1984. 2/28/2021 15
Modification of Omega Channel via HMT on Image Channel, Ex. Halftone Edges § Digital printer image paths often utilize plane of metadata that corresponds to plane of pixel values. One use is to direct halftoning or image rendering from contone to binary values. Based on HMT on image, modify values in omega channel to specify high frequency dots or lines § HMT can be applied to thresholded gray image to identify gray edges. Values in the omega channel are modified for special halftoning that avoids edge serration. See US Patent 6, 243, 499, US Patent app. 20050129328 2/28/2021 16
Small Text Protection against Halftones via Tophat Halftone damaged text (106 cpi) Missing serifs, ragged edges Small light text (9 pt) HT AT = T[A] Threshold AF = AT (AT o F) Tophat to identify fine features AC = C[AF] Colorize fine features to selected dark level A ' = AC ∨ A Max with input image Minimal damage Darkened small text HT See US Patent 6, 275, 304 2/28/2021 17
Trapping § Definition § Need for trapping § Basic trapping operations and terminology § Special considerations 2/28/2021 18
Definition Top View Intentional overlap of colorants to avoid appearance of paper at their interface § Conditional gray-scale dilations and erosions § Types of conditioning Gray level of dilated image plane Gray level of one or more other image planes (e. g. , CMYKP…) Proximity of other image planes Size of objects in other image planes Object types (defined by yet another plane of meta image data) Millions of images are trapped and printed every day. § Common and valuable image processing method possessing many interesting aspects of morphological operations § Seldom or never described with a morphological formalism § This community could provide that formalism! 2/28/2021 Paper Traps (dilated edges) Gray value and width depend upon - Values in all planes - Object type - Size of objects - Corners - Proximity to other traps 19
Why Trap? HVS is very sensitive to white borders between darker objects § White gaps as small as 15 µm may be objectionable Primary cause of white gap is misregistration Misregistration Sources § PR movement § Multiple laser scanners with differential bow, skew, nonlinearity, . . § Timing: jitter, page sync Trapping introduces an image artifact to avoid a more objectionable artifact § Darker overlap of multiple colorants not as objectionable as white gap 2/28/2021 20
Confusing Name Other uses of the term “trapping” in printing industry § Dry trapping: Adhesion of a coat of ink onto previously printed dry ink or paper § Wet trapping: Adhesion and contamination of a coat of ink onto a previously printed wet ink § Ink trap: Notched out inside corner of text or graphic master (e. g. , bit map) that produces a sharper edge upon printing Other names for trapping § Making grips, Lap register, Frame 2/28/2021 21
Approximate Trap Dimensions Desirable to trap as little as possible since trapping introduces an artifact into image. Commercial software often defaults to around 0. 25 pt (88 microns - two 600 dpi pixels or eight 2400 spi pixels) 2/28/2021 22
Basic Analog Method The combination of spacers to provide diffusion and overexposure of selected separations allowed for edges to be moved with some degree of control. Light Source Diffuser Glass Cover Original Spacer Trapped Duplicate Image taken from B. Lawler, Complete guide to Trapping, Hyden Books 1995 2/28/2021 23
Traditional Basic Operations Spread § Graphic element is increased in size to extend into a surrounding neighboring edge possessing a different colorant Choke § Graphic element is reduced in size to extend into a surrounded neighboring edge possessing a different colorant Spread/Choke not so important for digital methods Spread/Choke terminology from analog methods § Spread achieved by overexposure of foreground objects § Choke ″ ″ background objects 2/28/2021 24
Choke and Spread The concepts of “Choke” and “Spread” are rooted in old analog methods. The modern equivalents move edge positions and tend to have morphological characteristics Choke Spread 2/28/2021 25
Basic operations Knockout/Overprint These operations are usually performed while the image is being rasterized and are not as morphological in nature. Knockout § Deletion of color/image beneath a graphical element § Important for nonblack foreground and background objects Overprint § Opposite of knockout § Often used with black 2/28/2021 Knockout Replaces underlying colors Overprint Combines top color with underlying colors 26
Basic Operations - Overprinting with black can have a desirable appearance when the black object lies completely within the support of an object of another color plane. 2/28/2021 27
Basic Operations - Overprinting with black can produce an undesirable appearance when the black object lies only partly within the support of an object within another color plane. Overprint with black can result in undesirable two-tone effects Even black color is achieved when background is knocked out 2/28/2021 28
Trapping Lighter into Darker No Trap (Choke) Y Dilation C Dilation Image plane that is dilated depends upon co-located local lightness within all image planes. Extend lighter color into darker color § Less impact on local luminance § e. g. , Y into anything, anything into K § Luminance calculation must be performed for your colorants Must take Dmax into account Dmax C may or may not be darker than Dmax M § Edge may be between light (unsaturated) C and saturated M Weight of T is better maintained No Trap C Dilation Y Dilation (Spread) Trapping software has many color mixing and density rules that can be adjusted for your colorants Note again, “choke” and “spread” not relevant if you think if dilating Y into C. Image taken from http: //www. techcolor. com/help/trapping. html 2/28/2021 29
Centerline Trap Centerline luminance traps - extend both sides to maintain visual center of luminance. § More complicated than extending lighter into darker, but can yield better appearance, especially for small structures No Trap 2/28/2021 Trap 30
Sweeps: Preferable to use Centerline Large potential for discontinuous artifacts. “Sliding traps” (type of centerline trap) require a dilation with smooth varying conditioning on gray level. Sliding Trap Abrupt Trap Sliding Trap Images on left taken from http: //www. adobe. com/products/extreme/pdfs/trapping. pdf 2/28/2021 31
“Natural Trapping” - Common colors Edges sharing common colors do not need trapping § e. g. , red on yellow background has common color (Y) across objects. Typically common color should be at least 10% - 20% in value on its lighter side. Advantageous to convert custom colors to CMYK § Difficult: only about 35% of Pantone colors can be produced with ink CMYK. § Process color equivalents tend to require less trapping. § Varnish is also often trapped to avoid differential gloss. 2/28/2021 32
Type Difficulties § Trapping type changes its weight § We have significant memory for type shape – Low tolerance for edge artifacts Overprint small type § Problem: overprinting can produce unwanted colors § No trapping for small type, AKA “small text protection” Trap large type like any graphic object § Size: Some programs default at 10 - 24 point The resulting morphological edge movement operation is dependent upon a code (tag) for text in a corresponding image meta data plane. 2/28/2021 33
Type Examples Note that darker was trapping into lighter in this example to dramatize the effect on trapping text weight. 2/28/2021 34
Vignette trapping Trap a picture to its border CT to LW. § Often a switch to allow (1) no trap, (2) constant trap, (3) variable trap that conforms to the image. Photo placed in knockout Trap using background color Knockout for photo Image taken from B. Lawler, Complete guide to Trapping, Hyden Books 1995 2/28/2021 35
Vignette Trapping Must be subtle to avoid ragged boundaries and discontinuities Images taken from http: //www. adobe. com/products/extreme/pdfs/trapping. pdf 2/28/2021 36
Keepaways Conditional Erosions are used in Trapping § Erode lighter color Keep aways - reverse objects on multicolored background require protection Image taken from B. Lawler, Complete guide to Trapping, Hyden Books 1995 2/28/2021 37
Trap Shapes at Corners and Intersections Trap Intersection and End Styles No Trap Round Miter Bevel Desirable to avoid sharp corners Sophisticated programs create mitered shapes No Trap Miter Overlap Images taken from http: //www. adobe. com/products/extreme/pdfs/trapping. pdf 2/28/2021 38
Summary § Morphological type operations are in widespread use in the electronic printing industry, although morphological terminology is seldom used. § Many applications Line width modification Corner/Serif Sharpening Resampling: Resolution Conversion and Enhancement Halftone structure Color object adjacency: Trapping § The key morphological operations are forms of dilation, erosion, hit-and-miss transforms, and tophat transforms § Color trapping seems to be an area that could use a more formal mathematical structure 2/28/2021 39
- Slides: 39