A Very Low Cost Video Wavelet Codec Krasimir

Goals and Purpose • Research into compact representation and computation on continuous spaces •

Features • Very low cost real-time video encoding of NTSC composite signal from legacy

WZD Video Compression • Codec Thumbnail – 3 -D Wavelet Transform (Taubman) – Dyadic

Distortion (PSNR) 40 37 33 less memory less processing = lower cost PSNR= 20

Set of test sequences • Laserdisk sequences - bridge, fence, staircase, dancers • Cable

12/2/2020 Interval Research Corp copyright 1998 7

Generating Videos • Collect sequences on Betacam SP source • Digitize in 8 bits/color

Video of Component Compressor (basketball sequence) 12/2/2020 Interval Research Corp copyright 1998 9

Videos of Composite Interval Code Vs. MPEG 2 for 4 sequences and several bitrates

Drivers of Low Cost • Temporal transform rather than motion estimation • Simple wavelet

ASIC Macro Cost Projections • ASIC capabilities @ l = 0. 35 m (1997)

ASIC Area Requirements • 1 Mbit SRAM • 100 K GATES • Opportunities =

WZD Features • Traditional transform, quantize, entropy code method • Direct compression of composite

Alternatives require auxiliary RAM chips • Analog Devices ADV 601 requires auxiliary DSP •

2 -6 Wavelets • The basic wavelet used is the TS (2 -6) transform,

Edge Filters • Fields of 240 x 704 pixels are divided into 30 x

2 -6 Edge filters 12/2/2020 Interval Research Corp copyright 1998 18

4 Field G. O. P. - Temporal Processing • Each field is temporarily compressed

Component Codec • Input - targa or. bmp format • Use 2 -6 wavelets

Component Wavelet Pyramid 1: LLTTLLTL (apex), 2: LLTTLLTR, 3: LLTTLRT, 4: LLTTLLB, 5: LLTTLRB

Component Codec vs. ADV • • • 2 -6 Wavelet Pyramid Temporal Wavelet Quantize

Interval WZD vs. ADV 12/2/2020 Interval Research Corp copyright 1998 23

Quantization of the Wavelet Coefficients Contrast Sensitivity Curves for the Human Visual System source

Quantization • Quantization coefficients are powers of 2 – Implemented by right shifts •

WZD Composite Codec • Input - D 2 composite format (chroma 1 and chroma

Composite Pyramid 12/2/2020 Interval Research Corp copyright 1998 27

WZD Codec vs. MPEG 2 • • 2 -6 Wavelet Pyramid Temporal Wavelet Quantize

Interval WZD vs. MPEG 2 12/2/2020 Interval Research Corp copyright 1998 29

Conclusions • Temporary internal compression can reduce memory, bandwidth, and processing requirements • Short

Cost - Rate Bubbles $100 Encoder Hardware Cost MPEG-2 ADV 601 $25 IRC-WZD $5

Slides: 31

Download presentation

Goals and Purpose • Research into compact representation and computation on continuous spaces • 2 major tasks: - Video Compression Applications - Compression of Data defined on highdimensional manifolds (3 D and higher) 12/2/2020 Interval Research Corp copyright 1998 2

Features • Very low cost real-time video encoding of NTSC composite signal from legacy sources – figure of merit is ASIC silicon area required • NTSC quality video at moderate data rates – 5 - 11 Mbits/sec = 0. 5 - 1. 0 bits/pixel • Comparable quality to MPEG-2 02 -Dec-20 Interval Research Corp Copyright 1998 3

WZD Video Compression • Codec Thumbnail – 3 -D Wavelet Transform (Taubman) – Dyadic Quantization – Various Entropy codecs • Very low cost solution in ASIC – not about the best in R-D 12/2/2020 Interval Research Corp copyright 1998 4

Set of test sequences • Laserdisk sequences - bridge, fence, staircase, dancers • Cable TV sequences - talkshow • DSS sequences - basketball, news • Camera shoots - Wayne, camcorder footage • Games - Nintendo 12/2/2020 Interval Research Corp copyright 1998 6

Generating Videos • Collect sequences on Betacam SP source • Digitize in 8 bits/color ITU 601 format (done by Pacific Video Resources in SF) • Use Arriba Pro to read tapes (in Abekas format) and save in bitmap or targa format • Use Bob’s box to generate D 2 composite format • Use Sierra DDR to render videos 12/2/2020 Interval Research Corp copyright 1998 8

Drivers of Low Cost • Temporal transform rather than motion estimation • Simple wavelet transform - no multiplies – parallel multiplies use ASIC area • 2 nd order boundary filters facilitate block-by-block processing – saves RAM and RAM bandwidth • Field buffers are compressed and decompressed for temporal wavelets – saves the most RAM 12/2/2020 Interval Research Corp copyright 1998 11

ASIC Macro Cost Projections • ASIC capabilities @ l = 0. 35 m (1997) – DRAM: 300 K bits/mm 2 – SRAM: 60 -75 K bits/mm 2 – GATES: 12 K /mm 2 • 1998: l = 0. 25 m – parameters improve by (0. 35/0. 25) 2 = 1. 96 • COST: $0. 25 /mm 2 12/2/2020 Interval Research Corp copyright 1998 12

ASIC Area Requirements • 1 Mbit SRAM • 100 K GATES • Opportunities = 12 mm 2 = = 8 mm 2 = $3. 00 $2. 00 – 1 Mbit DRAM = 4 mm 2 = $1. 00 – l = 0. 25 m cuts area (and cost) in half • Risks – ADC and/or DACs not included in estimates 12/2/2020 Interval Research Corp copyright 1998 13

WZD Features • Traditional transform, quantize, entropy code method • Direct compression of composite NTSC – compression also decodes chroma from carrier • Modes for S-Video and component video • Low latency - less than 6 frame times • Cheap color conversion seems possible • Low power in ASIC - 28. 7 MHz clock 12/2/2020 Interval Research Corp copyright 1998 14

Alternatives require auxiliary RAM chips • Analog Devices ADV 601 requires auxiliary DSP • Other Wavelet require multiplies, usually motion image codec • Motion JPEG require multiplies, worse rate/distortion in range of interest • MPEG requires multiplies, encoder requires motion estimation, expensive encode for good rate/distortion 12/2/2020 Interval Research Corp copyright 1998 15

Edge Filters • Fields of 240 x 704 pixels are divided into 30 x 22 blocks of 8 x 32 pixels each • Edge filters fit a quadratic to each block boundary from both left and right - 98% fit – Therefore blocking artifacts are virtually eliminated • Memory bandwidth is greatly reduced • Data handling is vastly simplified • Only adds and shifts required 12/2/2020 Interval Research Corp copyright 1998 17

4 Field G. O. P. - Temporal Processing • Each field is temporarily compressed as a still image – each requires ~200 K bits ( = 1. 18 bits/px. ) • All 4 fields are partially decompressed in parallel lockstep – temporal linear filters are applied with serial-by -bit arithmetic – result is recompressed 12/2/2020 Interval Research Corp copyright 1998 19

Component Codec • Input - targa or. bmp format • Use 2 -6 wavelets for the transform part • Use zerotree structure for quantization and entropy coding • Do block-by-block processing • Perform intermediate compression in the time direction 12/2/2020 Interval Research Corp copyright 1998 20

Component Codec vs. ADV • • • 2 -6 Wavelet Pyramid Temporal Wavelet Quantize Zerotree Encode Entropy Decode Reverse Temporal Wavelet • 2 -6 Reverse Wavelet Pyramid 12/2/2020 • 7 -9 Wavelets • Quantize • Entropy Encode • Entropy Decode • Inverse 7 -9 Wavelets Interval Research Corp copyright 1998 22

Quantization • Quantization coefficients are powers of 2 – Implemented by right shifts • Quantization shifts align coefficient bits of equal importance in the same bit positions – Facilitates progressive coding by bitplanes of decreasing importance – e. g. SPIHT coding or arithmetic coding of bitplanes 12/2/2020 Interval Research Corp copyright 1998 25

WZD Composite Codec • Input - D 2 composite format (chroma 1 and chroma 2 quadrature modulated on a 3. 4 MHz sub-carrier, as in NTSC input) • 2 -6 wavelet pyramid for transform • Quantization step (account for HVS characteristics, less bits in color) • Entropy Coder (zerotree, arithmetic) 12/2/2020 Interval Research Corp copyright 1998 26

WZD Codec vs. MPEG 2 • • 2 -6 Wavelet Pyramid Temporal Wavelet Quantize (1. 0 bpp or 0. 5 bpp) Entropy Encode (zerotree, arithmetic) DCT (Discrete Cosine Transform) Motion Estimation Quantize Entropy Encode (Huffman) • Entropy Decode • Reverse Temporal Wavelet • 2 -6 Reverse Wavelet Pyramid 12/2/2020 • • • Entropy Decode • Motion Compensation • Inverse DCT Interval Research Corp copyright 1998 28

Conclusions • Temporary internal compression can reduce memory, bandwidth, and processing requirements • Short Temporal Filters can compete with motion estimation • Very low cost symmetric codecs are possible 12/2/2020 Interval Research Corp copyright 1998 30