Examining Photo Response NonUniformity for the Comparison of

Examining Photo Response Non-Uniformity for the Comparison of Cameras Zeno Geradts Ph. D / Maarten van der Mark BS / Wiger van Houten MS Partially funded by the European Commission within the project FIDIS www. fidis. net Nederlands Forensisch Instituut Laan van Ypenburg 6, 2497 GB Den Haag

Overview • • Introduction Noise and PRNU sources Comparison of denoising methods Application to You. Tube videos Performance Statistics (Bayesian) Conclusions

FIDIS • Network of Excellence within the European Union, with 20 partners ranging from Universities, Privacy protection agencies and companies such as IBM and Microsoft • www. fidis. net

Introduction • Each image sensor has a unique `fingerprint’, the PRNU pattern, that is detectable in all images the sensor produces • This `fingerprint’ is used to establish the image origin

Noise and PRNU sources • Different types of noise: • 2 Categories: I. Random noise I. III. Pattern noise (PRNU and FPN) I. • Temporally variable Statistical distributions Can be reduced by averaging multiple frames Not temporally variable but systematic (spatial variable) How do image sensors work? • • CCD image sensors CMOS Active Pixel Sensors

CCD image sensors • Basic building block (gate) • Silicon substrate • Apply positive voltage to the gate • Electron-hole pairs generated in depletion region are confined • Read out • Charge converted to voltage at the sense node

CMOS APS image sensors • Cpd: photodiode capacitor • M 1: reset transistor • M 2: source follower • M 3: row select transistor • M 4: bias transistor

Noise and PRNU sources (2) • The characteristic pattern extracted from the images contains 2 systematic contributions: • Multiplicative PRNU, depends on illumination • Fixed Pattern Noise (FPN), does not depend on illumination • Also: common components in the extracted pattern (e. g. CFA interpolation)

Noise and PRNU sources (3) • FPN (general): • Crystal defects in the silicon lattice introduced during growth • Impurities • The size of the detector/potential well • Contamination during fabrication • Non-uniform oxide/gate thickness • In CMOS: additional sources (each transistor)

Noise and PRNU sources (4) • PRNU The • • depth of the detector/potential well Larger active area: more incident photons Non-uniform oxide layer: results in nonuniform potential wells Deeper potential well: more photons absorbed (wavelength dependent) In CMOS: additional sources

Image denoising – pattern extraction • Denoising algorithms do not discriminate between noise and image details. There is always a tradeoff. • PRNU is a nonperiodic discontinuous signal • Pattern = Image – F(Image) , F is the denoising filter • Correlate the patterns from `questioned’ videos with reference patterns

Old method • Gaussian smoothing filter • Advantages: • Simple • Very fast • Disadvantages: • Distorts edge integrity • Image residue left behind in the pattern

New method • Wavelet based denoising filter [Lukas et al]* • Disadvantages • Slower • Diadic images only • Advantages • Preserves edges (edge detection) • Spatial adaptive • Works really well *Digital Camera Identification from Sensor Pattern Noise (2005) – Lukas, Fridrich, Goljan

You. Tube (1) • Accepts large amount of input formats: Xvid, Div. X, WMV, 3 GP, … • Downloadable as H. 264/MPEG-4 AVC with e. g. keepvid. com • Maximum resolution (H. 264): 480 x 360 • Aspect ratio generally does not change

You. Tube (2) • When the resolution of the video uploaded to You. Tube is smaller than 480 x 360, generally no resolution change occurs (exceptions) • When the resolution of the video exceeds 480 x 360, the resolution is changed to 480 x 360 or lower (depending on the aspect ratio) e. g. 640 x 480 x 360 640 x 360 480 x 270

In practice – natural video • Record 30 seconds of natural video, recorded with XVID/WMV • Upload this video to You. Tube • Download the video with keepvid. com • Extract individual frames • Estimate noise pattern

In practice – flatfield video • If natural video downloaded has a resolution lower than 480 x 360, record the flatfield video (RAW) with the same resolution, use these frames • Otherwise (e. g. 640 x 480), record flatfield video in native resolution, and upload (download) them to (from) You. Tube • Alternative: resize flatfield video to match the dimension of the You. Tube video (e. g. 640 x 480 x 360)

Performance (1) • Works really well when natural video is recorded in native resolution, both with XVID and WMV 9 (Messenger) with large range of codec settings, even for shorter samples • Works reasonably well when aspect ratio has been changed • Problematic with very low resolution (Vodafone: 176 x 144) • Does not always work when the video was binned during recording (e. g. native resolution of 640 x 480, recorded in 320 x 240 x)

Performance (2) – Creative Live! IM Webcam • Problem: need to set two parameters for extracting the patterns • These parameters depend on a large amount of variables: • • • Content Compression Codec Resolution etc. • Only from empirical data; impossible in casework

Comparison – Creative Live! IM, 640 x 480 native, recorded in 352 x 288 • Old method New Method (σ_n=4. 5, (σ=0. 6, threshold σ_f=4. 5): 5, 4 x 4 averaging): Camera Correlation 11. 5 0. 2048 11. 4 0. 0531 11. 6 0. 1893 11. 1 0. 0271 11. 4 0. 1669 11. 2 0. 003 11. 1 0. 1273 11. 7 -0. 0106 11. 2 0. 1191 11. 5 -0. 0237 11. 7 0. 1061 11. 6 -0. 0263

Logitech Communicate STX • (1) Logitech : 640 x 480 (native), XVID q 4 • Logitech : 640 x 480 (native), variable XVID quality

Logitech Communicate STX • (2) Logitech : 320 x 240, XVID q 4 • Logitech : 320 x 240, variable XVID quality

Test yourself sourceforge. net search PRNU

Likelihood ratio • Comparing patterns • Value of the evidence:

Likelihood ratio • In practice: • No overlapping histograms • We do not know the origin of the questioned image

Likelihood ratio • To artificially find the pdf of Hd at Hp we use an estimator • Example: add-constant estimator Suppose you look at the colours of passing cars: • 1 blue car, 1 red car, and 1 green car • Q: what is the chance the next car is blue? • A: 1/3? Or smaller? • A: (1+1)/7 for blue; 1/7 for a new colour

Likelihood ratio • More complicated estimators available, but the essence is the same: reserve a probability for unseen species, based upon the data we encountered so far • Simple Good-Turing approach

Likelihood ratio • SGT: problems: • Car colours (green, blue, red, etc) is discrete, but some colours exist that can be seen as both (blue or green? ) • We are considering correlation values, continuous data. We have to divide the values into `species’ ourselves

Likelihood ratio, recap

Conclusion (1) • We understand the origins of the pattern noise that is used to perform device identification • We understand which random noise sources are present • A likelihood ratio can be found, but there are some unanswered questions before this approach can be used

Conclusions (2) • It is possible to identify the source video camera based on the PRNU pattern, even after the video has been uploaded to You. Tube • The new method, although computationally more intensive, performs much better

Conclusions (3) • Limitations: (very) low resolution videos, changed aspect ratios, ‘subsampled’ recordings • No way to find the ideal parameter directly from the video or frames: this is a problem when the abovementioned limitations are met

Future research • Using the methods on databases of child pornography to link cameras • Improvement of algorithms • Validation in casework

Questions ? • zeno@holmes. nl