NIR Preprocessing With focus on NIR smund Rinnan

NIR Preprocessing With focus on NIR Åsmund Rinnan , Frans van den Berg, Rasmus Bro, Søren Balling Engelsen, Lars Nørgaard , Jonas Thygesen

Introduction • • NIR How often is it used? What methods are used? Why is it used? • NIR is a measurement technique “screaming” for preprocessing Techniques Summary

Introduction NIR Fructose Glucose 0 100 50 50 75 25 Introduction Techniques Summary

Introduction Techniques NIR • Correction of light scatter – MSC/ ISC – PMSC – EMSC/ EISC • SNV • Derivation – Norris&Williams – Savitsky&Golay • • Introduction Techniques Summary Baseline-correction Normalization Detrend Use of reference values – – O-PLS OSC OS SIS

Introduction Why NIR Baseline with a slope/ curve Nonlinearity A “fat” baseline Introduction Techniques Summary

Introduction ISC MSC Techniques Summary NIR Raw spectrum • Offset • Normalization Reference 1. P Geladi, D Mac. Dougal, H Martens (1985): Linearization and scatter correction for near-infrared reflectance spectra of meat , Applied Spectroscopy, 39, 491 -500

Introduction MSC/ ISC Techniques Summary NIR • Offset • Normalization

Introduction EISC Techniques Summary NIR • Offset • Baseline • Normalization 1. 2. 3. 4. H. Martens and E. Stark (1991): Extended multiplicative signal correction and spectral interference subtraction: new preprocessing methods for near infrared spectroscopy, Journal of Pharmaceutical and Biomedicinal Analysis, 9, 625 -635 H Martens, JP Nielsen, SB Engelsen (2003): Light scattering and light absorbance separated by extended multiplicative signal correction. The application to near -infrared transmission analysis of powder mixtures , Analytical chemistry, 75, 394 -404 DK Pedersen, H Martens, JP Nielsen, SB Engelsen (2002): Near-Infrared Absorption and Scattering Separated by Extended Inverse Signal Correction (EISC): Analysis of Near-Infrared Transmittance Spectra of Single Wheat , Applied Spectroscopy, 56, 1206 -1214 Thennadil, SN, Martin, EB (2005): Empirical preprocessing methods and their impact on NIR calibrations - a simulation study , Journal of Chemometrics, 19, 77 -89

EMSC (w/ 2. degree & wavelength) NIR Introduction Techniques Summary • Offset • Baseline • Normalization

Introduction Px. SC Techniques Summary NIR • Offset • Baseline • Normalization X = EM EI M I 1. Isaksson, T. ; Kowalski, B. R. (1993): Piece-wise multiplicative scatter correction applied to Near-Infrared diffuse transmittance data from meat products, Applied Spectroscopy, 47, 702 -09

PMSC (window size = 35) Introduction Techniques Summary NIR • Offset • Normalization

Introduction SNV Techniques Summary NIR • Offset • Normalization 1. R. J. Barnes, M. S. Dhanoa, S. J. Lister (1989): Standard Normal Variate Transformation and De-trending of Near-Infrared Diffuse Reflectance Spectra, Applied Spectroscopy, 43, 772 -777

Introduction Techniques SNV vs MSC Summary NIR MSC SNV

Introduction Techniques Detrend Summary NIR • Offset • Baseline 1. R. J. Barnes, M. S. Dhanoa, S. J. Lister (1989): Standard Normal Variate Transformation and De-trending of Near-Infrared Diffuse Reflectance Spectra, Applied Spectroscopy, 43, 772 -777

Derivation Savitsky-Golay Introduction Techniques Summary NIR • Offset • Baseline 1. 2. A Savitsky, MJE Golay (1964): Smoothing and differentiation of data by simplified least squares procedures, Analytical Chemistry, 36 (8), 1627 -1639 P. A. Gorry (1990): General Least-Squares smoothing and differentiation by the convolution ( Savitsky-Golay) method, Analytical Chemistry, 62, 570 -573

Derivation Savitsky-Golay Introduction Techniques Summary NIR • Offset • Baseline

Derivation Savitsky-Golay Introduction Techniques Summary NIR • Offset • Baseline

Derivation Norris-Williams Introduction Techniques Summary NIR • Offset • Baseline • Normalization 1. 2. Norris, KH (1983): in Food Research and Data Analysis (Eds: Martens H and Russwurm H Jr), Applied Science, London, 95 -114 Norris KH, Williams PC (1984): Optimization of mathematical treatments of raw near-infrared signal in the measurement of protein in hard red spring wheat: I. Influence of particle size, Cereal Chemistry, 61, 158 -165

Derivation Norris-Williams Introduction Techniques Summary NIR • Offset • Baseline • Normalization

Derivation Norris-Williams Introduction Techniques Summary NIR • Offset • Baseline • Normalization

Introduction Preprocessing NIR Techniques Summary • PMSC: Sensitivite to window size • EMSC: Small effect on baseline Detrend is better • MSC and SNV are practical the same • Savitsky-Golay is more robust than Norris. Williams

Introduction Techniques Preprocessing Summary NIR Offset Baselinj Normalization e (P)MSC/ISC/SNV (P)EMSC/EISC Detrend Deriveration x x x x x

NIR Thanks for the attention www. models. life. ku. dk