EMERGING APPLICATIONS IN IMAGE PROCESSING image analysis operations

EMERGING APPLICATIONS IN IMAGE PROCESSING image analysis operations applied to hyperspectral images for noninvasive sensing of food quality. 7 Jan 2016 -Gamal M. Elmasry [ Dept. of CS&E Toyohasi University of Technology, japan ] &, [Suez Canal University, Agricultural Engg. Dept. , Egypt] -Shigeki Nakauchi [ Dept. of CS&E Toyohasi University of Technology, japan ]

Overview • Introduction • Fundamentals of Hyperspectral imaging • Type of Hyperspectral imaging • Analysis framework of Hyperspectral images 1: Image acquisition • 2: Data extraction • 3: Data modeling • • Image Segmentation • conclusion

Introduction • Traditionally process of food quality assurance is performed by human panels, chemical analysis or mechanical method. • • Problem : destructive, time consuming, laborious and costly and inconsistent due to difference in human response. Effective monitoring systems and an intelligent approach is required • providing information about composition of food material. • Need for fast and non-invasive method • Quality seeks many properties and characteristics • • Measurement of optical properties of food is one of the most successful non-destructive techniques Traditional Imaging vs Spectroscopy : Traditional Imaging used in rapid identification of quality problems but it doesn’t provide information about composition of food. • Spectroscopy is non-invasive technique for getting physicochemical characteristics of food but fails to provides virtual information of food. • • Hyperspectral Imaging : integrating spectroscopic and photographic technique. • It is inspection tool for providing geometrical, textural, appearance feature and also physicochemical characteristics of food during quality and safety assessment. •

Fundamentals of Hyperspectral imaging • Goal is to obtain spectrum for each pixel in the image in order to find objects, identify material etc. • Human eyes sees color mostly in three bands (red, green, blue) but spectral imaging divides spectrum into many more bands beyond visible band. • Sensors collects information as a set of images where each one represents a narrow wavelength range of electromagnetic spectrum. • Images are combine to form 3 -D hyperspectral data cube for processing and analysis. • Hyperspectral sensors and processing systems used in various fields • astronomy, agriculture, biomedical, geoscience, physics, surveillance etc.

Hyperspectral data cube

Types of Hyperspectral Imaging [ Based on relative movement between samples and the detection unit (i. e. camera and spectrograph) ] • Point-to-point scanning ( whisk-broom/spotlight /across track imaging) Use a mirror to reflect light onto a single detector. • The mirror moves back and forth, to collect measurements from one pixel in the image at a time. • Moving parts make this type of sensor expensive and more likely to wear out. • A whisk broom scanner sweeps in a direction perpendicular to the flight path, collecting one pixel at a time. Image courtesy of Florian Hillen.

Types of Hyperspectral Image [ Based on relative movement between samples and the detection unit (i. e. camera and spectrograph) ] • Line-by-line scanning (push broom imaging/ along track scanners) use a line of detectors arranged perpendicular to the flight direction of the spacecraft. • As the spacecraft flies forward, the image is collected one line at a time, with all of the pixels in a line being measured simultaneously. • A push broom scanner receives a stronger signal than a whisk broom scanner because it looks at each pixel area for longer duration. • A linear array detector advances with the spacecraft's motion, producing successive lines of image data (analogous to the forward sweep of a push broom). Image courtesy of Florian Hillen.

• a) Line-by-line image scanning for acquiring hyperspectral image. • b) Hyperspectral image could be displayed as individual 2 -D sub image I(x, y) at any given wavelength. • spectra I(λ) at any given pixel in the image. • • 3 -D Voxel I(x, y, λ) , 2 -D pixel I( x, y) • Set of voxels at all wavelength is called spectrum. Low reflectance( high absorption) looks darker High reflectance( low absorption) looks brighter No single sub-image at one wavelength has sufficient information

• Advance digital color camera but with a fine spectral resolution at given wavelengths of illumination. • The value of any pixel within a hyperspectral image relies on light irradiance at the corresponding point • sensitivity and exposure time of the camera. • • Instead of measuring three primary color, the device measures the radiation reflected by each pixel at a large number of visible or invisible frequency(or wavelength) bands.

Analysis framework of Hyperspectral images • To get successful processing of hyperspectral data-cube Acquire hyperspectral image • Extract spectral data • Build the calibration multivariate model • Process the final visualization • • Three module are usually involved in processing of hyperspectral images. 1: Image acquisition and calibration • 2: Data extraction and data treatment • 3: Data modeling •

1: Image acquisition, calibration and image pre-processing • Acquisition of high-quality hyperspectral image in a definite spectral range • Ideal acquisition conditions in term of • • acquisition mode Illumination arrangement Spectral resolution Selection of detector Scanning speed Frame rate Exposure time

1: Image acquisition, calibration and image pre-processing • Calibration • Dark image is acquired by capturing an image in dark condition and with the lens of camera covered with its opaque cap White image is captured using a standard surface to obtain maximum possible intensity of each pixel in each wavelength. Correct the acquired images: use two reference images (dark D ij , white Wij ) called radiometric calibration because it converts the digital intensity values registered by the sensor to real reflectance value. Pij is pixel value at position ( i, j ). Wavelength calibration Food samples may be spherical, elliptic or even irregular in shape. Previous kind of correction use to correct spectral variation but doesn’t consider variation in geometry. • Shape is a factor during calibration as it affect scattering and reflectance patterns. • Reflectance of the samples at point (x, y) at wavelength Reflectance calibration Factor that relates the percentage of direct and diffuse light arrive at samples Modulates the amount of direct light reflected at each point

2: Data extraction and data treatment • Spectral information of image that represents physicochemical properties could be extracted from • • Segmented objects in image as main region of interest(ROI). Spectral enhancement involve various pretreatment schemes • • • Spectral filtering : control the impact of noise Smoothing: capture important patterns in the data, while leaving out noise Mean centric: using mean low-pass filter Auto scaling : scaling its height and width Baseline correction : correct baseline of signal with peak.

3: Data modeling and image post-processing • Numbers of appropriate multivariate chemo-metric schemes are available to build calibration model for different sample type and to acquire accuracy. • MLR-multi-linear regression. is used to find relationship between one continuous dependent variable from two or more independent variables PLS-partial least square regression: statistical method that find some relation between partial components. • SVM-support vector machine : analyze data used for classification and regression analysis. • ANN-artificial neural network : used to estimate or approximate functions that can depend on a large • number of inputs. • Selection of these feature-related wavelengths is usually achieved once in offline experiment(in quality control labs) and then implemented to construct a real-time multi-spectral imaging system for online screening in industries.

Image Segmentation • The process of image segmentation in intended to divide the image into multiple segments. • This process is typically used to locate different objects by assigning a label to every pixel. • Pixel with same label share same visual and/or spectral characteristics. • Methods typically used in image segmentation • • • Thresholding Clustering Histogram based method Edge detection method Split and merge method Compression based method

Image Segmentation

Conclusion • Hyperspectral imaging is a logical extension to traditional spectroscopy • Use of Hyperspectral imaging as non-invasive assessment of food quality. • Used to get spectral characteristics and physicochemical properties of food. • Hyperspectral imaging technique have a lot to offer not only in food quality monitoring but also In medicine • Pharmaceutical • Cosmetics • Polymer and other industries. •