Vital Sign Estimation from Passive Thermal Video Ming

Vital Sign Estimation from Passive Thermal Video Ming Yang 2, Qiong Liu 1, Thea Turner 1, Ying Wu 2 1 FX Palo Alto Laboratory, Inc. , 3400 Hillview Ave. , Palo Alto, CA 94304 Goal Challenges Ø Robust harmonic analysis with low signal-to-noise ratio (SNR) temperature modulation signal, e. g. modulation magnitude 0. 1 K vs. camera sensitivity 0. 025 K. Motivations Pioneering work Ø A novel contact-free vital sign measurement method. Ø N. Sun, M. Garbey, A. Merla, I. Pavlidis. Imaging the cardiovascular pulse. CVPR 2005. (S) Ø Low risk of harm & convenience for quick deployment. Ø S. Y. Chekmenev, A. A. Farag, E. A. Essock. Multiresolution approach for non-contact measurements of arterial pulse using thermal Ø Potential applications: airport heath screening, longimaging. CVPR 2006 Workshop. term elder care, workplace preventive care, etc. Overview of our approach Automatic ROI segmentation and alignment Ø Region-of-interests segmentation by thresholding the isotherms and alignment by contour tracking. Ø Segment the initial ROI by selecting the isotherm with the sharpest gradient. Ø Align the ROI by tracking the contour Ø Extract the temporal signals for individual pixels inside the ROI and denote by CVPR 24 June 2008 – 26, 2008 Ø Heart rate estimation results Est. bpm Diff. RMSE 65. 3 65. 8 +0. 5 1. 9 2000 66. 6 63. 9 -2. 7 3. 9 30 1750 65. 7 64. 7 -1. 0 3. 3 4 60 3000 59. 8 60. 7 +0. 9 2. 5 5 60 3500 60. 7 60. 3 -0. 4 3. 3 6 60 2500 66. 3 53. 0 -3. 3 3. 9 7 60 3000 61. 1 60. 9 -0. 2 2. 3 8 115 5000 64. 0 65. 0 +1. 0 3. 8 9 115 5000 78. 9 80. 1 +1. 2 1. 9 Ø 20 subjects for heart rate estimation 10 115 5000 65. 2 64. 4 -0. 8 1. 7 11 115 5000 62. 8 66. 2 +3. 4 4. 2 Ø 7 subjects for respiratory rate estimation Ground truth: ADI Power. Lab 4/30 12 115 5000 63. 5 62. 4 -1. 1 3. 2 13 115 5000 73. 3 72. 6 -0. 7 1. 8 14 115 5000 86. 6 87. 9 +1. 3 4. 9 15 115 5000 78. 7 76. 5 -2. 2 3. 1 16 115 5000 75. 3 74. 7 -0. 7 1. 9 17 115 5000 83. 1 83. 2 +0. 1 2. 1 18 115 5000 67. 2 68. 2 -1. 0 1. 3 19 115 5000 67. 6 69. 3 +1. 7 2. 8 20 115 5000 68. 7 70. 1 +1. 4 2. 9 Infrared camera: Subject # fps 1 30 2000 2 30 3 Ø Mid wave: 3. 0 -5. 0 microns Ø Resolution: 640*512 pixels with 14 bits Ø Frame rate: 30/60/115 fps Ø Sensitivity: about 25 m. K Test dataset: Age 20 -60, F: 8 and M: 12 Ø The initial ROI segmentation results # of frames GT bpm Ø Point-by-point comparisons Ø Respiratory rate estimation results Subject # fps Est. bpm Diff. 4 60 3000 18 15. 8 -2. 2 7 60 3000 17 15. 1 -1. 9 10 115 5000 11 11. 8 +0. 8 11 115 5000 17 16. 8 -0. 2 14 115 5000 16 13. 9 -2. 1 15 115 5000 15 13. 1 -1. 9 17 115 5000 20 18. 5 -1. 5 19 115 5000 16 15. 2 -0. 8 Signal enhancement and outlier removal # of frames GT bpm Robust harmonic analysis Ø Perform N-point (N=1024/2048/4096) FFT of all temperature signals of all pixels using a sliding window: Ø Signal enhancement using a non-linear filter, and outlier removal by pixels-of-interests clustering. Ø Robust harmonic analysis by dominant frequency voting. of EECS, Northwestern Univ. , 2145 Sheridan Rd. , Evanston, IL 60208 Experiments Ø Accurate subject alignment for temporal signal extraction, e. g. involuntary muscular movements are inevitable. To explore contact-free heart rate and respiratory rate detection through measuring infrared light modulation emitted near superficial blood vessels or a nasal area. 2 Dept. ØNon-linear filtering by taking the point-by-point minimum of a rectangle window Wr(t) and a Hamming window Wh(t) Ø Cluster H(xj, f ) in the band of interest (40 -100 bpm Conclusions for heart rates, and 6 -30 bpm for respiratory rates) using K-means, then select the largest cluster to estimate. Ø Insensitive to initialization and robust to gentle subject movement and facial expressions. Ø More stable estimation results compared with the state-of-the-art methods.