Environmental Data Analysis with Mat Lab 2 nd
Environmental Data Analysis with Mat. Lab 2 nd Edition Lecture 19: Smoothing, Correlation and Spectra
SYLLABUS Lecture 01 Lecture 02 Lecture 03 Lecture 04 Lecture 05 Lecture 06 Lecture 07 Lecture 08 Lecture 09 Lecture 10 Lecture 11 Lecture 12 Lecture 13 Lecture 14 Lecture 15 Lecture 16 Lecture 17 Lecture 18 Lecture 19 Lecture 20 Lecture 21 Lecture 22 Lecture 23 Lecture 24 Lecture 25 Lecture 26 Using Mat. Lab Looking At Data Probability and Measurement Error Multivariate Distributions Linear Models The Principle of Least Squares Prior Information Solving Generalized Least Squares Problems Fourier Series Complex Fourier Series Lessons Learned from the Fourier Transform Power Spectra Filter Theory Applications of Filters Factor Analysis Orthogonal functions Covariance and Autocorrelation Cross-correlation Smoothing, Correlation and Spectra Coherence; Tapering and Spectral Analysis Interpolation Linear Approximations and Non Linear Least Squares Adaptable Approximations with Neural Networks Hypothesis testing Hypothesis Testing continued; F-Tests Confidence Limits of Spectra, Bootstraps
Goals of the lecture examine interrelationships between smoothing, correlation and power spectral density
review
Autocorrelation and Cross-correlation
Autocorrelation Measure of correlation in time series at different lags u(t) t
Autocorrelation Measure of correlation in time series at different lags t t lag, multiply and sum area no lag a(t) 0 lag, t
Autocorrelation Measure of correlation in time series at different lags t t lag, multiply and sum area small lag a(t) 0 lag, t
Autocorrelation Measure of correlation in time series at different lags t t lag, multiply and sum area large lag a(t) 0 lag, t
Autocorrelation Measure of correlation in time series at different lags t t lag, multiply and sum area a(t) 0 lag, t
Autocorrelation Measure of correlation in time series at different lags t t lag, multiply and sum area a(t) 0 lag, t a(t)=u(t)⋆u(t)
crooss-correlation Measure of correlation between two time series at different lags u(t) v(t) t t
crooss-correlation Measure of correlation between two time series at different lags u(t) v(t) t t c(t)=u(t)⋆v(t)
important relationships c(t) = u(t)⋆v(t) = u(-t)*v(t) c(ω) = u*(ω) v(ω) a(ω)= |u(ω)|2
rough time series u(t) t sharp autocorrelation a(t) 0 lag, t wide spectrum 0 frequency, ω |u(ω)|2
smooth time series u(t) t wide autocorrelation a(t) 0 lag, t narrow spectrum 0 frequency, ω |u(ω)|2
v(t) t rough timeseries a(t) 0 lag, t 0 v(t) lag, t t a(t) 0 lag, t
Part 1 Smoothing a Time Series
smoothing as filtering (example of 3 -point smoothing)
smoothing as filtering (example of 3 -point smoothing) non-causal
fix-up allow for a delay
fix-up allow for a delay dsmoothed and delayed = s * dobs causal filter, s
triangular smoothing filters 3 points si index, i si 21 points index, i
smoothing if Neuse River Hydrograph
question how does smoothing effect the autocorrelation of d
answer the autocorrelation of s acts as a smoothing filter on the autocorrelation of d
effect of smoothing on autocorrelation
effect of smoothing on autocorrelation of smoothed time series
effect of smoothing on autocorrelation of smoothed time series everything written as convolution
effect of smoothing on autocorrelation of smoothed time series everything written as convolution regrouped
effect of smoothing on autocorrelation of smoothing filter convolved with * autocorrelation of time series
answer the autocorrelation of s acts as a smoothing filter on the autocorrelation of d
Part 2 What Makes a Good Smoothing Filter?
dsmoothed(t) = s(t) * dobs(t) then by the convolution theorem
dsmoothed(t) = s(t) * dobs(t) then by the convolution theorem so what’s this look like?
example of a uniform or “boxcar” smoothing filter s(t) 1/T 0 T time, t
take Fourier Transform where sinc(x) = sin(πx) / (πx)
A) T=3 B) T=21
B) T=21 falls off with frequency (good)
B) T=21 bumpy side lobes (bad)
a box car filter does not suppress high frequencies evenly the challenge find a filter that suppresses high frequencies evenly
Normal Function Fourier Transform of a Normal Function is a Normal Function (which has no side lobes)
B) T=3 A) L=3 B) T=21
but a Normal Function is non-causal (unless you truncate it, in which case it is not exactly a Normal Function)
Box Car Normal Function simplicity sidelobes Triangle
Part 3 Designing a Filter that Suppresses Specific Frequencies
General form of the IIR Filter, f
z-transform of the IIR filter
General form of the IIR Filter z-transform
General form of the IIR Filter z-transform ratio of polynomials
z-transform of the IIR filter u(z) as a product of its factors roots of u(z) roots of v(z) as a v(z) z-transform ratio of polynomials product of its factors
so designing a filter is equivalent to specifying the roots of the two polynomails u(z) and v(z)
at this point we need to explore the relationship between the Fourier Transform and the z-transform
Answer the Fourier Transform is the z-transform evaluated at a specific set of z’s
Relationship between Fourier Transform and Z-transform since
Relationship between Fourier Transform and Z-transform Fourier Transform since
Relationship between Fourier Transform and Z-transform Fourier Transform discrete times and frequencies since
Relationship between Fourier Transform and Z-transform Fourier Transform discrete times and frequencies z-transform since
Relationship between Fourier Transform and Z-transform Fourier Transform discrete times and frequencies z-transform since specific choice of z’s
in words the Fourier Transform is the z-transform evaluated at a specific set of z’s
there are N specific z’s zk or with θ
they plot as equally-spaced points around a “unit circle” in the complex z-plane unit circle, |z|2=1 q real z zero frequency Nyquist frequency imag z
Back to the IIR Filter roots of u(z) roots of v(z)
Back to the IIR Filter (z-zju) is zero at z=zju produces a low amplitude region near z=zju called a “zero”
Back to the IIR Filter 1/(z-zkv) is infinite at z=zku produces a high amplitude region near z=zkv called a “pole”
so build a filter by placing the poles and zeros at strategic points in the complex z-plane
Rules zeros suppress frequencies poles amplify frequencies all poles must be outside the unit circle (so vinv converges) all poles, zeros must be in complex conjugate pairs (so filter is real)
A) B)
A) zero near zero frequency suppresses low frequencies “high pass filter” B) zero near the Nyquist frequency suppresses high frequencies “low pass filter”
A) B)
A) poles near ± a given frequency amplify that frequency “band pass filter” B) poles and zeros near ± a given frequency attenuate that frequency “notch filter”
something useful a tunable band pass filter |f(ω)|2 -fny -f 2 -f 1 0 f 1 f 2 +fny frequency, f
Chebychev band-pass filter: 4 zeros, 4 poles
Chebychev band-pass filter: 4 zeros, 4 poles 2 zeros pole
not quite as boxy as one might hope …
In Mat. Lab
Ground velocity at Palisades NY
Ground velocity at Palisades NY Low pass filter
Ground velocity at Palisades NY high pass filter
- Slides: 81