Lecture 10 Time Series Model Introduction to ANN

Outline • Time series models • Linear Time Series Models – Moving Average Model

Time Series • What is a time series? Modeling of a time series –

Time Series Models • Most time series are sampled • Notations from continuous time

Time Series Prediction • Problem Statement Given {y(i); i = t 1, …} estimate

A Dynamic Time Series Model • State {x(t)} y(t) – Past values of a

Linear Time Series Models • • y(t) is a linear combination of x(t) and/or

Moving Average Model • • Cross correlation function • • • Auto-correlation function: (C)

Finding MA Coefficients • • Problem: Given a MA time series • {y(t)}, how

Auto-Regressive Model • • (C) 2001 -2013 by Yu Hen Hu Ry(m) is replaced

Auto-Regressive, Moving Average (ARMA) Model • • A combination of AR and MA model.

Nonlinear Time Series Model • f(x(t), u(t)) is a nonlinear function or mapping: •

System Identification Problem Consider an unknown system (Plant) with output y(t) which depends on

Control Problem Given: desired output y*(t), t 1 t t 2 Find: input u(t),

System Identification • With the same input {u(t)}, find a mathematical model which will

Slides: 15

Download presentation

Lecture 10 Time Series Model Introduction to ANN & Fuzzy Systems

Outline • Time series models • Linear Time Series Models – Moving Average Model – Auto-regressive Model – ARMA model • Nonlinear Time Series Estimation • Applications (C) 2001 -2013 by Yu Hen Hu Introduction to ANN & Fuzzy Systems 2

Time Series • What is a time series? Modeling of a time series – A scalar or vector-valued function of time indices • Examples: – Stock prices – Temperature readings – Measured signals of all kinds • What is the use of a time series? – Prediction of future time series values based on past observations (C) 2001 -2013 by Yu Hen Hu Values of a time series at successive time indices are often correlated. Otherwise, prediction is impossible. Most time series can be modeled mathematically as a wide-sense stationary (WSS) random process. The statistical properties do not change with respect to time. Some time series exhibits chaotic nature. A chaotic time series can be described by a deterministic model but behaves as if it is random, and highly un-predictable. Introduction to ANN & Fuzzy Systems 3

Time Series Models • Most time series are sampled • Notations from continuous time physical – y(t): time series value at quantities at regular sampling present time index t. intervals. One may label each – y(t-1): time series value one such interval with an integer unit sample interval before t. index. E. g. – y(t+1): the next value in the future. {y(t); t = 0, 1, 2, …}. • Basic assumption • A time series may have a – y(t) can be predicted with starting time, say t = 0. If so, certainly degree of accuracy it will have an initial value. by its past values {y(t k); k > 0} • In other applications, a time and/or the present and past series may have been run for values of other time series a while, and its past value can such as {x(t m); m 0} be traced back to t = . (C) 2001 -2013 by Yu Hen Hu Introduction to ANN & Fuzzy Systems 4

Time Series Prediction • Problem Statement Given {y(i); i = t 1, …} estimate y(t+to), to 0 such that is minimized. when to = 1, it is called a 1 step prediction. Sometimes, additional time series {u(i); i = t, t 1, …} may be available to aid the prediction of y(i) (C) 2001 -2013 by Yu Hen Hu • The estimate of y(t + t 0)that minimizes C is the conditional expectation given past value and other relevant time series. • This conditional expectation can be modeled by a linear function (linear time series model) or a nonlinear function. Introduction to ANN & Fuzzy Systems 5

A Dynamic Time Series Model • State {x(t)} y(t) – Past values of a time series can be summarized by a finitedimensional state vector. Time series model • Input {u(t)} – Time series that is not dependent on {y(t)} • The mapping is a dynamic system as y(t) depends on both present time inputs as well as past values. (C) 2001 -2013 by Yu Hen Hu x(t) memory Input u(t) x(t+1) x(t) = [x(t) x(t 1) … x(t p)] consists past values of {y(t)} u(t) = [u(t) u(t 1) … u(t q)] Introduction to ANN & Fuzzy Systems 6

Linear Time Series Models • • y(t) is a linear combination of x(t) and/or u(t). White noise random process model of input {u(t)}: – – • E(u(t)) = 0 E{u(t)u(s)} = 0 if t s; = s 2 if t = s. Three popular linear time series models: 1. Moving Average (MA) Model: 2. Auto-Regressive (AR) Model: 3. Moving Average, Auto-regressive (ARMA) Model: (C) 2001 -2013 by Yu Hen Hu Introduction to ANN & Fuzzy Systems 7

Moving Average Model • • Cross correlation function • • • Auto-correlation function: (C) 2001 -2013 by Yu Hen Hu An MA model is recognized by the finite number of non-zero auto-correlation lags. {b(m)} can be solved from {Ry(k)} using optimization procedure. Example: If {u(t)} is the stock price, then is a moving average model – An average that moves with respect to time! Introduction to ANN & Fuzzy Systems 8

Finding MA Coefficients • • Problem: Given a MA time series • {y(t)}, how to find {b(m)} withtout knowing {u(t)}, except the knowledge of s 2? One way to find the MA model coefficients {b(m)} is spectral factorization. Consider an example: Given {y(t); t = 1, …, T}, estimate autocorrelation lag • Power spectrum B(z) B(1/z*) zeros b(1)/b(0) (b(1)/b(0))* poles 0 Spectral factorization: – Compute S(z) from {R(k)} and factorize its zeros and poles to construct B(z) • For this MA(1) model, R(k) 0 for k > 1. (C) 2001 -2013 by Yu Hen Hu • Or comparing the coefficients of polynomial and solve a set of nonlinear equations. Introduction to ANN & Fuzzy Systems 9

Auto-Regressive Model • • (C) 2001 -2013 by Yu Hen Hu Ry(m) is replaced with R(m) to simplify notations. R is a Töeplitz matrix and is positive definite. Fast Cholesky factorization algorithms such as the Levinson algorithm can be devised to solve the Y-W equation effectively. Introduction to ANN & Fuzzy Systems 10

Nonlinear Time Series Model • f(x(t), u(t)) is a nonlinear function or mapping: • A neuronal filter u(t) – MLP – RBF D • Time Lagged Neural Net (TLNN) u(t 1) D – The input of a MLP network is formed by a time-delayed segment of a time series. y(t) + u(t) (C) 2001 -2013 by Yu Hen Hu f( ) D MLP + u(t M) e(t) D u(t 1) D D u(t M) Introduction to ANN & Fuzzy Systems 12

System Identification Problem Consider an unknown system (Plant) with output y(t) which depends on current and past input u(t). • System Identification Problem – Given: input u(t) and output y(t), 0 t tmax, Find T[ • ] such that (C) 2001 -2013 by Yu Hen Hu Introduction to ANN & Fuzzy Systems 13

Control Problem Given: desired output y*(t), t 1 t t 2 Find: input u(t), t 0 t t 2 (t 0 t 1) such that y(t) —> y*(t) for t 1 t t 2 • Path-Following Control Problem – Entire tragectory of the desired output sequence is specified (t 1 ~ t 0) • Reinforcement Learning Problem – Only the destination is given. The intermediate path is not specified (t 1 >> t 0). (C) 2001 -2013 by Yu Hen Hu Introduction to ANN & Fuzzy Systems 14

System Identification • With the same input {u(t)}, find a mathematical model which will best approximate the output sequence. y(t) u(t) Unknown + e(t) Plant Model ^ y(t) • Essentially, a function approximation problem. Due to the particular dynamics of the plant, recurrent ANN are often considered. (C) 2001 -2013 by Yu Hen Hu Introduction to ANN & Fuzzy Systems 15