Minimization of Residuals of ARMA Models

We consider an algorithm for optimization of parameters of the ARMA model. This model is simple and may be regarded as a good first approximation. Denote by $y_t$ the value of $y$ at the moment $t$. Denote by $a=(a_1,...,a_q)$ a vector of auto-regression (AR) parameters, and by $b=(b_1,...,b_q)$ a vector of moving-average (MA) parameters.

$$y_t-\sum_{i=1}^p a_i y_{t-i}= \epsilon_t - \sum_{j=1}^q b_j \epsilon_{t-j},\ t=1,...,T.$$ (5)

The residual

$$\epsilon_t = y_t-\sum_{i=1}^p a_i y_{t-i}+ \sum_{j=1}^q b_j \epsilon_{t-j}$$ (6)

or

$$\epsilon_t = B_t+\sum_{i=1}^p a_i A_t(i).$$ (7)

Here

$$B_t=y_t+\sum_{j=1}^q b_j B_{t-j-1}$$ (8)

and

$$A_t(i) = -y_{t-i-1} + \sum_{j=1}^q b_j A_{t-j-1}$$ (9)

where $t-i > 0$ and $t-j >0$.