Optimization of AR parameters

Denote

$$S(a,b) =\sum_{t=1}^T \epsilon^2,$$ (10)

where $a=(a_1,...,a_p)$ and $b=(b_1,...,b_q)$.

From expressions (1.11) and (1.8) the minimum condition is

$${\partial S(a,b) \over \partial a_j} = 2 \sum_{t=1}^T \epsilon_t A_t(j)=0,\ j=1,...,p$$ (11)

or

$$\sum_{i=1}^p A(i,j) a_i = -B(j),\ j=1,...,p,$$ (12)

where

$$A(i,j) =\sum_{t=1}^T A_t(i) A_t(j)$$ (13)

and

$$B(j) =\sum_{t=1}^T A_t(j) B_t.$$ (14)

The minimum of expression (1.11) at fixed parameters $b$ is defined by a system of linear equations:

$$a(b)= A^{-1} B.$$ (15)

Here matrix $A=(A(i,j),\ i,j=1,...p)$ and vector $B=(B(j), \ j=1,...p)$, where elements $A(i,j)$ are from (1.14), components $B(j)$ are from (1.15), and $A^{-1}$ is an inverse matrix $A$. This way one define the vector $a(b)=(a_i(b),\ i=1,...,p)$ that minimize sum (1.11) at fixed parameters $b$.