Recursive Least Squares

The corresponding convergence rate in the Recursive Least Squares (RLS) algorithm is faster, but the implementation is more complex than that of LMS-based algorithms.

The (RLS algorithm uses the following equations to modify the cost function J(k) = E[e²(k)].

Compare this modified cost function, which uses the previous N error terms, to the cost function, J(k) =  E[e²(k)], which uses only the current error information e(k). The modified cost function J(k) is more robust.

The following procedure describes how to implement the RLS algorithm:

1. Initialize the parametric vector ⃗w(k) using a small positive number ε.
   $\stackrel{\rightharpoonup }{w}\left(0\right)={\left(\epsilon , \epsilon ,..., \epsilon \right)}^T$

2. Initialize the data vector ⃗φ(k).
3. Initialize the k × k matrix P(0).
   $P\left(0\right)=\left(\begin{array}{cccc}\epsilon & 0& 0& 0\\ 0& \epsilon & 0& 0\\ 0& 0& ...& 0\\ 0& 0& 0& \epsilon \end{array}\right)$

4. For k = 1, update the data vector ⃗φ(k) based on ⃗φ(k-1) and the current input data u(k) and output data y(k).
5. Compute the predicted response ŷ(k) by using the following equation.
   $\hat{y}\left(k\right)={\stackrel{\rightharpoonup }{\phi }}^T\left(k\right)·\stackrel{\rightharpoonup }{w}\left(k\right)$

6. Compute the error e(k) by solving the following equation:
   $e\left(k\right)=y\left(k\right)-\hat{y}\left(k\right)$

7. Update the gain vector ⃗k(k) defined by the following equation:
   $\stackrel{\rightharpoonup }{K}\left(k\right)=\frac{P\left(k\right)·\stackrel{\rightharpoonup }{\phi }\left(k\right)}{\lambda +{\stackrel{\rightharpoonup }{\phi }}^T·P\left(k\right)·\stackrel{\rightharpoonup }{\phi }\left(k\right)}$
   The properties of a system might vary with time, so you must ensure that the algorithm tracks the variations. You can use the forgetting factor λ, which is an adjustable parameter, to track these variations. The smaller the forgetting factor λ, the less previous information this algorithm uses. When you use small forgetting factors, the adaptive filter is able to track time-varying systems that vary rapidly. The range of the forgetting factor λ is between zero and one, typically 0.98 < λ < 1. P(k) is a k × k matrix whose initial value is defined by P(0) in step 3.
8. Update the parametric vector
   $\stackrel{\rightharpoonup }{w}\left(k+1\right)$
   .
   $\stackrel{\rightharpoonup }{w}\left(k+1\right)=\stackrel{\rightharpoonup }{w}\left(k\right)+e\left(k\right)·\stackrel{\rightharpoonup }{K}\left(k\right)$

9. Update the P(k) matrix.
   $P\left(k+1\right)={\lambda }^{-1}P\left(k\right)-{\lambda }^{-1}\stackrel{\rightharpoonup }{K}\left(k\right)·{\stackrel{\rightharpoonup }{\phi }}^T\left(k\right)·P\left(k\right)$

10. Stop if the error is small enough, else set k = k + 1 and repeat steps 4–10.