This article will discuss QR Decomposition in Python. In previous articles we have looked at LU Decomposition in Python and Cholesky Decomposition in Python as two alternative matrix decomposition methods. QR Decomposition is widely used in quantitative finance as the basis for the solution of the linear least squares problem, which itself is used for statistical regression analysis.

1. Qr Householder Image
2. Qr Householder Algorithm
3. Householder Qr Factorization Complete Example
4. Eigen Householder Qr

The Householder Algorithm. Compute the factor R of a QR factorization of m × n matrix A (m ≥ n). Leave result in place of A, store reflection vectors vk for later use Algorithm: Householder QR Factorization for k = 1 to n x = Ak:m,k vk = sign(x1) x 2e1 + x vk = vk/ vk 2 Ak:m,k:n = Ak:m,k:n −2vk(vk ∗A k:m,k:n) 8. A = QR: This factorization can be constructed by three methods: 1. Givens † Property 3.3 (Reduced QR) Suppose the rank of A 2 Rm£n is n for which A = QR is known. Then A = QR where Q and R are submatrices of Q and R given respectively by Q = Q = Q(1: m;1: n); R = R(1: n;1: n): Moreover Q has.

One of the key benefits of using QR Decomposition over other methods for solving linear least squares is that it is more numerically stable, albeit at the expense of being slower to execute. Hence if you are performing a large quantity of regressions as part of a trading backtest, for instance, you will need to consider very extensively whether QR Decomposition is the best fit (excuse the pun).

The Householder reflection method of QR decomposition works by finding appropriate latexH /latex matrices and multiplying them from the left by the original matrix latexA /latex to construct the upper triangular matrix latexR /latex. Def householderreflection (A): 'Perform QR decomposition of matrix A using Householder reflection.' ' (numrows, numcols) = np. Shape (A) # Initialize orthogonal matrix Q and upper triangular matrix R. Identity (numrows) R = np. Copy (A) # Iterative over column sub-vector and # compute Householder matrix to zero-out lower. Typically this approach to QR is performed using Householder or Givens transformations. As discussed in the next section, another method for producing the QR decomposition, and a far more transparent method for producing an orthonormal basis, uses the Gram- Schmidt (G-S) algorithm.

For a square matrix $A$ the QR Decomposition converts $A$ into the product of an orthogonal matrix $Q$ (i.e. $Q^TQ=I$) and an upper triangular matrix $R$. Hence:

begin{eqnarray*} A = QR end{eqnarray*}

There are a few different algorithms for calculating the matrices $Q$ and $R$. We will outline the method of Householder Reflections, which is known to be more numerically stable the the alternative Gramm-Schmidt method. I've outlined the Householder Reflections method below.

Note, the following explanation is an expansion of the extremely detailed article on QR Decomposition using Householder Reflections over at Wikipedia.

A Householder Reflection is a linear transformation that enables a vector to be reflected through a plane or hyperplane. Essentially, we use this method because we want to create an upper triangular matrix, $R$. The householder reflection is able to carry out this vector reflection such that all but one of the coordinates disappears. The matrix $Q$ will be built up as a sequence of matrix multiplications that eliminate each coordinate in turn, up to the rank of the matrix $A$.

The first step is to create the vector $mathbb{x}$, which is the $k$-th column of the matrix $A$, for step $k$. We define $alpha = -sgn(mathbb{x}_k)( mathbb{x} )$. The norm $ cdot $ used here is the Euclidean norm. Given the first column vector of the identity matrix, $I$ of equal size to $A$, $mathbb{e}_1 = (1,0,...,0)^T$, we create the vector $mathbb{u}$:

begin{eqnarray*} mathbb{u} = mathbb{x} + alpha mathbb{e}_1 end{eqnarray*}

Once we have the vector $mathbb{u}$, we need to convert it to a unit vector, which we denote as $mathbb{v}$:

begin{eqnarray*} mathbb{v} = mathbb{u}/ mathbb{u} end{eqnarray*}

Now we form the matrix $Q$ out of the identity matrix $I$ and the vector multiplication of $mathbb{v}$:

begin{eqnarray*} Q = I - 2 mathbb{v} mathbb{v}^T end{eqnarray*}

$Q$ is now an $mtimes m$ Householder matrix, with $Qmathbb{x} = left( alpha, 0, ..., 0right)^T$. We will use $Q$ to transform $A$ to upper triangular form, giving us the matrix $R$. We denote $Q$ as $Q_k$ and, since $k=1$ in this first step, we have $Q_1$ as our first Householder matrix. We multiply this with $A$ to give us:

begin{eqnarray*} Q_1A = begin{bmatrix} alpha_1&star&dots&star 0 & & & vdots & & A' & 0 & & & end{bmatrix} end{eqnarray*}

The whole process is now repeated for the minor matrix $A'$, which will give a second Householder matrix $Q'_2$. Now we have to 'pad out' this minor matrix with elements from the identity matrix such that we can consistently multiply the Householder matrices together. Hence, we define $Q_k$ as the block matrix:

begin{eqnarray*} Q_k = begin{pmatrix} I_{k-1} & 0 0 & Q_k'end{pmatrix} end{eqnarray*}

Once we have carried out $t$ iterations of this process we have $R$ as an upper triangular matrix:

begin{eqnarray*} R = Q_t ... Q_2 Q_1 A end{eqnarray*}

$Q$ is then fully defined as the multiplication of the transposes of each $Q_k$:

begin{eqnarray*} Q = Q^T_1 Q^T_2 ... Q^T_t end{eqnarray*}

This gives $A=QR$, the QR Decomposition of $A$.

To calculate the QR Decomposition of a matrix $A$ with NumPy/SciPy, we can make use of the built-in linalg library via the linalg.qr function. This is significantly more efficient than using a pure Python implementation:

The output of the QR decomposition includes $A$, $Q$ and $R$. As a basic sanity check we can see that $R$ is in fact an upper triangular matrix:

You aren't likely to ever need a pure Python implementation of QR Decomposition (homework notwithstanding), but I feel that it is helpful to gain an understanding of the Householder Reflections algorithm, so I have written my own implementation:

The output from the Householder method implemented in pure Python is given below:

You can see that we get the same answers as above in the SciPy implementation, albeit with a few more significant figures! One has to be extremely careful in numerical algorithms when dealing with floating point arithmetic, but that is a discussion for another day.

Householder Reflections

Householder reflections and QR decomposition

Keywords

array

Usage

Arguments

A

numeric matrix with nrow(A)>=ncol(A).

Qr Householder Image

Details

The Householder method applies a succession of elementary unitary matrices to the left of matrix A. These matrices are the so-called Householder reflections.

Value

List with two matrices Q and R, Q orthonormal and R upper triangular, such that A=Q%*%R.

References

Trefethen, L. N., and D. Bau III. (1997). Numerical Linear Algebra. SIAM, Society for Industrial and Applied Mathematics, Philadelphia.

See Also

Aliases

• householder

Qr Householder Algorithm

Examples

Householder Qr Factorization Complete Example

Documentation reproduced from package pracma, version 1.9.9, License: GPL (>= 3)

Eigen Householder Qr