Section23Low Rank Approximations (NM, 24 Oct)¶ permalink

First we proved the following result.

Theorem23.1

\(A\) is the sum of the \(r\) rank-one matrices

\begin{equation*} A = \sum_{j=1}^r \sigma_j u_j v_j^\star. \end{equation*}

Then we moved on to the following, highly important theorem.

Theorem23.2

Let \(A_V\) be the rank-\(v\) approximation to \(A\)

\begin{equation*} A_v = \sum_{j=1}^r \sigma_j u_j v_j^\star. \end{equation*}

Then

\begin{equation*} \|A- A_v\|_2 = \inf_{\Rank(X)\leq v}\|A-X\|_2 = \sigma_{v+1}, \end{equation*}

where if \(v=p=\min(m,n)\text{,}\) and we define \(\sigma_{v+1}=0\text{.}\)

A similar result is true for the \(\| \cdot\|_F\) norm, the proof of which is left to an exercise.

Theorem23.3

\begin{equation*} \|A- A_v\|_F = \inf_{\Rank(X)\leq v}\|A-X\|_F = \sqrt{\sigma_{v+1}^2+\sigma_{v+1}^2+\cdots+\sigma_r^2}. \end{equation*}

All the above results emphasise the power the power of the SVD as an important theoretical and computational tool. We demonstrated this in class by using the SVD to compute “approximations” of certain images.

Computing the SVD is an important task, but not a trivial one. There are a few different approaches, but a key idea is the \(QR\)-factorisation.

Subsection23.1Exercises

Exercise23.4

Prove Theorem 23.3.

During this section of the course, we occasionally made use of arguments based on partitioning matrices. Use this idea to prove the following theorem.

Exercise23.5

Suppose that onc can partition the matrix \(A\) as

\begin{equation*} A = \begin{pmatrix} A_{11} \amp A_{12} \\ A_{21} \amp A_{22}, \end{pmatrix} \end{equation*}

where \(A\) and \(A_{11}\) are nonsingular. Let \(S=A_{22} - A_{21}A_{11}^{-1}A_{12}\text{.}\) Show that

\begin{equation*} A^{-1} = \begin{pmatrix} A_11^{-1} + A_{11}^{-1}A_{12}S^{-1}A_{21}A_{11}^-1 \amp -A_{11}^{-1}A_{12}S^{-1}\\ -S^{-1}A_{21}A_{11}^{-1} \amp S^{-1}. \end{pmatrix} \end{equation*}

Exercise23.6

Recall the definition of a lower triangular and unit lower triangular matrices in Exercise 2.6. Also, an upper triangular matrix is one whose transpose is lower triangular. Let \(L_1\) and \(L_2\) be unit lower triangular matrices. Let \(U_1\) and \(U_2\) be nonsingular upper triangular matrices. Suppose that \(L_1U_1=L_2U_2\text{.}\) Show that \(L_1=L_2\) and \(U_1=U_2\text{.}\)