Skip to main content
\(\newcommand{\Cs}{\mathbb{Cs}} \newcommand{\Cm}{\mathbb{C}^{m}} \newcommand{\Cmm}{\mathbb{C}^{m\times m}} \newcommand{\Cmn}{\mathbb{C}^{m\times n}} \newcommand{\R}{\mathbb{R}} \newcommand{\Rm}{\mathbb{R}^{m}} \newcommand{\Rn}{\mathbb{R}^{n}} \newcommand{\Rmm}{\mathbb{R}^{m\times m}} \DeclareMathOperator{\Range}{range} \DeclareMathOperator{\Rank}{rank} \DeclareMathOperator{\diag}{diag} \DeclareMathOperator{\Span}{span} \DeclareMathOperator{\range}{range} \DeclareMathOperator{\Null}{null} \newcommand{\definiteintegral}[4]{\int_{#1}^{#2}\,#3\,d#4} \newcommand{\indefiniteintegral}[2]{\int#1\,d#2} \graphicspath{ {images/} } \usepackage{amsmath} \newcommand{\lt}{<} \newcommand{\gt}{>} \newcommand{\amp}{&} \)

Section13Introduction to the Singular Value Decomposition (NM, 2 Oct)

Subsection13.1Singular Values

The singular values \(\{\sigma_1, \sigma_2, \dots, \sigma_m\}\) of a matrix \(A\) can be defined in several ways:

  • the square roots of the eigenvalues of \(B=A^\star A\text{,}\)
  • the lengths of the semiaxes of the hyperellipse that is the image of the unit circle under the linear transformation defined by \(A\text{.}\)
  • The diagonal entries of the diagonal matrix \(\Sigma\) when we write \(A\) has a factorisation \(A=U \Sigma V^\star\) where $U$ and $V$ are unitary matrices, and $\Sigma$ is a diagonal matrix. (We'll be a little more precise below).

Please read Lecture 4 of Trefethen and Bau. T+B make the important observation in their introduction: the image of the unit sphere under any \(m \times n\) matrix is a hyperellipse.

Suppose that \(\{u_i\}_{i=1}^m\) is a set of orthonormal vectors, such that principal semiaxes of the hyperellipse are \(\{\sigma_i u_i\}\text{.}\) These scalars \(\sigma_1, \sigma_2, \dots, \sigma_m\) are the singular values of \(A\), and encapsulate many important properties of \(A\text{.}\)

Subsection13.2Singular Value Decomposition

The rest of the exposition followed Lecture 4 of Trefethen and Bau pretty closely. In summary, the SVD of \(A \in \Cmn\) is

\begin{equation*} AV = \Sigma U, \end{equation*}


  • \(U=(u_1 | u_2 | u_3 | \dots | u_m) \text{,}\) the matrix that has as its columns the orthoronormal vectors in the direction of the principal semiaxes of the image of the unit sphere \(S \in \Rn \text{.}\)
  • \(\Sigma = \diag(\sigma_1, \sigma_2, \dots, \sigma_n)\) is the diagonal matrix containing the lengths of the principal semiaxes. We order them as \begin{equation*} \sigma_1 \geq \sigma_2 \geq \cdots \geq \sigma_n \geq 0. \end{equation*}
  • Then \(V=(v_1 | v_2 | v_3 | \dots | v_n)\) is the matrix whose columns are the preimages of the principal semiaxes, i.e., \(A v_i = \sigma_i u_i.\) It should be clear that the \(\{v_i\}\) form an orthonormal set, so \(V^{-1}=V^\star.\)



Prove that the product of two unitary matrices in \(\Cmm\) is unitary.


Write down the SVDs of the following matrices.

  1. \(\begin{pmatrix} 4 \amp 0 \\ 0 \amp -2 \end{pmatrix}\)
  2. \(\begin{pmatrix} 0 \amp 4 \\ -2 \amp 0 \end{pmatrix}\)
  3. \(\begin{pmatrix} 4 \amp -2 \\ 0 \amp 0 \end{pmatrix}\)

The matrices \(A\) and \(B\) in \(\Cmm\)are unitarily equivalent if there exists a untiary matrix, \(Q \in \Cmm\) such that \(A=QBQ^\star\text{.}\)

  1. Show that, if \(A\) and \(B\) are unitarily equivalent, then they have the same singular values.
  2. Suppose that \(A, B \in \Cmm\) have the same singular values. Must they be unitarily equivalent?