This is the first in a short series of posts on an application of matrix theory to the mechanics of continuous media. I’m planning my Vector and Tensor Analysis course for the winter quarter, and wanted to include a discussion of stress in elastic media as an application of tensors interesting to the engineering students. Identifying the prerequisite material needed for this topic and to what extent I can expect the students to have been exposed to it through our curriculum has been an interesting exercise. The physical interpretation of the stress tensor (which I will define in a later post) comes through the real polar decomposition of a matrix. The real polar decomposition can be deduced from the singular value decomposition which can be deduced from the real spectral theorem. Here’s how that last step goes.
Theorem. (Singular Value Decomposition) Every 
real matrix 
can be factored as 
where 
is an $m\times m$ orthogonal matrix, 
is an matrix with non-negative real numbers on the diagonal (called the singular values of ) and 
is an 
orthogonal matrix.
With an understanding of the spectral theorem and how to interpret matrix multiplication one can actually derive a procedure for computing this factorization. The important things to remember are:
- if
is an matrix then 
is a symmetric matrix and 
is a symmetric $m\times m$ matrix because of the two properties of the transpose that 
and 
;
- what it means for a matrix
to be orthogonal is that is square and 
, i.e., is invertible and 
;
- the condition
also says that the columns of are orthonormal vectors since the 
-entry of the product 
is precisely the dot product of row 
of 
(i.e., column of ) with column 
of ;
- if
![D=[d_{ij}]](http://s0.wp.com/latex.php?latex=D%3D%5Bd_%7Bij%7D%5D&bg=ffffff&fg=4e4e4e&s=0 "D=[d_{ij}]")
is a diagonal matrix of size 
then 
is the result of multiplying the rows of by the corresponding diagonal entries of 
, while if is diagonal of size $n\times n$ then 
is the result of multiplying the columns of by the corresponding entries of .
Let’s suppose for a moment that with , , and as in the statement of the theorem. Then
and
by the orthogonality of and . Due to the structure of , the matrix 
is an diagonal matrix whose first 
diagonal entries are the squares of the 
diagonal entries of . Likewise, the matrix 
is an matrix whose first diagonal entries are the squares of the diagonal entries of (in the same order). The matrix equation 
then says that the columns of are eigenvectors of 
. Similarly, the equation 
then says that the columns of are eigenvectors of 
.
Fortunately, the real spectral theorem guarantees that the eigenvectors of the symmetric matrices and are orthogonal, and thus can be chosen to be orthonormal. If 
is an normalized eigenvector of with associated eigenvalue 
(i.e., 
) then 
being the dot product of a vector with itself. Similarly, the eigenvalues of 
are non-negative. So it is certainly possible that the non-negative diagonal entries of and are the eigenvalues of and , respectively. But, it must be the case that the 
eigenvalue of is equal to the eigenvalue of for 
in order for our factorization to work since both must be equal to the square of the diagonal entry of . This is where a clever observation saves the day. The matrix converts eigenvectors of into eigenvectors of ! Indeed, if 
is an eigenvector of with eigenvalue , then
Hence, if 
then it is an eigenvalue of with the same eigenvalue ! Of course, if 
but is non-zero (being an eigenvector of ) then 
and so the associated eigenvalue of is zero. Likewise, if is an eigenvector of and 
is non-zero then is an eigenvector of with the same eigenvalue. It follows that at most of the eigenvalues of and are nonzero and, in fact, of them are the same. The square roots of these joint eigenvalues of and are called the singular values of .
The upshot is that we can compute a set of normalized eigenvectors for and order them in descending order of their associated eigenvalues (so that those in the kernel occur at the end) and use them to form the columns of the orthogonal matrix . We can compute a set of normalized eigenvectors for and order them in descending order of their associated eigenvalues to form the columns of the orthogonal matrix . Then, if is the matrix whose non-diagonal entries are zero but with diagonal entries equal to the singular values of (also listed in descending order) we are guaranteed that . This constructs the singular value decomposition of .
Like this:
Like Loading...
A quick couple of things you probably already know:
1) there are some java demos out there illustrating svd used to do image compression. Nice eye-candy!
2) also, svd gives a great way to visualize linear transformations from R^n to R^n–i.e. the vectors in some priveleged o.n. basis are first scaled individually (the ones scaled by 0 span the kernel) and then the resulting orthogonal set is mapped isometrically into the target (i.e. by rotation/reflection if target = source)
Finally, a couple of my honors alg students last year took on the project of programming svd from scratch. The business of finding evals/evecs of a largish symmetric matrix was pretty enlightening.
-Jeff