A data-science team is fitting an ordinary least-squares model on a design matrix X with thousands of features. Directly computing ((X^\top X){-1}X\top y) sometimes throws a "matrix is singular or nearly singular" exception. To make the pipeline robust they want a matrix factorization that
lets them form the Moore-Penrose pseudoinverse of X in a numerically stable way,
exposes very small components so they can be truncated or Tikhonov-regularized, and
immediately reveals the effective rank of X.
Which factorization best satisfies all three requirements?
Singular value decomposition (SVD) writes X = UΣVᵀ, where Σ is diagonal. The non-zero entries of Σ are the singular values; their count is the rank of X. Very small σᵢ identify directions that can be dropped or ridge-regularized, and the Moore-Penrose pseudoinverse follows directly as VΣ⁻¹Uᵀ after setting tiny σᵢ to zero. Because no squaring of the condition number is required, SVD stays stable even when X is nearly singular.
LU with pivoting solves linear systems efficiently but does not expose singular values or provide an immediate pseudoinverse. Cholesky decomposition can only be applied to symmetric positive-definite matrices; if X is rank-deficient, XᵀX is not positive-definite and Cholesky fails. Eigen-decomposition of XᵀX does show eigenvalues, but forming XᵀX magnifies numerical problems and still requires extra steps to build a pseudoinverse.
Therefore, SVD is the only decomposition that meets all three criteria.
Ask Bash
Bash is our AI bot, trained to help you pass your exam. AI Generated Content may display inaccurate information, always double-check anything important.
What is the Moore-Penrose pseudoinverse and why is it important?
Open an interactive chat with Bash
Why is SVD numerically stable for nearly singular matrices?
Open an interactive chat with Bash
What is the difference between SVD and eigen-decomposition?