A data scientist performs principal component analysis (PCA) on an n-dimensional data set that has been centered and each feature scaled to unit variance. Let Σ be the resulting n × n sample covariance matrix. Which of the following statements about the eigenvalues λᵢ and eigenvectors vᵢ of Σ is true?
If Σ has repeated eigenvalues, its eigenvectors cannot form an orthonormal basis for ℝⁿ.
The sum of all eigenvalues of Σ equals the total variance present in the standardized data set.
The eigenvectors associated with the largest eigenvalues represent directions of minimum variance in the data.
The eigenvectors of Σ are orthogonal only when Σ is non-symmetric; otherwise they need not be orthogonal.
The trace of a square matrix is the sum of the elements on its main diagonal, and this sum is always equal to the sum of the matrix's eigenvalues. For a covariance matrix, the diagonal entries represent the variance of each corresponding feature. In this scenario, the data has been standardized, which means each feature has a variance of 1. Therefore, for the n x n covariance matrix Σ, each of the n diagonal entries is 1, and its trace equals n. This trace represents the total variance in the standardized dataset. Since the trace equals the sum of the eigenvalues, the sum of all eigenvalues of Σ equals the total variance present in the standardized data set.
The remaining statements are incorrect:
The principal components, which are the eigenvectors of the covariance matrix, are ordered by the amount of variance they explain. The eigenvector corresponding to the largest eigenvalue points in the direction of maximum variance, not minimum.
A sample covariance matrix is, by definition, a real and symmetric matrix. The spectral theorem guarantees that for any real symmetric matrix, there exists an orthonormal basis of its eigenvectors. This means the eigenvectors can always be chosen to be orthogonal.
The existence of an orthonormal basis of eigenvectors also holds true if the matrix has repeated eigenvalues. For an eigenspace corresponding to a repeated eigenvalue, it is always possible to construct an orthogonal basis.
Ask Bash
Bash is our AI bot, trained to help you pass your exam. AI Generated Content may display inaccurate information, always double-check anything important.
What is the trace of a matrix and why does it equal the sum of eigenvalues?
Open an interactive chat with Bash
Why do the eigenvectors of a covariance matrix represent principal components?
Open an interactive chat with Bash
What does it mean for eigenvectors to form an orthonormal basis?