Eigenvalues and Scree Plots: Determining the Number of Factors

Eigenvalues in Factor Analysis

Eigenvalues represent the variance explained by each factor in a dataset. In PCA and factor analysis, each factor is associated with an eigenvalue, which quantifies the proportion of total variance that the factor captures. The goal is to reduce dimensionality while preserving as much variance as possible.

Eigenvalues are derived from the covariance or correlation matrix and help assess the importance of each factor. Larger eigenvalues signify more variance explained, making those factors more meaningful.

Kaiser Criterion

The Kaiser criterion is a heuristic that recommends retaining factors with eigenvalues greater than 1. This rule suggests that factors explaining less variance than a single observed variable are not worth retaining.

Although widely used, the Kaiser criterion has limitations, especially in large or small datasets. It is often combined with other methods, such as scree plots, to make more informed decisions.

Scree Plots

A scree plot is a visual tool that displays the eigenvalues of factors in descending order. It helps to identify a point where the eigenvalues begin to level off, known as the "elbow," indicating that subsequent factors explain little variance.

The scree plot helps distinguish between meaningful factors (before the elbow) and noise (after the elbow), simplifying the decision-making process.

Example: Interpreting a Scree Plot

Consider the following data, which summarizes the eigenvalues for five factors:

Factor	Eigenvalue	Variability (%)	Cumulative Variability (%)
F1	3.574	71.472	71.472
F2	0.616	12.329	83.801
F3	0.320	6.399	90.200
F4	0.266	5.312	95.513
F5	0.224	4.487	100.000

Using the Kaiser criterion, only Factor 1 would be retained. This factor explains a significant portion of the variance, as shown by the steep drop in the scree plot after Factor 1.

How to Combine Eigenvalues and Scree Plots

By combining the eigenvalue rule and the scree plot, researchers can make more reliable decisions on the number of factors to retain. The Kaiser criterion identifies factors with eigenvalues greater than 1, while the scree plot confirms the meaningfulness of those factors visually.

In our example, both methods suggest retaining only one factor. However, judgment may be required if the scree plot shows a gradual decline with no clear elbow.

Challenges in Factor Determination

While eigenvalues and scree plots are helpful, there are challenges in determining the right number of factors. Issues include:

• No Clear Elbow in the Scree Plot: Some datasets do not show a distinct elbow, making it harder to decide which factors are meaningful.
• Over-Factoring or Under-Factoring: Retaining too many or too few factors can either introduce noise or oversimplify the data.
• Dataset Size: Larger datasets tend to retain more factors using the Kaiser criterion, while smaller datasets may underrepresent complexity.
• Factor Interpretability: Even if a factor has a high eigenvalue, it must be interpretable and theoretically meaningful.