Title: Crucial differences between principal component and factor analyses solutions elucidated by some inequalities
Authors: Kohei Adachi - Osaka University (Japan) [presenting]
Nickolay Trendafilov - Open University (United Kingdom)
Abstract: Some inequalities are presented to unmask the differences between the principal component analysis (PCA) and factor analysis (FA) solutions for the same data set. For this reason, we take advantage of the matrix decomposition (MD) formulation of FA established recently. In summary, the resulting inequalities show that  FA provides a better fit to the data than PCA,  PCA extracts a larger amount of common information than FA, and  For each variable, its unique variance in FA is larger than its residual variance in PCA minus the one in FA. The resulting inequalities can be useful to suggest whether PCA or FA should be used for a particular data set. The answers can also be valid for the classic random FA not relying on the MD-FA formulation, as both types of FA are found to provide almost equal solutions. Additionally, the inequalities give theoretical explanation of some empirically observed tendencies in PCA and FA solutions, e.g., that the absolute values of PCA loadings tend to be larger than those for FA loadings, and that the unique variances in FA tend to be larger than the residual variances of PCA.