Title: Notion of information and independent component analysis
Authors: Una Radojicic - Technical University of Vienna (Austria) [presenting]
Klaus Nordhausen - Vienna University of Technology (Austria)
Hannu Oja - University of Turku (Finland)
Abstract: In the engineering literature, independent component analysis is often described as a search for the uncorrelated linear combinations of the original variables that maximize non-Gaussianity. The estimation procedure usually has two steps. First, the vector of principal components is found, and the components are standardized to have zero means and unit variances. Second, the vector is further rotated so that the new components maximize a selected measure of non-Gaussianity. It is then argued that the components obtained in this way are made as independent as possible or that they display the components with maximal information. The information measures and measures of non-Gaussianity, including third and fourth cumulants are generally used as projection indices in the projection pursuit approach for the independent component analysis. We discuss and clarify the vague connections between non-Gaussianity, independence and notions of information in the context of the independent component analysis by discussing partial orderings and various measures of information for continuous univariate random variables with special roles of Gaussian and uniform distributions.