How can we overcome the limitations of traditional Principal Component Analysis (PCA)? This paper introduces a mixture model approach to Principal Component Analysis, addressing the limitations imposed by PCA's global linearity. While nonlinear PCA variants exist, this research explores capturing data complexity through a combination of local linear PCA projections. The main purpose is to introduce PCA within a maximum likelihood framework. PCA is formulated within a maximum likelihood framework, using a gaussian latent variable model. This framework leads to a well-defined mixture model for probabilistic principal component analyzers, whose parameters can be determined using an expectation-maximization algorithm. The use of an expectation-maximization algorithm helps in calculating the parameters. The model's advantages are discussed in the context of clustering, density modeling, and local dimensionality reduction. The mixture model's utility is demonstrated through applications in image compression and handwritten digit recognition. This approach offers a powerful tool for capturing data complexity and improving the performance of PCA in various applications.
Published in Neural Computation, this paper focuses on a core area of research in neural networks and machine learning: dimensionality reduction. The development of a mixture model for PCA is a significant contribution to the field, aligning with the journal's interest in novel computational techniques for data analysis. This approach is appropriate for the journal's theoretical and applied focus.