Speeding up the convergence of the alternating least squares algorithm using vector ε acceleration and restarting for nonlinear principal component analysis

Masahiro Kuroda, Yuichi Mori, Masaya Iizuka

Research output: Contribution to journalArticlepeer-review

Abstract

Principal component analysis (PCA) is a widely used descriptive multivariate technique in the analysis of quantitative data. When applying PCA to mixed quantitative and qualitative data, we utilize an optimal scaling technique for quantifying qualitative data. PCA with optimal scaling is called nonlinear PCA. The alternating least squares (ALS) algorithm is used for computing nonlinear PCA. The ALS algorithm is stable in convergence and simple in implementation; however, the algorithm tends to converge slowly for large data matrices owing to its linear convergence. Then the vε-ALS algorithm, which incorporates the vector ε accelerator into the ALS algorithm, is used to accelerate the convergence of the ALS algorithm for nonlinear PCA. In this paper, we improve the vε-ALS algorithm via a restarting procedure and further reduce its number of iterations and computation time. The restarting procedure is performed under simple restarting conditions, and it speeds up the convergence of the vε-ALS algorithm. The vε-ALS algorithm with a restarting procedure is referred to as the vεR-ALS algorithm. Numerical experiments examine the performance of the vεR-ALS algorithm by comparing its number of iterations and computation time with those of the ALS and vε-ALS algorithms.

Original languageEnglish
JournalComputational Statistics
DOIs
Publication statusAccepted/In press - 2022

Keywords

  • Acceleration of convergence
  • Alternating least squares algorithm
  • Nonlinear principal component analysis
  • Restarting procedure
  • Vector ε algorithm

ASJC Scopus subject areas

  • Statistics and Probability
  • Statistics, Probability and Uncertainty
  • Computational Mathematics

Fingerprint

Dive into the research topics of 'Speeding up the convergence of the alternating least squares algorithm using vector ε acceleration and restarting for nonlinear principal component analysis'. Together they form a unique fingerprint.

Cite this