Speeding up the convergence of the alternating least squares algorithm using vector ε acceleration and restarting for nonlinear principal component analysis

Principal component analysis (PCA) is a widely used descriptive multivariate technique in the analysis of quantitative data. When applying PCA to mixed quantitative and qualitative data, we utilize an optimal scaling technique for quantifying qualitative data. PCA with optimal scaling is called nonlinear PCA. The alternating least squares (ALS) algorithm is used for computing nonlinear PCA. The ALS algorithm is stable in convergence and simple in implementation; however, the algorithm tends to converge slowly for large data matrices owing to its linear convergence. Then the vε-ALS algorithm, which incorporates the vector ε accelerator into the ALS algorithm, is used to accelerate the convergence of the ALS algorithm for nonlinear PCA. In this paper, we improve the vε-ALS algorithm via a restarting procedure and further reduce its number of iterations and computation time. The restarting procedure is performed under simple restarting conditions, and it speeds up the convergence of the vε-ALS algorithm. The vε-ALS algorithm with a restarting procedure is referred to as the vεR-ALS algorithm. Numerical experiments examine the performance of the vεR-ALS algorithm by comparing its number of iterations and computation time with those of the ALS and vε-ALS algorithms.