Sparse estimation via nonconcave penalized likelihood in factor analysis model

Kei Hirose, Michio Yamamoto

Research output: Contribution to journalArticle

17 Citations (Scopus)

Abstract

We consider the problem of sparse estimation in a factor analysis model. A traditional estimation procedure in use is the following two-step approach: the model is estimated by maximum likelihood method and then a rotation technique is utilized to find sparse factor loadings. However, the maximum likelihood estimates cannot be obtained when the number of variables is much larger than the number of observations. Furthermore, even if the maximum likelihood estimates are available, the rotation technique does not often produce a sufficiently sparse solution. In order to handle these problems, this paper introduces a penalized likelihood procedure that imposes a nonconvex penalty on the factor loadings. We show that the penalized likelihood procedure can be viewed as a generalization of the traditional two-step approach, and the proposed methodology can produce sparser solutions than the rotation technique. A new algorithm via the EM algorithm along with coordinate descent is introduced to compute the entire solution path, which permits the application to a wide variety of convex and nonconvex penalties. Monte Carlo simulations are conducted to investigate the performance of our modeling strategy. A real data example is also given to illustrate our procedure.

Original languageEnglish
Pages (from-to)863-875
Number of pages13
JournalStatistics and Computing
Volume25
Issue number5
DOIs
Publication statusPublished - May 28 2014
Externally publishedYes

Fingerprint

Penalized Likelihood
Factor analysis
Factor Analysis
Maximum likelihood
Maximum Likelihood Estimate
Penalty
Coordinate Descent
Entire Solution
Maximum Likelihood Method
EM Algorithm
Model
Monte Carlo Simulation
Path
Methodology
Modeling
Factor loadings

Keywords

  • Coordinate descent algorithm
  • Factor analysis
  • Nonconvex penalty
  • Penalized likelihood
  • Rotation technique

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Statistics and Probability
  • Statistics, Probability and Uncertainty
  • Computational Theory and Mathematics

Cite this

Sparse estimation via nonconcave penalized likelihood in factor analysis model. / Hirose, Kei; Yamamoto, Michio.

In: Statistics and Computing, Vol. 25, No. 5, 28.05.2014, p. 863-875.

Research output: Contribution to journalArticle

@article{71b5958485e94027ac410df8bf4958c1,
title = "Sparse estimation via nonconcave penalized likelihood in factor analysis model",
abstract = "We consider the problem of sparse estimation in a factor analysis model. A traditional estimation procedure in use is the following two-step approach: the model is estimated by maximum likelihood method and then a rotation technique is utilized to find sparse factor loadings. However, the maximum likelihood estimates cannot be obtained when the number of variables is much larger than the number of observations. Furthermore, even if the maximum likelihood estimates are available, the rotation technique does not often produce a sufficiently sparse solution. In order to handle these problems, this paper introduces a penalized likelihood procedure that imposes a nonconvex penalty on the factor loadings. We show that the penalized likelihood procedure can be viewed as a generalization of the traditional two-step approach, and the proposed methodology can produce sparser solutions than the rotation technique. A new algorithm via the EM algorithm along with coordinate descent is introduced to compute the entire solution path, which permits the application to a wide variety of convex and nonconvex penalties. Monte Carlo simulations are conducted to investigate the performance of our modeling strategy. A real data example is also given to illustrate our procedure.",
keywords = "Coordinate descent algorithm, Factor analysis, Nonconvex penalty, Penalized likelihood, Rotation technique",
author = "Kei Hirose and Michio Yamamoto",
year = "2014",
month = "5",
day = "28",
doi = "10.1007/s11222-014-9458-0",
language = "English",
volume = "25",
pages = "863--875",
journal = "Statistics and Computing",
issn = "0960-3174",
publisher = "Springer Netherlands",
number = "5",

}

TY - JOUR

T1 - Sparse estimation via nonconcave penalized likelihood in factor analysis model

AU - Hirose, Kei

AU - Yamamoto, Michio

PY - 2014/5/28

Y1 - 2014/5/28

N2 - We consider the problem of sparse estimation in a factor analysis model. A traditional estimation procedure in use is the following two-step approach: the model is estimated by maximum likelihood method and then a rotation technique is utilized to find sparse factor loadings. However, the maximum likelihood estimates cannot be obtained when the number of variables is much larger than the number of observations. Furthermore, even if the maximum likelihood estimates are available, the rotation technique does not often produce a sufficiently sparse solution. In order to handle these problems, this paper introduces a penalized likelihood procedure that imposes a nonconvex penalty on the factor loadings. We show that the penalized likelihood procedure can be viewed as a generalization of the traditional two-step approach, and the proposed methodology can produce sparser solutions than the rotation technique. A new algorithm via the EM algorithm along with coordinate descent is introduced to compute the entire solution path, which permits the application to a wide variety of convex and nonconvex penalties. Monte Carlo simulations are conducted to investigate the performance of our modeling strategy. A real data example is also given to illustrate our procedure.

AB - We consider the problem of sparse estimation in a factor analysis model. A traditional estimation procedure in use is the following two-step approach: the model is estimated by maximum likelihood method and then a rotation technique is utilized to find sparse factor loadings. However, the maximum likelihood estimates cannot be obtained when the number of variables is much larger than the number of observations. Furthermore, even if the maximum likelihood estimates are available, the rotation technique does not often produce a sufficiently sparse solution. In order to handle these problems, this paper introduces a penalized likelihood procedure that imposes a nonconvex penalty on the factor loadings. We show that the penalized likelihood procedure can be viewed as a generalization of the traditional two-step approach, and the proposed methodology can produce sparser solutions than the rotation technique. A new algorithm via the EM algorithm along with coordinate descent is introduced to compute the entire solution path, which permits the application to a wide variety of convex and nonconvex penalties. Monte Carlo simulations are conducted to investigate the performance of our modeling strategy. A real data example is also given to illustrate our procedure.

KW - Coordinate descent algorithm

KW - Factor analysis

KW - Nonconvex penalty

KW - Penalized likelihood

KW - Rotation technique

UR - http://www.scopus.com/inward/record.url?scp=84938416978&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84938416978&partnerID=8YFLogxK

U2 - 10.1007/s11222-014-9458-0

DO - 10.1007/s11222-014-9458-0

M3 - Article

AN - SCOPUS:84938416978

VL - 25

SP - 863

EP - 875

JO - Statistics and Computing

JF - Statistics and Computing

SN - 0960-3174

IS - 5

ER -