Gauss-Seidel HALS algorithm for nonnegative matrix factorization with sparseness and smoothness constraints

Takumi Kimura, Norikazu Takahashi

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

Nonnegative Matrix Factorization (NMF) with sparseness and smoothness constraints has attracted increasing attention. When these properties are considered, NMF is usually formulated as an optimization problem in which a linear combination of an approximation error term and some regularization terms must be minimized under the constraint that the factor matrices are nonnegative. In this paper, we focus our attention on the error measure based on the Euclidean distance and propose a new iterative method for solving those optimization problems. The proposed method is based on the Hierarchical Alternating Least Squares (HALS) algorithm developed by Cichocki et al. We first present an example to show that the original HALS algorithm can increase the objective value. We then propose a new algorithm called the Gauss-Seidel HALS algorithm that decreases the objective value monotonically. We also prove that it has the global convergence property in the sense of Zangwill. We finally verify the effectiveness of the proposed algorithm through numerical experiments using synthetic and real data.

Original languageEnglish
Pages (from-to)2925-2935
Number of pages11
JournalIEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences
VolumeE100A
Issue number12
DOIs
Publication statusPublished - Dec 1 2017

Fingerprint

Alternating Least Squares
Non-negative Matrix Factorization
Gauss-Seidel
Least Square Algorithm
Factorization
Smoothness
Optimization Problem
Approximation Error
Euclidean Distance
Error term
Global Convergence
Convergence Properties
Linear Combination
Regularization
Non-negative
Numerical Experiment
Verify
Iteration
Decrease
Iterative methods

Keywords

  • Euclidean distance
  • Global convergence
  • Hierarchical alternating least squares algorithm
  • Nonnegative matrix factorization
  • Smoothness
  • Sparseness

ASJC Scopus subject areas

  • Signal Processing
  • Computer Graphics and Computer-Aided Design
  • Electrical and Electronic Engineering
  • Applied Mathematics

Cite this

@article{bf546538e9994cd2ae9e08b2e44913b4,
title = "Gauss-Seidel HALS algorithm for nonnegative matrix factorization with sparseness and smoothness constraints",
abstract = "Nonnegative Matrix Factorization (NMF) with sparseness and smoothness constraints has attracted increasing attention. When these properties are considered, NMF is usually formulated as an optimization problem in which a linear combination of an approximation error term and some regularization terms must be minimized under the constraint that the factor matrices are nonnegative. In this paper, we focus our attention on the error measure based on the Euclidean distance and propose a new iterative method for solving those optimization problems. The proposed method is based on the Hierarchical Alternating Least Squares (HALS) algorithm developed by Cichocki et al. We first present an example to show that the original HALS algorithm can increase the objective value. We then propose a new algorithm called the Gauss-Seidel HALS algorithm that decreases the objective value monotonically. We also prove that it has the global convergence property in the sense of Zangwill. We finally verify the effectiveness of the proposed algorithm through numerical experiments using synthetic and real data.",
keywords = "Euclidean distance, Global convergence, Hierarchical alternating least squares algorithm, Nonnegative matrix factorization, Smoothness, Sparseness",
author = "Takumi Kimura and Norikazu Takahashi",
year = "2017",
month = "12",
day = "1",
doi = "10.1587/transfun.E100.A.2925",
language = "English",
volume = "E100A",
pages = "2925--2935",
journal = "IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences",
issn = "0916-8508",
publisher = "Maruzen Co., Ltd/Maruzen Kabushikikaisha",
number = "12",

}

TY - JOUR

T1 - Gauss-Seidel HALS algorithm for nonnegative matrix factorization with sparseness and smoothness constraints

AU - Kimura, Takumi

AU - Takahashi, Norikazu

PY - 2017/12/1

Y1 - 2017/12/1

N2 - Nonnegative Matrix Factorization (NMF) with sparseness and smoothness constraints has attracted increasing attention. When these properties are considered, NMF is usually formulated as an optimization problem in which a linear combination of an approximation error term and some regularization terms must be minimized under the constraint that the factor matrices are nonnegative. In this paper, we focus our attention on the error measure based on the Euclidean distance and propose a new iterative method for solving those optimization problems. The proposed method is based on the Hierarchical Alternating Least Squares (HALS) algorithm developed by Cichocki et al. We first present an example to show that the original HALS algorithm can increase the objective value. We then propose a new algorithm called the Gauss-Seidel HALS algorithm that decreases the objective value monotonically. We also prove that it has the global convergence property in the sense of Zangwill. We finally verify the effectiveness of the proposed algorithm through numerical experiments using synthetic and real data.

AB - Nonnegative Matrix Factorization (NMF) with sparseness and smoothness constraints has attracted increasing attention. When these properties are considered, NMF is usually formulated as an optimization problem in which a linear combination of an approximation error term and some regularization terms must be minimized under the constraint that the factor matrices are nonnegative. In this paper, we focus our attention on the error measure based on the Euclidean distance and propose a new iterative method for solving those optimization problems. The proposed method is based on the Hierarchical Alternating Least Squares (HALS) algorithm developed by Cichocki et al. We first present an example to show that the original HALS algorithm can increase the objective value. We then propose a new algorithm called the Gauss-Seidel HALS algorithm that decreases the objective value monotonically. We also prove that it has the global convergence property in the sense of Zangwill. We finally verify the effectiveness of the proposed algorithm through numerical experiments using synthetic and real data.

KW - Euclidean distance

KW - Global convergence

KW - Hierarchical alternating least squares algorithm

KW - Nonnegative matrix factorization

KW - Smoothness

KW - Sparseness

UR - http://www.scopus.com/inward/record.url?scp=85038215741&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85038215741&partnerID=8YFLogxK

U2 - 10.1587/transfun.E100.A.2925

DO - 10.1587/transfun.E100.A.2925

M3 - Article

VL - E100A

SP - 2925

EP - 2935

JO - IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences

JF - IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences

SN - 0916-8508

IS - 12

ER -