A unified global convergence analysis of multiplicative update rules for nonnegative matrix factorization

Norikazu Takahashi, Jiro Katayama, Masato Seki, Jun’ichi Takeuchi

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

Multiplicative update rules are a well-known computational method for nonnegative matrix factorization. Depending on the error measure between two matrices, various types of multiplicative update rules have been proposed so far. However, their convergence properties are not fully understood. This paper provides a sufficient condition for a general multiplicative update rule to have the global convergence property in the sense that any sequence of solutions has at least one convergent subsequence and the limit of any convergent subsequence is a stationary point of the optimization problem. Using this condition, it is proved that many of the existing multiplicative update rules have the global convergence property if they are modified slightly so that all variables take positive values. This paper also proposes new multiplicative update rules based on Kullback–Leibler, Gamma, and Rényi divergences. It is shown that these three rules have the global convergence property if the same modification as above is made.

Original languageEnglish
Pages (from-to)1-30
Number of pages30
JournalComputational Optimization and Applications
DOIs
Publication statusAccepted/In press - Mar 15 2018

Fingerprint

Non-negative Matrix Factorization
Global Analysis
Factorization
Convergence Analysis
Global Convergence
Multiplicative
Update
Convergence Properties
Computational methods
Subsequence
Stationary point
Computational Methods
Divergence
Optimization Problem
Sufficient Conditions

Keywords

  • Global convergence
  • Multiplicative update rule
  • Nonnegative matrix factorization

ASJC Scopus subject areas

  • Control and Optimization
  • Computational Mathematics
  • Applied Mathematics

Cite this

A unified global convergence analysis of multiplicative update rules for nonnegative matrix factorization. / Takahashi, Norikazu; Katayama, Jiro; Seki, Masato; Takeuchi, Jun’ichi.

In: Computational Optimization and Applications, 15.03.2018, p. 1-30.

Research output: Contribution to journalArticle

@article{698542c3e178473892129307285ac4f8,
title = "A unified global convergence analysis of multiplicative update rules for nonnegative matrix factorization",
abstract = "Multiplicative update rules are a well-known computational method for nonnegative matrix factorization. Depending on the error measure between two matrices, various types of multiplicative update rules have been proposed so far. However, their convergence properties are not fully understood. This paper provides a sufficient condition for a general multiplicative update rule to have the global convergence property in the sense that any sequence of solutions has at least one convergent subsequence and the limit of any convergent subsequence is a stationary point of the optimization problem. Using this condition, it is proved that many of the existing multiplicative update rules have the global convergence property if they are modified slightly so that all variables take positive values. This paper also proposes new multiplicative update rules based on Kullback–Leibler, Gamma, and R{\'e}nyi divergences. It is shown that these three rules have the global convergence property if the same modification as above is made.",
keywords = "Global convergence, Multiplicative update rule, Nonnegative matrix factorization",
author = "Norikazu Takahashi and Jiro Katayama and Masato Seki and Jun’ichi Takeuchi",
year = "2018",
month = "3",
day = "15",
doi = "10.1007/s10589-018-9997-y",
language = "English",
pages = "1--30",
journal = "Computational Optimization and Applications",
issn = "0926-6003",
publisher = "Springer Netherlands",

}

TY - JOUR

T1 - A unified global convergence analysis of multiplicative update rules for nonnegative matrix factorization

AU - Takahashi, Norikazu

AU - Katayama, Jiro

AU - Seki, Masato

AU - Takeuchi, Jun’ichi

PY - 2018/3/15

Y1 - 2018/3/15

N2 - Multiplicative update rules are a well-known computational method for nonnegative matrix factorization. Depending on the error measure between two matrices, various types of multiplicative update rules have been proposed so far. However, their convergence properties are not fully understood. This paper provides a sufficient condition for a general multiplicative update rule to have the global convergence property in the sense that any sequence of solutions has at least one convergent subsequence and the limit of any convergent subsequence is a stationary point of the optimization problem. Using this condition, it is proved that many of the existing multiplicative update rules have the global convergence property if they are modified slightly so that all variables take positive values. This paper also proposes new multiplicative update rules based on Kullback–Leibler, Gamma, and Rényi divergences. It is shown that these three rules have the global convergence property if the same modification as above is made.

AB - Multiplicative update rules are a well-known computational method for nonnegative matrix factorization. Depending on the error measure between two matrices, various types of multiplicative update rules have been proposed so far. However, their convergence properties are not fully understood. This paper provides a sufficient condition for a general multiplicative update rule to have the global convergence property in the sense that any sequence of solutions has at least one convergent subsequence and the limit of any convergent subsequence is a stationary point of the optimization problem. Using this condition, it is proved that many of the existing multiplicative update rules have the global convergence property if they are modified slightly so that all variables take positive values. This paper also proposes new multiplicative update rules based on Kullback–Leibler, Gamma, and Rényi divergences. It is shown that these three rules have the global convergence property if the same modification as above is made.

KW - Global convergence

KW - Multiplicative update rule

KW - Nonnegative matrix factorization

UR - http://www.scopus.com/inward/record.url?scp=85044089143&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85044089143&partnerID=8YFLogxK

U2 - 10.1007/s10589-018-9997-y

DO - 10.1007/s10589-018-9997-y

M3 - Article

AN - SCOPUS:85044089143

SP - 1

EP - 30

JO - Computational Optimization and Applications

JF - Computational Optimization and Applications

SN - 0926-6003

ER -