MATLAB library for nonnegative matrix factorization (NMF)
Authors: Hiroyuki Kasai
Last page update: Oct. 14, 2020
Latest library version: 1.8.1 (see Release notes for more info)
The NMFLibrary is a pureMatlab library of a collection of algorithms of nonnegative matrix factorization (NMF).
Base NMF
MU (multiplicative updates)
PGD (projected gradient descent)
ALS (alternative least squares)
ANLS (alternative nonnegative least squares)
Variant
GNMF (Graph Regularized NMF)
SemiNMF
NeNMF (NMF with Sinkhorn Distance)
SDNMF (NMF with Sinkhorn Distance)
Robust NMF
Sparse
sparseMU (Sparse multiplicative upates (MU))
sparseNMF (Sparse NMF)
NMFsc (NMF with sparseness constraints)
nsNMF (Nonsmooth NMF)
fnsNMF (Fast nonsmooth NMF)
NMFHALSSO (Hierarchical ALS with soft orthogonal constraint)
Orthgotonal
DTPP (Orthgotonal multiplicative upates (MU))
orthMU (Orthgotonal multiplicative upates (MU))
OrthNMF
Symmetric
SymmANLS (Symmetric ANLS)
D. Kuang, C. Ding, H. Park, "Symmetric Nonnegative Matrix Factorization for Graph Clustering," The 12th SIAM International Conference on Data Mining (SDM'12), pp.106117, 2012.
D. Kuang, S. Yun, H. Park, "SymNMF Nonnegative lowrank approximation of a similarity matrix for graph clustering," Journal of Global Optimization, vol.62, no.3, pp.545574, 2015.
Z. Zhu, X. Li, K. Liu, Q. Li, "Dropping Symmetry for Fast Symmetric Nonnegative Matrix Factorization," NIPS, 2018.
SymmHALS (Symmetric HALS)
SymmNewton (Symmetric Newton)
Online/stochstic NMF
INMF (Incremental NMF) and ONMF (Online NMF)
SPG (Stochastic projected gradient descent)
RONMF (Robust online NMF)
SAGAMUNMF (SAGA multiplicative updates)
SMU (Stochastic multiplicative updates) and SVRMU (Stochastic variance reduced multiplicative updates)
Probabilistic NMF
PNMFGIBBS (Gibbs sampler for nonnegative matrix factorisation, with ARD.)
M.N. Schmidt, O. Winther, L.K. Hansen, "Bayesian nonnegative matrix factorization," International Conference on Independent Component Analysis and Signal Separation, Springer Lecture Notes in Computer Science, Vol. 5441, 2009.
T. Brouwer, P. Lio, "Bayesian Hybrid Matrix Factorisation for Data Integration," 20th International Conference on Artificial Intelligence and Statistics (AISTATS), 2017.
PNMFVB (Variational Bayesian inference for nonnegative matrix factorisation, with ARD)
Category  Name in example codes  function  options.alg 
other options


Base  MUEUC  nmf_mu 
mu 
metric='EUC' 
MUKL  nmf_mu 
mu 
metric='KL' 

MUALPHA  nmf_mu 
mu 
metric='ALPHAD' 

MUBETA  nmf_mu 
mu 
metric='BETAD' 

Modified MU  nmf_mu 
mod_mu 

Acceralated MU  nmf_mu 
acc_mu 

PGD  nmf_pgd 
pgd 

Direct PGD  nmf_pgd 
direct_pgd 

ALS  nmf_als 
als 

Hierarchical ALS  nmf_als 
hals_mu 

Acceralated hierarchical ALS  nmf_als 
acc_hals_mu 

ASGROUP  nmf_anls 
anls_asgroup 

ASGIVENS  nmf_anls 
anls_asgivens 

BPP  nmf_anls 
anls_bpp 

Variant  SemiNMF  semi_nmf 

NeNMF  nenmf 

GNMF  GNMF 

SDNMF  SDNMF 

Sparse  sparseMUEUC  nmf_sparse_mu 
metric='EUC' 

sparseMUKL  nmf_sparse_mu 
metric='KL' 

sparseNMF  sparse_nmf 

NMFsc  nmf_sc 

nsNMF  ns_nmf 

fnsNMF  ns_nmf 
metric='EUC' , update_alg='apg'


Orthogonal  DTPP  nmf_dtpp 

orthMU  nmf_orth_mu 

OrthNMF  
NMFHALSSO  
Symmetric  SymmANLS  symm_anls 

SymmHALS  symm_halsacc 

SymmNewton  symm_newton 

Online  INMF  inmf 

ONMF  onmf 

Acceralated ONMF  omf_acc 

SPG  spg_nmf 

RONMF  ronmf 

SAGAMUNMF  asag_mu_nmf 

SMU  smu_nmf 

SVRMU  svrmu_nmf 

Probabilistic  PNMFVB  pnmf_vb 

PNMFGIBBS  pnmf_gibbs 
./  Top directory. ./README.md  This readme file. ./run_me_first.m  The scipt that you need to run first. ./demo.m  Demonstration script to check and understand this package easily. ./demo_face.m  Demonstration script to check and understand this package easily. plotter/  Contains plotting tools to show convergence results and various plots. auxiliary/  Some auxiliary tools for this project. solver/  Contains various optimization algorithms.  base/  Basic NMF solvers.  online/  Online/stochstic NMF solvers.  sparse/  Sparse NMF solvers.  robust/  Robust NMF solvers.  orthogonal/  Orthogonal NMF solvers.  symm/  Symmetric NMF solvers.  nenmf/  Nesterov's accelerated NMF solver.  probabilistic/  Probabilistic NMF solvers.  3rd_party/  Solvers provided by 3rd_party.
Run run_me_first
for path configurations.
%% First run the setup script
run_me_first;
Just execute demo
for the simplest demonstration of this package. .
%% Execute the demonstration script
demo;
The "demo.m" file contains below.
%% generate synthetic data nonnegative matrix V size of (mxn)
m = 500;
n = 100;
V = rand(m,n);
%% Initialize rank to be factorized
rank = 5;
%% perform factroization
% MU
options.alg = 'mu';
[w_nmf_mu, infos_nmf_mu] = nmf_mu(V, rank, options);
% Hierarchical ALS
options.alg = 'hals';
[w_nmf_hals, infos_nmf_hals] = nmf_als(V, rank, options);
%% plot
display_graph('epoch','cost', {'MU', 'HALS'}, {w_nmf_mu, w_nmf_hals}, {infos_nmf_mu, infos_nmf_hals});
Let's take a closer look at the code above bit by bit. The procedure has only 4 steps!
Step 1: Generate data
First, we generate synthetic data of V of size (mxn).
m = 500;
n = 100;
V = rand(m,n);
Step 2: Define rank
We set the rank value.
rank = 5;
Step 3: Perform solver
Now, you can perform optimization solvers, e.g., MU and Hierarchical ALS (HALS), calling solver functions, i.e., nmf_mu()
function and nmf_als()
function after setting some optimization options.
% MU
options.alg = 'mu';
[w_nmf_mu, infos_nmf_mu] = nmf_mu(V, rank, options);
% Hierarchical ALS
options.alg = 'hals';
[w_nmf_hals, infos_nmf_hals] = nmf_als(V, rank, options);
They return the final solutions of w
and the statistics information that include the histories of epoch numbers, cost values, norms of gradient, the number of gradient evaluations and so on.
Step 4: Show result
Finally, display_graph()
provides output results of decreasing behavior of the cost values in terms of the number of iterrations (epochs) and time [sec].
display_graph('epoch','cost', {'MU', 'HALS'}, {w_nmf_mu, w_nmf_hals}, {infos_nmf_mu, infos_nmf_hals});
display_graph('time','cost', {'MU', 'HALS'}, {w_nmf_mu, w_nmf_hals}, {infos_nmf_mu, infos_nmf_hals});
That's it!
"demo_face.m" illustrates the learned basis (dictrionary) in case of CBCL face datasets.
The dataset is first loaded into V instead of generating synthetic data in Step 1.
V = importdata('./data/CBCL_face.mat');
Then, we can display basis elements (W: dictionary) obtained with different algorithms additionally in Step 4.
plot_dictionnary(w_nmf_mu.W, [], [7 7]);
plot_dictionnary(w_nmf_hals.W, [], [7 7]);
nnlsm_activeset.m
, nnls1_asgivens.m
, nnlsm_blockpivot.m
, and normalEqComb.m
written by Jingu Kim.nlssubprob.m
.GNMF.m
, GNMF_Multi.m
, constructW.m
and litekmeans.m
writtnen by Deng Cai.SDNMF.m
, and SDNMF_Multi.m
writtnen by Wei Qian.nmf_mu.m
and nmf_als.m
for MU and HALS from Nicolas Gillis.plot_dictionnary.m
, rescale.m
, and getoptions.m
.If you have any problems or questions, please contact the author: Hiroyuki Kasai (email: hiroyuki dot kasai at waseda dot jp)