tnmf.TransformInvariantNMF

Transform-Invariant Non-Negative Matrix Factorization

Authors: Adrian Šošić, Mathias Winkel

Module Contents

Classes

MiniBatchAlgorithm

MiniBatch algorithms that can be used with TransformInvariantNMF.fit_minibatch().

TransformInvariantNMF

Transform Invariant Non-Negative Matrix Factorization.
class tnmf.TransformInvariantNMF.MiniBatchAlgorithm

Bases: enum.Enum

MiniBatch algorithms that can be used with TransformInvariantNMF.fit_minibatch().

Cyclic_MU = 4
ASG_MU = 5
GSG_MU = 6
ASAG_MU = 7
GSAG_MU = 8
class tnmf.TransformInvariantNMF.TransformInvariantNMF(n_atoms: int, atom_shape: Tuple[int, Ellipsis], inhibition_range: Union[int, Tuple[int, Ellipsis]] = None, backend: str = 'numpy_fft', logger: logging.Logger = None, verbose: int = 0, **kwargs)

Transform Invariant Non-Negative Matrix Factorization.

Finds non-negative tensors W (dictionary) and H (activations) that approximate the non-negative tensor V (samples) for a given transform operator`. # TODO: add link to TNMF model

Note

Currently, only a single transform type, corresponding to shift invariance, is supported and hard-coded. In contrast to other generic types of transforms, shift invariance can be efficiently achieved through convolution operations (or, equivalently, multiplication in Fourier domain). Therefore, shift invariance will remain hard-coded and retained as an optional transform type even when additional transforms become supported in future releases.

Optimization is performed via multiplicative updates to W and H, see 1. Minibatch updates are possible via a selection of algorithms from 2. Different backend implementations (NumPy, PyTorch, with/without FFT, etc.) can be selected by the user.

Parameters

  • n_atoms (int) – Number of elementary atoms. The shape of W will be (n_atoms, n_channels, *atom_shape).

  • atom_shape (Tuple[int, ]) – Shape of the elementary atoms. The shape of W will be (n_atoms, n_channels, *atom_shape).

  • inhibition_range (Union[int, Tuple[int, ]], default = None) – Lateral inhibition range. If set to None, the value is set to *atom_shape, which ensures that activations are pairwise sufficiently far apart, that the corresponding atoms do not overlap in the reconstruction.

  • backend ({'numpy', 'numpy_fft', 'numpy_caching_fft', 'pytorch'}, default = 'numpy_fft') –

    Defines the optimization backend.

  • logger (logging.Logger, default = None) – Logger instance used for intermediate output. If None, an internal logger instance will be created.

  • verbose ({0, 1, 2, 3}, default = 0) –

    Verbosity level.

    • 0: Show only errors.

    • 1: Include warnings.

    • 2: Include info.

    • 3: Include debug messages.

  • **kwargs – Keyword arguments that are handed to the constructor of the backend class.

W

The dictionary tensor of shape (n_atoms, num_channels, *atom_shape).

Type

np.ndarray

H

The activation tensor of shape (num_samples, num_atoms, *shift_shape).

Type

np.ndarray

R

The reconstruction of the sample tensor using the current W and H.

Type

np.ndarray

Examples

TODO: add examples

Notes

Planned features:

  • Batch processing

  • Arbitrary transform types

  • Nested transformations

  • Additional reconstruction norms

References

# TODO: add bibtex file

1

D.D. Lee, H.S. Seung, 2000. Algorithms for Non-negative Matrix Factorization, in: Proceedings of the 13th International Conference on Neural Information Processing Systems. pp. 535–541. https://doi.org/10.5555/3008751.3008829

2

R. Serizel, S. Essid, G. Richard, 2016. Mini-batch stochastic approaches for accelerated multiplicative updates in nonnegative matrix factorisation with beta-divergence, in: 26th International Workshop on Machine Learning for Signal Processing (MLSP). pp 1-6. http://ieeexplore.ieee.org/document/7738818/

property W(self) → numpy.ndarray
property H(self) → numpy.ndarray
property V(self) → numpy.ndarray
property R(self) → numpy.ndarray
R_partial(self, i_atom: int) → numpy.ndarray
fit_batch(self, V: numpy.ndarray, n_iterations: int = 1000, update_H: bool = True, update_W: bool = True, keep_W: bool = False, sparsity_H: float = 0.0, inhibition_strength: float = 0.0, cross_atom_inhibition_strength: float = 0.0, progress_callback: Callable[[TransformInvariantNMF, int], bool] = None)

Perform non-negative matrix factorization of samples V, i.e. optimization of dictionary W and activations H.

Parameters

  • V (np.ndarray) – Samples to be reconstructed. The shape of the sample tensor is (n_samples, n_channels, *sample_shape), where sample_shape is the shape of the individual samples and each sample consists of n_channels individual channels.

  • n_iterations (int, default = 1000) – Maximum number of iterations (W and H updates) to be performed.

  • update_H (bool, default = True) – If False, the activation tensor H will not be updated.

  • update_W (bool, default = True) – If False, the dictionary tensor W will not be updated.

  • keep_W (bool, default = False) – If False, the dictionary tensor W will not be (re)initialized before starting iteration.

  • sparsity_H (float, default = 0.) – Sparsity enforcing regularization for the H update.

  • inhibition_strength (float, default = 0.) – Lateral inhibition regularization factor for the H update within the same atom.

  • cross_atom_inhibition_strength (float, default = 0.) – Lateral inhibition regularization factor for the H update across different atoms.

  • progress_callback (Callable[[``’TransformInvariantNMF’``, int], bool], default = None) –

    If provided, this function will be called after every iteration, i.e. after every update to H and W. The first parameter to the function is the calling TransformInvariantNMF instance, which can be used to inspect intermediate results, etc. The second parameter is the current iteration step.

    If the progress_callback function returns False, iteration will be aborted, which allows to implement specialized convergence criteria.

fit_minibatches(self, V: numpy.ndarray, algorithm: MiniBatchAlgorithm = MiniBatchAlgorithm.ASG_MU, batch_size: int = 3, n_epochs: int = 1000, sag_lambda: float = 0.2, keep_W: bool = False, sparsity_H: float = 0.0, inhibition_strength: float = 0.0, cross_atom_inhibition_strength: float = 0.0, progress_callback: Callable[[TransformInvariantNMF, int], bool] = None)

Perform non-negative matrix factorization of samples V, i.e. optimization of dictionary W and activations H via mini-batch updates using a selection of algorithms from 3.

Parameters

  • V (np.ndarray) – Samples to be reconstructed. The shape of the sample tensor is (n_samples, n_channels, *sample_shape), where sample_shape is the shape of the individual samples and each sample consists of n_channels individual channels.

  • algorithm (MiniBatchAlgorithm) – MiniBatch update scheme to be used. See MiniBatchAlgorithm and 3 for the different choices.

  • batch_size (int, default = 3) – Number of samples per mini batch.

  • n_epochs (int, default = 1000) – Maximum number of epochs across the full sample set to be performed.

  • sag_lambda (float, default = 0.2) – Exponential forgetting factor for for the stochastic _average_ gradient updates, i.e. MiniBatchAlgorithm.ASAG_MU and MiniBatchAlgorithm.GSAG_MU

  • keep_W (bool, default = False) – If False, the dictionary tensor W will not be (re)initialized before starting iteration.

  • sparsity_H (float, default = 0.) – Sparsity enforcing regularization for the H update.

  • inhibition_strength (float, default = 0.) – Lateral inhibition regularization factor for the H update within the same atom.

  • cross_atom_inhibition_strength (float, default = 0.) – Lateral inhibition regularization factor for the H update across different atoms.

  • progress_callback (Callable[[``’TransformInvariantNMF’``, int], bool], default = None) –

    If provided, this function will be called after every (epoch-)iteration. The first parameter to the function is the calling TransformInvariantNMF instance, which can be used to inspect intermediate results, etc. The second parameter is the current iteration step.

    If the progress_callback function returns False, (epoch-)iteration will be aborted, which allows to implement specialized convergence criteria.

References

3(1,2)

R. Serizel, S. Essid, G. Richard, 2016. Mini-batch stochastic approaches for accelerated multiplicative updates in nonnegative matrix factorisation with beta-divergence, in: 26th International Workshop on Machine Learning for Signal Processing (MLSP). pp 1-6. http://ieeexplore.ieee.org/document/7738818/

fit_stream(self, V: Iterator[numpy.ndarray], subsample_size: int = 3, max_subsamples: int = None, **kwargs)
fit(self, V: numpy.ndarray, **kwargs)