This is a project on basic research. We focus on a fundamental optimization problem in machine learning. Successfully solving this problem will improve a wide range of tasks, for example, cluster analysis, topic discovery, signal processing, recommendation.
In the above machine learning tasks, the model is often parameterized by nonnegative matrices (or vectors). The learning, implemented by convectional additive gradient based optimization, is inefficient in finding the relative scales among the matrix entries. It has been shown that multiplicative updates are an effective way to overcome the drawback. However, multiplicative updates currently are in batch mode. That is, the whole data set is used in each update, which can cause slow convergence and are prone to poor stationary points. In this project we propose to improve the optimization by combining multiplicative updates and stochastic gradient descent, where the resulting method is called stochastic multiplicative updates.
Unlike those projects which just use off-the-shelf tools, the student(s) in this project should welcome the challenge to make breakthrough in the machine learning essence. Good programming and university mathematics are required to have fun the project.