Learning Low-Rank Kernel Matrices | Center for Big Data Analytics

Abstract: Kernel learning plays an important role in many machine learning tasks. However, algorithms for learning a kernel matrix often scale poorly, with running times that are cubic in the number of data points. In this paper, we propose efficient algorithms for learning low-rank kernel matrices; our algorithms scale linearly in the number of data points and quadratically in the rank of the kernel. We introduce and employ Bregman matrix divergences for rank-deficient matrices—these divergences are natural for our problem since they preserve the rank as well as positive semi-definiteness of the kernel matrix. Special cases of our framework yield faster algorithms for various existing kernel learning problems. Experimental results demonstrate the effectiveness of our algorithms in learning both low-rank and full-rank kernels.

Topics:
Kernel Methods

Download: pdf

Citation

Learning Low-Rank Kernel Matrices (pdf, software)
B. Kulis, M. Sustik, I. Dhillon.
In International Conference on Machine Learning (ICML), pp. 505-512, July 2006.

Bibtex:
@inproceedings{kulis2006learningl, author = "Brian J. Kulis AND Mátyás Sustik AND Inderjit S. Dhillon", title = "Learning Low-Rank Kernel Matrices", booktitle = "International Conference on Machine Learning (ICML)", page = "505–512", year = "2006", month = "jul", abstract = "Kernel learning plays an important role in many machine learning tasks. However, algorithms for learning a kernel matrix often scale poorly, with running times that are cubic in the number of data points. In this paper, we propose efficient algorithms for learning low-rank kernel matrices; our algorithms scale linearly in the number of data points and quadratically in the rank of the kernel. We introduce and employ Bregman matrix divergences for rank-deficient matrices—these divergences are natural for our problem since they preserve the rank as well as positive semi-definiteness of the kernel matrix. Special cases of our framework yield faster algorithms for various existing kernel learning problems. Experimental results demonstrate the effectiveness of our algorithms in learning both low-rank and full-rank kernels." }