Sparse Additive Models | Center for Big Data Analytics

Abstract: We present a new class of methods for high-dimensional nonparametric regression and classification called sparse additive models (SpAM). Our methods combine ideas from sparse linear modeling and additive nonparametric regression. We derive an algorithm for fitting the models that is practical and effective even when the number of covariates is larger than the sample size. SpAM is closely related to the COSSO model of Lin and Zhang (2006), but decouples smoothing and sparsity, enabling the use of arbitrary nonparametric smoothers. An analysis of the theoretical properties of SpAM is given. We also study a greedy estimator that is a nonparametric version of forward stepwise regression. Empirical results on synthetic and real data are presented, showing that SpAM can be effective in fitting sparse nonparametric models in high dimensional data.

Download: pdf, arXiv version

Citation

Sparse Additive Models (pdf, arXiv, software)
P. Ravikumar, J. Lafferty, H. Liu, L. Wasserman.
Journal of the Royal Statistical Society: Series B (Statistical Methodology) (JRSSB) 71(5), pp. 1009-1030, 2009.

Bibtex:
@article{ravikumar2009sparseadd, author = "Pradeep Ravikumar AND John D. Lafferty AND Han Liu AND Larry Wasserman", title = "Sparse Additive Models", journal = "Journal of the Royal Statistical Society: Series B (Statistical Methodology) (JRSSB)", page = "1009–1030", volume = "71", issue = "5", year = "2009", abstract = "We present a new class of methods for high-dimensional nonparametric regression and classification called sparse additive models (SpAM). Our methods combine ideas from sparse linear modeling and additive nonparametric regression. We derive an algorithm for fitting the models that is practical and effective even when the number of covariates is larger than the sample size. SpAM is closely related to the COSSO model of Lin and Zhang (2006), but decouples smoothing and sparsity, enabling the use of arbitrary nonparametric smoothers. An analysis of the theoretical properties of SpAM is given. We also study a greedy estimator that is a nonparametric version of forward stepwise regression. Empirical results on synthetic and real data are presented, showing that SpAM can be effective in fitting sparse nonparametric models in high dimensional data." }