Modeling data using Directional Distributions | Center for Big Data Analytics

Abstract: Traditionally multi-variate normal distributions have been the staple of data modeling in most domains. For some domains, the model they provide is either inadequate or incorrect because of the disregard for the directional components of the data. We present a generative model for data that is suitable for modeling directional data (as can arise in text and gene expression clustering). We use mixtures of von Mises-Fisher distributions to model our data since the von Mises-Fisher distribution is the natural distribution for directional data. We derive an Expectation Maximization (EM) algorithm to find the maximum likelihood estimates for the parameters of our mixture model, and provide various experimental results to evaluate the “correctness” of our formulation. In this paper we also provide some of the mathematical background necessary to carry out all the derivations and to gain insight for an implementation.

Topics:
Data Clustering

Download: pdf

Citation

Modeling data using Directional Distributions (pdf, software)
I. Dhillon, S. Sra.
University of Texas Computer Science Technical Report (UTCS Technical Report) TR-03-06, January 2003.

Bibtex:
@techreport{dhillon2003modelingd, author = "Inderjit S. Dhillon AND Suvrit Sra", title = "Modeling data using Directional Distributions", institution = "The University of Texas at Austin", number = "TR-03-06", year = "2003", month = "jan", abstract = "Traditionally multi-variate normal distributions have been the staple of data modeling in most domains. For some domains, the model they provide is either inadequate or incorrect because of the disregard for the directional components of the data. We present a generative model for data that is suitable for modeling directional data (as can arise in text and gene expression clustering). We use mixtures of von Mises-Fisher distributions to model our data since the von Mises-Fisher distribution is the natural distribution for directional data. We derive an Expectation Maximization (EM) algorithm to find the maximum likelihood estimates for the parameters of our mixture model, and provide various experimental results to evaluate the “correctness” of our formulation. In this paper we also provide some of the mathematical background necessary to carry out all the derivations and to gain insight for an implementation." }