Clustering Singular and Non-Singular Covariance Matrices for Classification

Volstorff A

Linear Discriminant Analysis (LDA) and Quadratic Discriminant Analysis (QDA) are commonly used methods for classification. LDA requires the strong assumption that each class has the same covariance matrix and QDA tends to require a large number of observations per class in order to have a stable estimate of the covariance matrix. There are situations, such as in forensic science, where there are a large number of classes in a high dimensional space with few observations per class. In these situations, the covariance estimate within each class is either unstable or singular. We introduce a novel, model-baed methodology which can relax the shared covariance assumptions of LDA by clustering sample covariance matrices, either singular or non-singular, into K clusters at which point the LDA assumption is made within each cluster.