STATISTICAL LEARNING WITH TENSOR VALUED DATA

David Rice

The prevalence of tensor structured data has never been greater. While there is a large history of regression and clustering methods for univariate and vector data the development of specialized methods pertaining directly to tensors are still emerging. A common technique is to leverage the multilinear structure of both the model and data through decomposition. This dissertation is an applied work on how multilinear algebra can be leveraged to develop informed statistic modeling. The first project concerns the clustering of matrix valued data. Our proposed method maintains the matrix structure of the data by modeling via a mixture matrix normal distributions. Clustering techniques often involve several cluster numbers are tried and analyzed post-hoc, however a Bayesian approach allows the number of clusters to grow and shrink based on the data. A natural assumption for matrix valued data, especially when modeling spatio-temporal or image data, is that the covariance between far entries are sparse. A natural approach to incorporate this assumption is to impose a shrinkage prior for the off diagonal elements. The viability of this approach to spatio-temporal and image data is demonstrated through an application to time series energy modeling and medical imaging. The modeling of heterogeneity in matrix valued data has the natural extension of considering the tensor valued data. For tensor valued data, the volume of estimated model coefficients increases rapidly, necessitating the leverage of tensor decompositions. The second project attacks the problem of clustering and regression simultaneously, where observations within each cluster have a separate linear relationship with the response. Modeling is extended to the tensor domain, where a low rank tensor structure is assumed for the regression coefficients, incorporating the multi-linear structure, and reducing the number of estimated parameters. we demonstrate the use case for modeling the relationships including facial and FMRI data. Motivated by the parameter reduction in tensor linear regression, the third project leverages tensor decompositions for nonlinear regression tasks. Kolmogorov-Arnold networks are spline based neural networks with a natural tensor structure in each layer. Our work imposes a low-rank structure on each layer for parameter compression, where the tensor rank is learned adaptively during training. This method is able to maintain expressivity, and performs well on simulations and machine learning benchmarks when compared to the dense network and to multilayer perceptron. The spline basis assumption makes the method well suited for physics modeling, and we demonstrate our method as an effective surrogate model for modeling the two-fluid Tolman-Oppenheimer-Volkov equations for admixed neutron stars.

STATISTICAL LEARNING WITH TENSOR VALUED DATA

Files and links (1)

Abstract

Metrics

Details