Linear Algebra for Modern Statistics: Efficiency and Interpretability Challenges in Regression and PCA

Abstrak

The development of modern statistics faces the challenge of high-dimensional data complexity, which requires an efficient yet interpretable approach. Linear algebra offers a solution through matrix representation, but its limitations in non-linear contexts and high-dimensional interpretation need to be examined in greater depth. This study analyzes algebraic methods through case studies of linear regression (normal equation solutions) and PCA (eigen decomposition), tested on synthetic datasets and MNIST. The results show: (1) a 40% computational acceleration in matrix-based regression, (2) PCA successfully reduces the MNIST dimension to 3 main components (retaining 85% of the variance), but a survey reveals that 73% of practitioners have difficulty interpreting high-dimensional components. Despite its efficiency advantages, algebraic methods require further development through hybrid approaches (kernel PCA) and interpretable techniques to address limitations in linearity and high-dimensional complexity.