Optimality conditions for penalized sparse PCA

This paper establishes the theoretical foundations of an alternating optimization scheme for penalized sparse principal component analysis (PCA) focusing on variance maximization. We provide a theoretical foundation for the optimality of solutions derived from this widely used algorithm, addressing a gap in the current literature where empirical results often lack theoretical support. We show the algorithm’s success when the dataset’s covariance matrix is positive definite. Additionally, we characterize sparsity-inducing penalties and examine the use of various ones, including the L1-norm, SCAD, and L0-norm. We conduct numerical experiments to evaluate standard metrics, such as explained variance, number of iterations, and computational time.