TY - JOUR T1 - Solving Optimization Problems over the Stiefel Manifold by Smooth Exact Penalty Functions AU - Xiao , Nachuan AU - Liu , Xin JO - Journal of Computational Mathematics VL - 5 SP - 1246 EP - 1276 PY - 2024 DA - 2024/07 SN - 42 DO - http://doi.org/10.4208/jcm.2307-m2021-0331 UR - https://global-sci.org/intro/article_detail/jcm/23277.html KW - Orthogonality constraint, Stiefel manifold, Penalty function. AB -

In this paper, we present a novel penalty model called ExPen for optimization over the Stiefel manifold. Different from existing penalty functions for orthogonality constraints, ExPen adopts a smooth penalty function without using any first-order derivative of the objective function. We show that all the first-order stationary points of ExPen with a sufficiently large penalty parameter are either feasible, namely, are the first-order stationary points of the original optimization problem, or far from the Stiefel manifold. Besides, the original problem and ExPen share the same second-order stationary points. Remarkably, the exact gradient and Hessian of ExPen are easy to compute. As a consequence, abundant algorithm resources in unconstrained optimization can be applied straightforwardly to solve ExPen.