  Many statistical methods require solutions to optimization problems. When the
global solution is hard to attain, statisticians always use the better if there
are two solutions for chosen, where the word "better" is understood in the
sense of optimization. This seems reasonable in that the better solution is
more likely to be the global solution, whose statistical properties of interest
usually have been well established. From the statistical perspective, we use
the better solution because we intuitively believe the principle, called better
solution principle (BSP) in this paper, that a better solution to a statistical
optimization problem also has better statistical properties of interest. BSP
displays some concordance between optimization and statistics, and is expected
to widely hold. Since theoretical study on BSP seems to be neglected by
statisticians, this paper aims to establish a framework for discussing BSP in
various statistical optimization problems. We demonstrate several simple but
effective comparison theorems as the key results of this paper, and apply them
to verify BSP in commonly encountered statistical optimization problems,
including maximum likelihood estimation, best subsample selection, and best
subset regression. It can be seen that BSP for these problems holds under
reasonable conditions, i.e., a better solution indeed has better statistical
properties of interest. In addition, guided by the BSP theory, we develop a new
best subsample selection method that performs well when there are clustered
outliers.
