  Real-world problems typically require the simultaneous optimization of
several, often conflicting objectives. Many of these multi-objective
optimization problems are characterized by wide ranges of uncertainties in
their decision variables or objective functions, which further increases the
complexity of optimization. To cope with such uncertainties, robust
optimization is widely studied aiming to distinguish candidate solutions with
uncertain objectives specified by confidence intervals, probability
distributions or sampled data. However, existing techniques mostly either fail
to consider the actual distributions or assume uncertainty as instances of
uniform or Gaussian distributions. This paper introduces an empirical approach
that enables an efficient comparison of candidate solutions with uncertain
objectives that can follow arbitrary distributions. Given two candidate
solutions under comparison, this operator calculates the probability that one
solution dominates the other in terms of each uncertain objective. It can
substitute for the standard comparison operator of existing optimization
techniques such as evolutionary algorithms to enable discovering robust
solutions to problems with multiple uncertain objectives. This paper also
proposes to incorporate various uncertainties in well-known multi-objective
problems to provide a benchmark for evaluating uncertainty-aware optimization
techniques. The proposed comparison operator and benchmark suite are integrated
into an existing optimization tool that features a selection of multi-objective
optimization problems and algorithms. Experiments show that in comparison with
existing techniques, the proposed approach achieves higher optimization quality
at lower overheads.
