  The challenge in biomarker discovery using machine learning from omics data
lies in the abundance of molecular features but scarcity of samples. Most
feature selection methods in machine learning require evaluating various sets
of features (models) to determine the most effective combination. This process,
typically conducted using a validation dataset, involves testing different
feature sets to optimize the model's performance. Evaluations have performance
estimation error and when the selection involves many models the best ones are
almost certainly overestimated. Biomarker identification with feature selection
methods can be addressed as a multi-objective problem with trade-offs between
predictive ability and parsimony in the number of features. Genetic algorithms
are a popular tool for multi-objective optimization but they evolve numerous
solutions thus are prone to overestimation. Methods have been proposed to
reduce the overestimation after a model has already been selected in
single-objective problems, but no algorithm existed capable of reducing the
overestimation during the optimization, improving model selection, or applied
in the more general multi-objective domain. We propose DOSA-MO, a novel
multi-objective optimization wrapper algorithm that learns how the original
estimation, its variance, and the feature set size of the solutions predict the
overestimation. DOSA-MO adjusts the expectation of the performance during the
optimization, improving the composition of the solution set. We verify that
DOSA-MO improves the performance of a state-of-the-art genetic algorithm on
left-out or external sample sets, when predicting cancer subtypes and/or
patient overall survival, using three transcriptomics datasets for kidney and
breast cancer.
