  Feature transformation aims to extract a good representation (feature) space
by mathematically transforming existing features. It is crucial to address the
curse of dimensionality, enhance model generalization, overcome data sparsity,
and expand the availability of classic models. Current research focuses on
domain knowledge-based feature engineering or learning latent representations;
nevertheless, these methods are not entirely automated and cannot produce a
traceable and optimal representation space. When rebuilding a feature space for
a machine learning task, can these limitations be addressed concurrently? In
this extension study, we present a self-optimizing framework for feature
transformation. To achieve a better performance, we improved the preliminary
work by (1) obtaining an advanced state representation for enabling reinforced
agents to comprehend the current feature set better; and (2) resolving Q-value
overestimation in reinforced agents for learning unbiased and effective
policies. Finally, to make experiments more convincing than the preliminary
work, we conclude by adding the outlier detection task with five datasets,
evaluating various state representation approaches, and comparing different
training strategies. Extensive experiments and case studies show that our work
is more effective and superior.
