  Technological innovations have revolutionized the process of scientific
research and knowledge discovery. The availability of massive data and
challenges from frontiers of research and development have reshaped statistical
thinking, data analysis and theoretical studies. The challenges of
high-dimensionality arise in diverse fields of sciences and the humanities,
ranging from computational biology and health studies to financial engineering
and risk management. In all of these fields, variable selection and feature
extraction are crucial for knowledge discovery. We first give a comprehensive
overview of statistical challenges with high dimensionality in these diverse
disciplines. We then approach the problem of variable selection and feature
extraction using a unified framework: penalized likelihood methods. Issues
relevant to the choice of penalty functions are addressed. We demonstrate that
for a host of statistical problems, as long as the dimensionality is not
excessively large, we can estimate the model parameters as well as if the best
model is known in advance. The persistence property in risk minimization is
also addressed. The applicability of such a theory and method to diverse
statistical problems is demonstrated. Other related problems with
high-dimensionality are also discussed.
