  In the era of big data, analysts usually explore various statistical models
or machine learning methods for observed data in order to facilitate scientific
discoveries or gain predictive power. Whatever data and fitting procedures are
employed, a crucial step is to select the most appropriate model or method from
a set of candidates. Model selection is a key ingredient in data analysis for
reliable and reproducible statistical inference or prediction, and thus central
to scientific studies in fields such as ecology, economics, engineering,
finance, political science, biology, and epidemiology. There has been a long
history of model selection techniques that arise from researches in statistics,
information theory, and signal processing. A considerable number of methods
have been proposed, following different philosophies and exhibiting varying
performances. The purpose of this article is to bring a comprehensive overview
of them, in terms of their motivation, large sample performance, and
applicability. We provide integrated and practically relevant discussions on
theoretical properties of state-of- the-art model selection approaches. We also
share our thoughts on some controversial views on the practice of model
selection.
