  Big models have achieved revolutionary breakthroughs in the field of AI, but
they might also pose potential concerns. Addressing such concerns, alignment
technologies were introduced to make these models conform to human preferences
and values. Despite considerable advancements in the past year, various
challenges lie in establishing the optimal alignment strategy, such as data
cost and scalable oversight, and how to align remains an open question. In this
survey paper, we comprehensively investigate value alignment approaches. We
first unpack the historical context of alignment tracing back to the 1920s
(where it comes from), then delve into the mathematical essence of alignment
(what it is), shedding light on the inherent challenges. Following this
foundation, we provide a detailed examination of existing alignment methods,
which fall into three categories: Reinforcement Learning, Supervised
Fine-Tuning, and In-context Learning, and demonstrate their intrinsic
connections, strengths, and limitations, helping readers better understand this
research area. In addition, two emerging topics, personal alignment, and
multimodal alignment, are also discussed as novel frontiers in this field.
Looking forward, we discuss potential alignment paradigms and how they could
handle remaining challenges, prospecting where future alignment will go.
