  The integration of observational data into numerical models, known as data
assimilation (DA), is fundamental for making Numerical Weather Prediction (NWP)
possible, with breathtaking success over the past 60 years (Bauer et al. 2015).
Traditional DA methods, such as variational techniques and ensemble Kalman
filters, are basic pillars of current NWP by incorporating diverse
observational data. However, the emergence of artificial intelligence (AI)
presents new opportunities for further improvements. AI-based approaches can
emulate the complex computations of traditional NWP models at a reduced
computational cost, offering the potential to speed up and improve analyses and
forecasts dramatically (e.g. Pathak et al., 2022; Bi et al., 2023; Lam et al.,
2023; Bouallegue et al., 2023). AI itself plays a growing role in optimization
(e.g. Fan et al., 2024), which offers new possibilities also beyond model
emulation. In this paper, we introduce a novel AI-based variational DA approach
designed to replace classical methods of DA by leveraging deep learning
techniques. Unlike previous hybrid approaches, our method integrates the DA
process directly into a neural network, utilizing the variational DA framework.
This innovative AI-based system, termed AI-Var, employs a neural network
trained to minimize the variational cost function, enabling it to perform DA
without relying on pre-existing analysis datasets. We present a
proof-of-concept implementation of this approach, demonstrating its feasibility
through a series of idealized and real-world test cases. Our results indicate
that the AI-Var system can efficiently assimilate observations and produce
accurate initial conditions for NWP, highlighting its potential to carry out
the DA process in weather forecasting. This advancement paves the way for fully
data-driven NWP systems, offering a significant leap forward in computational
efficiency.
