  In the context of modern environmental and societal concerns, there is an
increasing demand for methods able to identify management strategies for civil
engineering systems, minimizing structural failure risks while optimally
planning inspection and maintenance (I&M) processes. Most available methods
simplify the I&M decision problem to the component level due to the
computational complexity associated with global optimization methodologies
under joint system-level state descriptions. In this paper, we propose an
efficient algorithmic framework for inference and decision-making under
uncertainty for engineering systems exposed to deteriorating environments,
providing optimal management strategies directly at the system level. In our
approach, the decision problem is formulated as a factored partially observable
Markov decision process, whose dynamics are encoded in Bayesian network
conditional structures. The methodology can handle environments under equal or
general, unequal deterioration correlations among components, through Gaussian
hierarchical structures and dynamic Bayesian networks. In terms of policy
optimization, we adopt a deep decentralized multi-agent actor-critic (DDMAC)
reinforcement learning approach, in which the policies are approximated by
actor neural networks guided by a critic network. By including deterioration
dependence in the simulated environment, and by formulating the cost model at
the system level, DDMAC policies intrinsically consider the underlying
system-effects. This is demonstrated through numerical experiments conducted
for both a 9-out-of-10 system and a steel frame under fatigue deterioration.
Results demonstrate that DDMAC policies offer substantial benefits when
compared to state-of-the-art heuristic approaches. The inherent consideration
of system-effects by DDMAC strategies is also interpreted based on the learned
policies.
