  An important question that often arises in the operation of networked systems
is whether to collect the real-time data or to estimate them based on the
previously collected data. Various factors should be taken into account such as
how informative the data are at each time instant for state estimation, how
costly and credible the collected data are, and how rapidly the data vary with
time. The above question can be formulated as a dynamic decision making problem
with imperfect information structure, where a decision maker wishes to find an
efficient way to switch between data collection and data estimation while the
quality of the estimation depends on the previously collected data (i.e.,
duality effect). In this paper, the evolution of the state of each node is
modeled as an exchangeable Markov process for discrete features and equivariant
linear system for continuous features, where the data of interest are defined
in the former case as the empirical distribution of the states, and in the
latter case as the weighted average of the states. When the data are collected,
they may or may not be credible, according to a Bernoulli distribution. Based
on a novel planning space, a Bellman equation is proposed to identify a
near-optimal strategy. A reinforcement learning algorithm is developed for the
case when the model is not known exactly, and its convergence to the
near-optimal solution is shown subsequently. In addition, a certainty threshold
is introduced that determines when data estimation is more desirable than data
collection, as the number of nodes increases. For the special case of linear
dynamics, a separation principle is constructed wherein the optimal estimate is
computed by a Kalman-like filter, irrespective of the probability distribution
of random variables...
