  The prosperity of smart mobile devices has made mobile crowdsensing (MCS) a
promising paradigm for completing complex sensing and computation tasks. In the
past, great efforts have been made on the design of incentive mechanisms and
task allocation strategies from MCS platform's perspective to motivate mobile
users' participation. However, in practice, MCS participants face many
uncertainties coming from their sensing environment as well as other
participants' strategies, and how do they interact with each other and make
sensing decisions is not well understood. In this paper, we take MCS
participants' perspective to derive an online sensing policy to maximize their
payoffs via MCS participation. Specifically, we model the interactions of
mobile users and sensing environments as a multi-agent Markov decision process.
Each participant cannot observe others' decisions, but needs to decide her
effort level in sensing tasks only based on local information, e.g., its own
record of sensed signals' quality. To cope with the stochastic sensing
environment, we develop an intelligent crowdsensing algorithm IntelligentCrowd
by leveraging the power of multi-agent reinforcement learning (MARL). Our
algorithm leads to the optimal sensing policy for each user to maximize the
expected payoff against stochastic sensing environments, and can be implemented
at individual participant's level in a distributed fashion. Numerical
simulations demonstrate that IntelligentCrowd significantly improves users'
payoffs in sequential MCS tasks under various sensing dynamics.
