  The ever-increasingly urban populations and their material demands have
brought unprecedented burdens to cities. Smart cities leverage emerging
technologies like the Internet of Things (IoT), Cognitive Radio Wireless Sensor
Network (CR-WSN) to provide better QoE and QoS for all citizens. However,
resource scarcity is an important challenge in CR-WSN. Generally, this problem
is handled by auction theory or game theory. To make CR-WSN nodes intelligent
and more autonomous in resource allocation, we propose a multi-agent
reinforcement learning (MARL) algorithm to learn the optimal resource
allocation strategy in the oligopoly market model. Firstly, we model a
multi-agent scenario, in which the primary users (PUs) is the sellers and the
secondary users (SUs) is the buyers. Then, we propose the Q-probabilistic
multiagent learning (QPML) and apply it to allocate resources in the market. In
the multi-agent interactive learning process, the PUs and SUs learn strategies
to maximize their benefits and improve spectrum utilization. Experimental
results show the efficiency of our QPML approach, which can also converge
quickly.
