  The Internet of Things (IoT) will encompass a massive number of machine type
devices that must wirelessly transmit, in near real-time, a diverse set of
messages sensed from their environment. Designing resource allocation schemes
to support such coexistent, heterogeneous communication is hence a key IoT
challenge. In particular, there is a need for self-organizing resource
allocation solutions that can account for unique IoT features, such as massive
scale and stringent resource constraints. In this paper, a novel finite memory
multi-state sequential learning framework is proposed to enable diverse IoT
devices to share limited communication resources, while transmitting both
delay-tolerant, periodic messages and urgent, critical messages. The proposed
learning framework enables the IoT devices to learn the number of critical
messages and to reallocate the communication resources for the periodic
messages to be used for the critical messages. Furthermore, the proposed
learning framework explicitly accounts for IoT device limitations in terms of
memory and computational capabilities. The convergence of the proposed learning
framework is proved, and the lowest expected delay that the IoT devices can
achieve using this learning framework is derived. Furthermore, the
effectiveness of the proposed learning algorithm in IoT networks with different
delay targets, network densities, probabilities of detection, and memory sizes
is analyzed in terms of the probability of a successful random access request
and percentage of devices that learned correctly. The results show that the
proposed learning algorithm is very effective in reducing the delay of urgent,
critical messages by intelligently reallocating the communication resources
allocated to the delay-tolerant, periodic messages.
