  With the massive increase in the number of IoT devices being deployed in
smart cities, it becomes paramount for middlewares to be able to handle very
high loads and support demanding use-cases. In order to do so, middlewares must
scale horizontally while providing a commensurate increase in availability and
throughput. Currently, most open-source IoT middlewares do not provide
out-of-the-box support for scaling horizontally. In this paper, we present
"Vermillion'', a scalable, secure and open-source IoT middleware for smart
cities which provides in-built support for scaling-out. We make three
contributions in this paper. Firstly, the middleware platform itself along with
a formal process for data exchange between data producers and consumers.
Secondly, we propose the use of hash-based federation to distribute and manage
load across various message broker nodes while eliminating inter-node
synchronisation overheads. Thirdly, we discuss a case study where Vermillion
was deployed in a city and briefly discuss about deployment considerations
using the obtained results.
