  Exploiting big data knowledge on small devices will pave the way for building
truly cognitive Internet of Things (IoT) systems. Although machine learning has
led to great advancements for IoT-based data analytics, there remains a huge
methodological gap for the deployment phase of trained machine learning models.
For given resource-constrained platforms such as Microcontroller Units (MCUs),
model choice and parametrization are typically performed based on heuristics or
analytical models. However, these approaches are only able to provide rough
estimates of the required system resources as they do not consider the
interplay of hardware, compiler specific optimizations, and code dependencies.
In this paper, we present the novel open source framework LIghtweight Machine
learning for IoT Systems (LIMITS), which applies a platform-in-the-loop
approach explicitly considering the actual compilation toolchain of the target
IoT platform. LIMITS focuses on high level tasks such as experiment automation,
platform-specific code generation, and sweet spot determination. The solid
foundations of validated low-level model implementations are provided by the
coupled well-established data analysis framework Waikato Environment for
Knowledge Analysis (WEKA). We apply and validate LIMITS in two case studies
focusing on cellular data rate prediction and radio-based vehicle
classification, where we compare different learning models and real world IoT
platforms with memory constraints from 16 kB to 4 MB and demonstrate its
potential to catalyze the development of machine learning enabled IoT systems.
