  Large language models (LLMs) have demonstrated remarkable capacities on
various tasks, and integrating the capacities of LLMs into the Internet of
Things (IoT) applications has drawn much research attention recently. Due to
security concerns, many institutions avoid accessing state-of-the-art
commercial LLM services, requiring the deployment and utilization of
open-source LLMs in a local network setting. However, open-source LLMs usually
have more limitations regarding their performance, such as their arithmetic
calculation and reasoning capacities, and practical systems of applying LLMs to
IoT have yet to be well-explored. Therefore, we propose a text-based generative
IoT (GIoT) system deployed in the local network setting in this study. To
alleviate the limitations of LLMs and provide service with competitive
performance, we apply prompt engineering methods to enhance the capacities of
the open-source LLMs, design a Prompt Management Module and a Post-processing
Module to manage the tailored prompts for different tasks and process the
results generated by the LLMs. To demonstrate the effectiveness of the proposed
system, we discuss a challenging Table Question Answering (Table-QA) task as a
case study of the proposed system, as tabular data is usually more challenging
than plain text because of their complex structures, heterogeneous data types
and sometimes huge sizes. We conduct comprehensive experiments on two popular
Table-QA datasets, and the results show that our proposal can achieve
competitive performance compared with state-of-the-art LLMs, demonstrating that
the proposed LLM-based GIoT system can provide competitive performance with
tailored prompting methods and is easily extensible to new tasks without
training.
