  Consumer Internet of Things (IoT) devices are increasingly common, from smart
speakers to security cameras, in homes. Along with their benefits come
potential privacy and security threats. To limit these threats a number of
commercial services have become available (IoT safeguards). The safeguards
claim to provide protection against IoT privacy risks and security threats.
However, the effectiveness and the associated privacy risks of these safeguards
remains a key open question. In this paper, we investigate the threat detection
capabilities of IoT safeguards for the first time. We develop and release an
approach for automated safeguards experimentation to reveal their response to
common security threats and privacy risks. We perform thousands of automated
experiments using popular commercial IoT safeguards when deployed in a large
IoT testbed. Our results indicate not only that these devices may be
ineffective in preventing risks, but also their cloud interactions and data
collection operations may introduce privacy risks for the households that adopt
them.
