Created on 02-27-201805:57 PM - edited 08-17-201908:42 AM
Keeping track of where a cat is can be a tricky task. In this article, we'll design and prototype a smart IoT cat sensor which detects when a cat is in proximity. This sensor is meant to be part of a larger network of cat sensors covering a target space.
We start by polling an image sensor (camera) for image data. We use the GetUSBCamera processor configured for the USB camera device attached to our sensor controller.
On our system, we had to open up permissions on the USB camera device, otherwise MiNiFi's GetUSBCamera processor would record an access denied error in the logs:
chown user /dev/bus/usb/001/014
We then configure the sensor processor as such:
- name: Get
USB Vendor ID: 0x045e
USB Product ID: 0x0779
Machine Learning Inference on the Edge
We use TensorFlow to perform class inference on the image data. We do this at the sensor rather than in a centralized system in order to significantly reduce inference latency and network bandwidth consumption. This is a three-step process:
Convert image data to a tensor using TFConvertImageToTensor
We must perform some preliminary steps to get the NASNet graph into form that MiNiFi can use. First, we export the inference graph using the export_inference_graph.py script from TensorFlow models research/slim:
For the purposes of this prototype, we'll use RouteOnAttribute in conjunction with the NiFi Expression Language forwarded to an ExecuteProcess using notify-send to notify us of a CAT_DETECTED event. In a production system, we may want to use Remote Processing Groups to forward data of interest to a centralzed system.
MiNiFi - C++ makes it easy to create an IoT cat sensor. To complete our cat tracking system, we simply need to deploy a network of these sensors in the target space and configure the flow to deliver inferences to a centralized NiFi instance for storage and further analysis. We might also consider combining the image data with other data such as GPS sensor data using the GetGPS processor.