  Identifying pills given their captured images under various conditions and
backgrounds has been becoming more and more essential. Several efforts have
been devoted to utilizing the deep learning-based approach to tackle the pill
recognition problem in the literature. However, due to the high similarity
between pills' appearance, misrecognition often occurs, leaving pill
recognition a challenge. To this end, in this paper, we introduce a novel
approach named PIKA that leverages external knowledge to enhance pill
recognition accuracy. Specifically, we address a practical scenario (which we
call contextual pill recognition), aiming to identify pills in a picture of a
patient's pill intake. Firstly, we propose a novel method for modeling the
implicit association between pills in the presence of an external data source,
in this case, prescriptions. Secondly, we present a walk-based graph embedding
model that transforms from the graph space to vector space and extracts
condensed relational features of the pills. Thirdly, a final framework is
provided that leverages both image-based visual and graph-based relational
features to accomplish the pill identification task. Within this framework, the
visual representation of each pill is mapped to the graph embedding space,
which is then used to execute attention over the graph representation,
resulting in a semantically-rich context vector that aids in the final
classification. To our knowledge, this is the first study to use external
prescription data to establish associations between medicines and to classify
them using this aiding information. The architecture of PIKA is lightweight and
has the flexibility to incorporate into any recognition backbones. The
experimental results show that by leveraging the external knowledge graph, PIKA
can improve the recognition accuracy from 4.8% to 34.1% in terms of F1-score,
compared to baselines.
