  Having a comprehensive, high-quality dataset of road sign annotation is
critical to the success of AI-based Road Sign Recognition (RSR) systems. In
practice, annotators often face difficulties in learning road sign systems of
different countries; hence, the tasks are often time-consuming and produce poor
results. We propose a novel approach using knowledge graphs and a machine
learning algorithm - variational prototyping-encoder (VPE) - to assist human
annotators in classifying road signs effectively. Annotators can query the Road
Sign Knowledge Graph using visual attributes and receive closest matching
candidates suggested by the VPE model. The VPE model uses the candidates from
the knowledge graph and a real sign image patch as inputs. We show that our
knowledge graph approach can reduce sign search space by 98.9%. Furthermore,
with VPE, our system can propose the correct single candidate for 75% of signs
in the tested datasets, eliminating the human search effort entirely in those
cases.
