  While high-dimensional search-by-similarity techniques reached their maturity
and in overall provide good performance, most of them are unable to cope with
very large multimedia collections. The 'big data' challenge however has to be
addressed as multimedia collections have been explosively growing and will grow
even faster than ever within the next few years. Luckily, computational
processing power has become more available to researchers due to easier access
to distributed grid infrastructures. In this paper, we show how
high-dimensional indexing and searching methods can be used on scientific grid
environments and present a scalable workflow for indexing and searching over 30
billion SIFT descriptors using a cluster running Hadoop. Besides its
scalability, the proposed scheme not only provides good search quality, but
also achieves a stable throughput of around 210ms per image when searching a
100M image collection. Our findings could help other researchers and
practitioners to cope with huge multimedia collections.
