  Creating a vision pipeline for different datasets to solve a computer vision
task is a complex and time consuming process. Currently, these pipelines are
developed with the help of domain experts. Moreover, there is no systematic
structure to construct a vision pipeline apart from relying on experience,
trial and error or using template-based approaches. As the search space for
choosing suitable algorithms for achieving a particular vision task is large,
human exploration for finding a good solution requires time and effort. To
address the following issues, we propose a dynamic and data-driven way to
identify an appropriate set of algorithms that would be fit for building the
vision pipeline in order to achieve the goal task. We introduce a Transformer
Architecture complemented with Deep Reinforcement Learning to recommend
algorithms that can be incorporated at different stages of the vision workflow.
This system is both robust and adaptive to dynamic changes in the environment.
Experimental results further show that our method also generalizes well to
recommend algorithms that have not been used while training and hence
alleviates the need of retraining the system on a new set of algorithms
introduced during test time.
