  Edge detection is a long standing problem in computer vision. Recent deep
learning based algorithms achieve state of-the-art performance in publicly
available datasets. Despite the efficiency of these algorithms, their
performance, however, relies heavily on the pretrained weights of the backbone
network on the ImageNet dataset. This limits heavily the design space of deep
learning based edge detectors. Whenever we want to devise a new model, we have
to train this new model on the ImageNet dataset first, and then fine tune the
model using the edge detection datasets. The comparison would be unfair
otherwise. However, it is usually not feasible for many researchers to train a
model on the ImageNet dataset due to the limited computation resources. In this
work, we study the performance that can be achieved by state-of-the-art deep
learning based edge detectors in publicly available datasets when they are
trained from scratch, and devise a new network architecture, the multi-stream
and multi scale fusion net (msmsfnet), for edge detection. We show in our
experiments that by training all models from scratch to ensure the fairness of
comparison, out model outperforms state-of-the art deep learning based edge
detectors in three publicly available datasets.
