  Automatic organization of personal photos is a problem with many real world
ap- plications, and can be divided into two main tasks: recognizing the event
type of the photo collection, and selecting interesting images from the
collection. In this paper, we attempt to simultaneously solve both tasks:
album-wise event recognition and image- wise importance prediction. We
collected an album dataset with both event type labels and image importance
labels, refined from an existing CUFED dataset. We propose a hybrid system
consisting of three parts: A siamese network-based event-specific image
importance prediction, a Convolutional Neural Network (CNN) that recognizes the
event type, and a Long Short-Term Memory (LSTM)-based sequence level event
recognizer. We propose an iterative updating procedure for event type and image
importance score prediction. We experimentally verified that image importance
score prediction and event type recognition can each help the performance of
the other.
