  With the rapid growth of Internet applications, sequential prediction in
collaborative filtering has become an emerging and crucial task. Given the
behavioral history of a specific user, predicting his or her next choice plays
a key role in improving various online services. Meanwhile, there are more and
more scenarios with multiple types of behaviors, while existing works mainly
study sequences with a single type of behavior. As a widely used approach,
Markov chain based models are based on a strong independence assumption. As two
classical neural network methods for modeling sequences, recurrent neural
networks cannot well model short-term contexts, and the log-bilinear model is
not suitable for long-term contexts. In this paper, we propose a Recurrent
Log-BiLinear (RLBL) model. It can model multiple types of behaviors in
historical sequences with behavior-specific transition matrices. RLBL applies a
recurrent structure for modeling long-term contexts. It models several items in
each hidden layer and employs position-specific transition matrices for
modeling short-term contexts. Moreover, considering continuous time difference
in behavioral history is a key factor for dynamic prediction, we further extend
RLBL and replace position-specific transition matrices with time-specific
transition matrices, and accordingly propose a Time-Aware Recurrent
Log-BiLinear (TA-RLBL) model. Experimental results show that the proposed RLBL
model and TA-RLBL model yield significant improvements over the competitive
compared methods on three datasets, i.e., Movielens-1M dataset, Global
Terrorism Database and Tmall dataset with different numbers of behavior types.
