  Existing language models such as n-grams for software code often fail to
capture a long context where dependent code elements scatter far apart. In this
paper, we propose a novel approach to build a language model for software code
to address this particular issue. Our language model, partly inspired by human
memory, is built upon the powerful deep learning-based Long Short Term Memory
architecture that is capable of learning long-term dependencies which occur
frequently in software code. Results from our intrinsic evaluation on a corpus
of Java projects have demonstrated the effectiveness of our language model.
This work contributes to realizing our vision for DeepSoft, an end-to-end,
generic deep learning-based framework for modeling software and its development
process.
