  Being able to identify software discussions that are primarily about design,
which we call design mining, can improve documentation and maintenance of
software systems. Existing design mining approaches have good classification
performance using natural language processing (NLP) techniques, but the
conclusion stability of these approaches is generally poor. A classifier
trained on a given dataset of software projects has so far not worked well on
different artifacts or different datasets. In this study, we replicate and
synthesize these earlier results in a meta-analysis. We then apply recent work
in transfer learning for NLP to the problem of design mining. However, for our
datasets, these deep transfer learning classifiers perform no better than less
complex classifiers. We conclude by discussing some reasons behind the transfer
learning approach to design mining.
