  Domestic service robots (DSRs) are a promising solution to the shortage of
home care workers. However, one of the main limitations of DSRs is their
inability to interact naturally through language. Recently, data-driven
approaches have been shown to be effective for tackling this limitation;
however, they often require large-scale datasets, which is costly. Based on
this background, we aim to perform automatic sentence generation of fetching
instructions: for example, "Bring me a green tea bottle on the table." This is
particularly challenging because appropriate expressions depend on the target
object, as well as its surroundings. In this paper, we propose the attention
branch encoder--decoder network (ABEN), to generate sentences from visual
inputs. Unlike other approaches, the ABEN has multimodal attention branches
that use subword-level attention and generate sentences based on subword
embeddings. In experiments, we compared the ABEN with a baseline method using
four standard metrics in image captioning. Results show that the ABEN
outperformed the baseline in terms of these metrics.
