  Traditional sentence embedding models encode sentences into vector
representations to capture useful properties such as the semantic similarity
between sentences. However, in addition to similarity, sentence semantics can
also be interpreted via compositional operations such as sentence fusion or
difference. It is unclear whether the compositional semantics of sentences can
be directly reflected as compositional operations in the embedding space. To
more effectively bridge the continuous embedding and discrete text spaces, we
explore the plausibility of incorporating various compositional properties into
the sentence embedding space that allows us to interpret embedding
transformations as compositional sentence operations. We propose InterSent, an
end-to-end framework for learning interpretable sentence embeddings that
supports compositional sentence operations in the embedding space. Our method
optimizes operator networks and a bottleneck encoder-decoder model to produce
meaningful and interpretable sentence embeddings. Experimental results
demonstrate that our method significantly improves the interpretability of
sentence embeddings on four textual generation tasks over existing approaches
while maintaining strong performance on traditional semantic similarity tasks.
