  While pre-trained language models (LM) for code have achieved great success
in code completion, they generate code conditioned only on the contents within
the file, i.e., in-file context, but ignore the rich semantics in other files
within the same project, i.e., cross-file context, a critical source of
information that is especially useful in modern modular software development.
Such overlooking constrains code language models' capacity in code completion,
leading to unexpected behaviors such as generating hallucinated class member
functions or function calls with unexpected arguments. In this work, we develop
a cross-file context finder tool, CCFINDER, that effectively locates and
retrieves the most relevant cross-file context. We propose CoCoMIC, a framework
that incorporates cross-file context to learn the in-file and cross-file
context jointly on top of pretrained code LMs. CoCoMIC successfully improves
the existing code LM with a 33.94% relative increase in exact match and a
28.69% relative increase in identifier matching for code completion when the
cross-file context is provided.
