  The success of language models in code assistance has spurred the proposal of
repository-level code completion as a means to enhance prediction accuracy,
utilizing the context from the entire codebase. However, this amplified context
can inadvertently increase inference latency, potentially undermining the
developer experience and deterring tool adoption - a challenge we termed the
Context-Latency Conundrum. This paper introduces REPOFUSE, a pioneering
solution designed to enhance repository-level code completion without the
latency trade-off. REPOFUSE uniquely fuses two types of context: the analogy
context, rooted in code analogies, and the rationale context, which encompasses
in-depth semantic relationships. We propose a novel rank truncated generation
(RTG) technique that efficiently condenses these contexts into prompts with
restricted size. This enables REPOFUSE to deliver precise code completions
while maintaining inference efficiency. Through testing with the CrossCodeEval
suite, REPOFUSE has demonstrated a significant leap over existing models,
achieving a 40.90% to 59.75% increase in exact match (EM) accuracy for code
completions and a 26.8% enhancement in inference speed. Beyond experimental
validation, REPOFUSE has been integrated into the workflow of a large
enterprise, where it actively supports various coding tasks.
