  We investigate the problem of Chinese Grammatical Error Correction (CGEC) and
present a new framework named Tail-to-Tail (\textbf{TtT}) non-autoregressive
sequence prediction to address the deep issues hidden in CGEC. Considering that
most tokens are correct and can be conveyed directly from source to target, and
the error positions can be estimated and corrected based on the bidirectional
context information, thus we employ a BERT-initialized Transformer Encoder as
the backbone model to conduct information modeling and conveying. Considering
that only relying on the same position substitution cannot handle the
variable-length correction cases, various operations such substitution,
deletion, insertion, and local paraphrasing are required jointly. Therefore, a
Conditional Random Fields (CRF) layer is stacked on the up tail to conduct
non-autoregressive sequence prediction by modeling the token dependencies.
Since most tokens are correct and easily to be predicted/conveyed to the
target, then the models may suffer from a severe class imbalance issue. To
alleviate this problem, focal loss penalty strategies are integrated into the
loss functions. Moreover, besides the typical fix-length error correction
datasets, we also construct a variable-length corpus to conduct experiments.
Experimental results on standard datasets, especially on the variable-length
datasets, demonstrate the effectiveness of TtT in terms of sentence-level
Accuracy, Precision, Recall, and F1-Measure on tasks of error Detection and
Correction.
