---
abstract: 'By introducing a small set of additional parameters, a [*probe*]{} learns to solve specific linguistic tasks (e.g., dependency parsing) in a supervised manner using feature representations (e.g., contextualized embeddings). The effectiveness of such [*probing tasks*]{} is taken as evidence that the pre-trained model encodes linguistic knowledge. However, this approach of evaluating a language model is undermined by the uncertainty of the amount of knowledge that is learned by the probe itself. Complementary to those works, we propose a parameter-free probing technique for analyzing pre-trained language models (e.g., BERT). Our method does not require direct supervision from the probing tasks, nor do we introduce additional parameters to the probing process. Our experiments on BERT show that syntactic trees recovered from BERT using our method are significantly better than linguistically-uninformed baselines. We further feed the empirically induced dependency structures into a downstream sentiment classification task and find its improvement compatible with or even superior to a human-designed dependency schema.'
author:
- |
    Zhiyong Wu$^1$, Yun Chen$^2$, Ben Kao$^1$, Qun Liu$^3$\
    $^1$The University of Hong Kong, Hong Kong, China\
    $^2$Shanghai University of Finance and Economics, Shanghai, China\
    $^3$Huawei Noah’s Ark Lab, Hong Kong, China\
    {zywu,kao}@cs.hku.hk, yunchen@sufe.edu.cn, qun.liu@huawei.com\
bibliography:
- 'acl2019.bib'
title: 'Perturbed Masking: Parameter-free Probing for Analyzing and Interpreting BERT'
---

Discussion & Conclusion
=======================

One concern shared by our reviewers is that performance of our probes are underwhelming: the induced trees are barely closer to linguist-defined trees than simple baselines (e.g., rightbranching) and are even worse in the case of discourse parsing. However, this does not mean that supervised probes are wrong or that BERT captures less syntax than we thought. In fact, there is actually no guarantee that our probe will find a strong correlation with human-designed syntax, since we do not introduce the human-designed syntax as supervision. What we found is the “natural” syntax inherent in BERT, which is acquired from self-supervised learning on plain text. We would rather say our probe complements the supervised probing findings in two ways. First, it provides a lower-bound (on the unsupervised syntactic parsing ability of BERT). By improving this lower-bound, we could uncover more “accurate” information to support supervised probes’ findings. Second, we show that when combined with a down-stream application (sec \[sec:sentiment\]), the syntax learned by BERT might be empirically helpful despite not totally identical to the human design.

In summary, we propose a parameter-free probing technique to complement current line of work on interpreting BERT through probes. With carefully designed two-stage perturbation, we obtain impact matrices from BERT. This matrix mirrors the function of attention mechanism that captures inter-word correlations, except that it emerges through the output of BERT model, instead of from intermediate representations. We devise algorithms to extract syntactic trees from this matrix. Our results reinforce those of [@Hewitt2019; @liu2019linguistic; @Jawahar2019; @Tenney2019; @tenney2019bert] who demonstrated that BERT encodes rich syntactic properties. We also extend our method to probe document structure, which sheds lights on BERT’s effectiveness in modeling long sequences. Finally, we find that feeding the empirically induced dependency structures into a downstream system [@zhang2019aspect] can further improve its accuracy. The improvement is compatible with or even superior to a human-designed dependency schema. This offers an insight into BERT’s success in downstream tasks. We leave it for future work to use our technique to test other linguistic properties (e.g., coreference) and to extend our study to more downstream tasks and systems.

Acknowledgement
===============

We would like to thank Dr.Lingpeng Kong from DeepMind for his constructive feedback of the paper. This research is supported by Hong Kong Research Grant Council GRF grants 17254016.
