  Automated decision-making systems are becoming increasingly ubiquitous,
motivating an immediate need for their explainability. However, it remains
unclear whether users know what insights an explanation offers and, more
importantly, what information it lacks. We conducted an online study with 200
participants to assess explainees' ability to realise known and unknown
information for four representative explanations: transparent modelling,
decision boundary visualisation, counterfactual explainability and feature
importance. Our findings demonstrate that feature importance and decision
boundary visualisation are the most comprehensible, but their limitations are
not necessarily recognised by the users. In addition, correct interpretation of
an explanation -- i.e., understanding known information -- is accompanied by
high confidence, but a failure to gauge its limits -- thus grasp unknown
information -- yields overconfidence; the latter phenomenon is especially
prominent for feature importance and transparent modelling. Machine learning
explanations should therefore embrace their richness and limitations to
maximise understanding and curb misinterpretation.
