  Prompt Engineering has garnered significant attention for enhancing the
performance of large language models across a multitude of tasks. Techniques
such as the Chain-of-Thought not only bolster task performance but also
delineate a clear trajectory of reasoning steps, offering a tangible form of
explanation for the audience. Prior works on interpretability assess the
reasoning chains yielded by Chain-of-Thought solely along a singular axis,
namely faithfulness. We present a comprehensive and multifaceted evaluation of
interpretability, examining not only faithfulness but also robustness and
utility across multiple commonsense reasoning benchmarks. Likewise, our
investigation is not confined to a single prompting technique; it expansively
covers a multitude of prevalent prompting techniques employed in large language
models, thereby ensuring a wide-ranging and exhaustive evaluation. In addition,
we introduce a simple interpretability alignment technique, termed
Self-Entailment-Alignment Chain-of-thought, that yields more than 70\%
improvements across multiple dimensions of interpretability. Code is available
at https://github.com/SenticNet/CoT_interpretability
