  With the rise of individual and collaborative networks of autonomous agents,
AI is deployed in more key reasoning and decision-making roles. For this
reason, ethics-based audits play a pivotal role in the rapidly growing fields
of AI safety and regulation. This paper undertakes an ethics-based audit to
probe the 8 leading commercial and open-source Large Language Models including
GPT-4. We assess explicability and trustworthiness by a) establishing how well
different models engage in moral reasoning and b) comparing normative values
underlying models as ethical frameworks. We employ an experimental,
evidence-based approach that challenges the models with ethical dilemmas in
order to probe human-AI alignment. The ethical scenarios are designed to
require a decision in which the particulars of the situation may or may not
necessitate deviating from normative ethical principles. A sophisticated
ethical framework was consistently elicited in one model, GPT-4. Nonetheless,
troubling findings include underlying normative frameworks with clear bias
towards particular cultural norms. Many models also exhibit disturbing
authoritarian tendencies. Code is available at
https://github.com/jonchun/llm-sota-chatbots-ethics-based-audit.
