  The Agent Conversation Reasoning Engine (ACRE) is intended to aid agent
developers to improve the management and reliability of agent communication. To
evaluate its effectiveness, a problem scenario was created that could be used
to compare code written with and without the use of ACRE by groups of test
subjects.
  This paper describes the requirements that the evaluation scenario was
intended to meet and how these motivated the design of the problem. Two
experiments were conducted with two separate sets of students and their
solutions were analysed using a combination of simple objective metrics and
subjective analysis. The analysis suggested that ACRE by default prevents some
common problems arising that would limit the reliability and extensibility of
conversation-handling code.
  As ACRE has to date been integrated only with the Agent Factory multi agent
framework, it was necessary to verify that the problems identified are not
unique to that platform. Thus a comparison was made with best practice
communication code written for the Jason platform, in order to demonstrate the
wider applicability of a system such as ACRE.
