  The effectiveness of Large Language Models (LLMs) in legal reasoning is often
limited due to the unique legal terminologies and the necessity for highly
specialized knowledge. These limitations highlight the need for high-quality
data tailored for complex legal reasoning tasks. This paper introduces
LEGALSEMI, a benchmark specifically curated for legal scenario analysis.
LEGALSEMI comprises 54 legal scenarios, each rigorously annotated by legal
experts, based on the comprehensive IRAC (Issue, Rule, Application, Conclusion)
framework. In addition, LEGALSEMI is accompanied by a structured knowledge
graph (SKG). A series of experiments were conducted to assess the usefulness of
LEGALSEMI for IRAC analysis. The experimental results demonstrate the
effectiveness of incorporating the SKG for issue identification, rule
retrieval, application and conclusion generation using four different LLMs.
LEGALSEMI will be publicly available upon acceptance of this paper.
