  Transcription of legal proceedings is very important to enable access to
justice. However, speech transcription is an expensive and slow process. In
this paper we describe part of a combined research and industrial project for
building an automated transcription tool designed specifically for the Justice
sector in the UK. We explain the challenges involved in transcribing court room
hearings and the Natural Language Processing (NLP) techniques we employ to
tackle these challenges. We will show that fine-tuning a generic off-the-shelf
pre-trained Automatic Speech Recognition (ASR) system with an in-domain
language model as well as infusing common phrases extracted with a collocation
detection model can improve not only the Word Error Rate (WER) of the
transcribed hearings but avoid critical errors that are specific of the legal
jargon and terminology commonly used in British courts.
