  Legal professionals often grapple with navigating lengthy legal judgements to
pinpoint information that directly address their queries. This paper focus on
this task of extracting relevant paragraphs from legal judgements based on the
query. We construct a specialized dataset for this task from the European Court
of Human Rights (ECtHR) using the case law guides. We assess the performance of
current retrieval models in a zero-shot way and also establish fine-tuning
benchmarks using various models. The results highlight the significant gap
between fine-tuned and zero-shot performance, emphasizing the challenge of
handling distribution shift in the legal domain. We notice that the legal
pre-training handles distribution shift on the corpus side but still struggles
on query side distribution shift, with unseen legal queries. We also explore
various Parameter Efficient Fine-Tuning (PEFT) methods to evaluate their
practicality within the context of information retrieval, shedding light on the
effectiveness of different PEFT methods across diverse configurations with
pre-training and model architectures influencing the choice of PEFT method.
