  Augmented reality for laparoscopic liver resection is a visualisation mode
that allows a surgeon to localise tumours and vessels embedded within the liver
by projecting them on top of a laparoscopic image. Preoperative 3D models
extracted from CT or MRI data are registered to the intraoperative laparoscopic
images during this process. In terms of 3D-2D fusion, most of the algorithms
make use of anatomical landmarks to guide registration. These landmarks include
the liver's inferior ridge, the falciform ligament, and the occluding contours.
They are usually marked by hand in both the laparoscopic image and the 3D
model, which is time-consuming and may contain errors if done by a
non-experienced user. Therefore, there is a need to automate this process so
that augmented reality can be used effectively in the operating room. We
present the Preoperative-to-Intraoperative Laparoscopic Fusion Challenge
(P2ILF), held during the Medical Imaging and Computer Assisted Interventions
(MICCAI 2022) conference, which investigates the possibilities of detecting
these landmarks automatically and using them in registration. The challenge was
divided into two tasks: 1) A 2D and 3D landmark detection task and 2) a 3D-2D
registration task. The teams were provided with training data consisting of 167
laparoscopic images and 9 preoperative 3D models from 9 patients, with the
corresponding 2D and 3D landmark annotations. A total of 6 teams from 4
countries participated, whose proposed methods were evaluated on 16 images and
two preoperative 3D models from two patients. All the teams proposed deep
learning-based methods for the 2D and 3D landmark segmentation tasks and
differentiable rendering-based methods for the registration task. Based on the
experimental outcomes, we propose three key hypotheses that determine current
limitations and future directions for research in this domain.
