  Purpose: Vision-based robot tool segmentation plays a fundamental role in
surgical robots and downstream tasks. CaRTS, based on a complementary causal
model, has shown promising performance in unseen counterfactual surgical
environments in the presence of smoke, blood, etc. However, CaRTS requires over
30 iterations of optimization to converge for a single image due to limited
observability. Method: To address the above limitations, we take temporal
relation into consideration and propose a temporal causal model for robot tool
segmentation on video sequences. We design an architecture named Temporally
Constrained CaRTS (TC-CaRTS). TC-CaRTS has three novel modules to complement
CaRTS - temporal optimization pipeline, kinematics correction network, and
spatial-temporal regularization. Results: Experiment results show that TC-CaRTS
requires much fewer iterations to achieve the same or better performance as
CaRTS. TC- CaRTS also has the same or better performance in different domains
compared to CaRTS. All three modules are proven to be effective. Conclusion: We
propose TC-CaRTS, which takes advantage of temporal constraints as additional
observability. We show that TC-CaRTS outperforms prior work in the robot tool
segmentation task with improved convergence speed on test datasets from
different domains.
