  Visual grounding (VG) occupies a pivotal position in multi-modality
vision-language models. In this study, we propose ViLaM, a large multi-modality
model, that supports multi-tasks of VG using the cycle training strategy, with
abundant interaction instructions. The cycle training between referring
expression generation (REG) and referring expression comprehension (REC) is
introduced. It enhances the consistency between visual location and referring
expressions, and addresses the need for high-quality, multi-tasks VG datasets.
Moreover, multi-tasks of VG are promoted in our model, contributed by the cycle
training strategy. The multi-tasks in REC encompass a range of granularities,
from region-level to pixel-level, which include referring bbox detection,
referring keypoints detection, and referring image segmentation. In REG,
referring region classification determines the fine-grained category of the
target, while referring region captioning generates a comprehensive
description. Meanwhile, all tasks participate in the joint training,
synergistically enhancing one another and collectively improving the overall
performance of the model. Furthermore, leveraging the capabilities of large
language models, ViLaM extends a wide range of instructions, thereby
significantly enhancing its generalization and interaction potentials.
Extensive public datasets corroborate the superior capabilities of our model in
VG with muti-tasks. Additionally, validating its robust generalization, ViLaM
is validated under open-set and few-shot scenarios. Especially in the medical
field, our model demonstrates cross-domain robust generalization capabilities.
Furthermore, we contribute a VG dataset, especially with multi-tasks. To
support and encourage the community focused on VG, we have made both the
dataset and our code public: https://github.com/AnonymGiant/ViLaM.
