  Over recent decades, significant advancements in cross-modal retrieval are
mainly driven by breakthroughs in visual and linguistic modeling. However, a
recent study shows that multi-modal data representations tend to cluster within
a limited convex cone (as representation degeneration problem), which hinders
retrieval performance due to the inseparability of these representations. In
our study, we first empirically validate the presence of the representation
degeneration problem across multiple cross-modal benchmarks and methods. Next,
to address it, we introduce a novel method, called InvGC, a post-processing
technique inspired by graph convolution and average pooling. Specifically,
InvGC defines the graph topology within the datasets and then applies graph
convolution in a subtractive manner. This method effectively separates
representations by increasing the distances between data points. To improve the
efficiency and effectiveness of InvGC, we propose an advanced graph topology,
LocalAdj, which only aims to increase the distances between each data point and
its nearest neighbors. To understand why InvGC works, we present a detailed
theoretical analysis, proving that the lower bound of recall will be improved
after deploying InvGC. Extensive empirical results show that InvGC and InvGC
w/LocalAdj significantly mitigate the representation degeneration problem,
thereby enhancing retrieval performance.
  Our code is available at
https://github.com/yimuwangcs/Better_Cross_Modal_Retrieval
