  Tensor network (TN) representation is a powerful technique for computer
vision and machine learning. TN structure search (TN-SS) aims to search for a
customized structure to achieve a compact representation, which is a
challenging NP-hard problem. Recent "sampling-evaluation"-based methods require
sampling an extensive collection of structures and evaluating them one by one,
resulting in prohibitively high computational costs. To address this issue, we
propose a novel TN paradigm, named SVD-inspired TN decomposition (SVDinsTN),
which allows us to efficiently solve the TN-SS problem from a regularized
modeling perspective, eliminating the repeated structure evaluations. To be
specific, by inserting a diagonal factor for each edge of the fully-connected
TN, SVDinsTN allows us to calculate TN cores and diagonal factors
simultaneously, with the factor sparsity revealing a compact TN structure. In
theory, we prove a convergence guarantee for the proposed method. Experimental
results demonstrate that the proposed method achieves approximately 100 to 1000
times acceleration compared to the state-of-the-art TN-SS methods while
maintaining a comparable level of representation ability.
