  Normalized graph cut (NGC) has become a popular research topic due to its
wide applications in a large variety of areas like machine learning and very
large scale integration (VLSI) circuit design. Most of traditional NGC methods
are based on pairwise relationships (similarities). However, in real-world
applications relationships among the vertices (objects) may be more complex
than pairwise, which are typically represented as hyperedges in hypergraphs.
Thus, normalized hypergraph cut (NHC) has attracted more and more attention.
Existing NHC methods cannot achieve satisfactory performance in real
applications. In this paper, we propose a novel relaxation approach, which is
called relaxed NHC (RNHC), to solve the NHC problem. Our model is defined as an
optimization problem on the Stiefel manifold. To solve this problem, we resort
to the Cayley transformation to devise a feasible learning algorithm.
Experimental results on a set of large hypergraph benchmarks for clustering and
partitioning in VLSI domain show that RNHC can outperform the state-of-the-art
methods.
