  Most of the successful deep neural network architectures are structured,
often consisting of elements like convolutional neural networks and gated
recurrent neural networks. Recently, graph neural networks have been
successfully applied to graph structured data such as point cloud and molecular
data. These networks often only consider pairwise dependencies, as they operate
on a graph structure. We generalize the graph neural network into a factor
graph neural network (FGNN) in order to capture higher order dependencies. We
show that FGNN is able to represent Max-Product Belief Propagation, an
approximate inference algorithm on probabilistic graphical models; hence it is
able to do well when Max-Product does well. Promising results on both synthetic
and real datasets demonstrate the effectiveness of the proposed model.
