  Deep neural networks (DNNs) have been widely deployed across diverse domains
such as computer vision and natural language processing. However, the
impressive accomplishments of DNNs have been realized alongside extensive
computational demands, thereby impeding their applicability on
resource-constrained devices. To address this challenge, many researchers have
been focusing on basic neuron structures, the fundamental building blocks of
neural networks, to alleviate the computational and storage cost. In this work,
an efficient quadratic neuron architecture distinguished by its enhanced
utilization of second-order computational information is introduced. By virtue
of their better expressivity, DNNs employing the proposed quadratic neurons can
attain similar accuracy with fewer neurons and computational cost. Experimental
results have demonstrated that the proposed quadratic neuron structure exhibits
superior computational and storage efficiency across various tasks when
compared with both linear and non-linear neurons in prior work.
