  We introduce an electro-optic hardware platform for nonlinear activation
functions in optical neural networks. The optical-to-optical nonlinearity
operates by converting a small portion of the input optical signal into an
analog electric signal, which is used to intensity-modulate the original
optical signal with no reduction in processing speed. Our scheme allows for
complete nonlinear on-off contrast in transmission at relatively low optical
power thresholds and eliminates the requirement of having additional optical
sources between each layer of the network. Moreover, the activation function is
reconfigurable via electrical bias, allowing it to be programmed or trained to
synthesize a variety of nonlinear responses. Using numerical simulations, we
demonstrate that this activation function significantly improves the
expressiveness of optical neural networks, allowing them to perform well on two
benchmark machine learning tasks: learning a multi-input exclusive-OR (XOR)
logic function and classification of images of handwritten numbers from the
MNIST dataset. The addition of the nonlinear activation function improves test
accuracy on the MNIST task from 85% to 94%.
