  Deep neural network (DNN) typically involves convolutions, pooling, and
activation function. Due to the growing concern about privacy,
privacy-preserving DNN becomes a hot research topic. Generally, the convolution
and pooling operations can be supported by additive homomorphic and secure
comparison, but the secure implementation of activation functions is not so
straightforward for the requirements of accuracy and efficiency, especially for
the non-linear ones such as exponential, sigmoid, and tanh functions. This
paper pays a special attention to the implementation of such non-linear
functions in semi-honest model with two-party settings, for which SIRNN is the
current state-of-the-art. Different from previous works, we proposed improved
implementations for these functions by using their intrinsic features as well
as worthy tiny tricks. At first, we propose a novel and efficient protocol for
exponential function by using a divide-and-conquer strategy with most of the
computations executed locally. Exponential protocol is widely used in machine
learning tasks such as Poisson regression, and is also a key component of
sigmoid and tanh functions. Next, we take advantage of the symmetry of sigmoid
and Tanh, and fine-tune the inputs to reduce the 2PC building blocks, which
helps to save overhead and improve performance. As a result, we implement these
functions with fewer fundamental building blocks. The comprehensive evaluations
show that our protocols achieve state-of-the-art precision while reducing
run-time by approximately 57%, 44%, and 42% for exponential (with only negative
inputs), sigmoid, and Tanh functions, respectively.
