  We look at the internal structure of neural networks which is usually treated
as a black box. The easiest and the most comprehensible thing to do is to look
at a binary classification and try to understand the approach a neural network
takes. We review the significance of different activation functions, types of
network architectures associated to them, and some empirical data. We find some
interesting observations and a possibility to build upon the ideas to verify
the process for real datasets. We suggest some possible experiments to look
forward to in three different directions.
