  Deep networks have become very popular over the past few years. The main
reason for this widespread use is their excellent ability to learn and predict
knowledge in a very easy and efficient way. Convolutional neural networks and
auto-encoders have become the normal in the area of imaging and computer vision
achieving unprecedented accuracy levels in many applications. The most common
strategy is to build and train networks with many layers by tuning their
hyper-parameters. While this approach has proven to be a successful way to
build robust deep learning schemes it suffers from high complexity. In this
paper we introduce a module that learns color space transformations within a
network. Given a large dataset of colored images the color space transformation
module tries to learn color space transformations that increase overall
classification accuracy. This module has shown to increase overall accuracy for
the same network design and to achieve faster convergence. It is part of a
broader family of image transformations (e.g. spatial transformer network).
