A Neuroevolutionary Alternative for Learning Features without Supervision
Paul A.~Szerlip
Gregory Morse
Justin K.~Pugh
and Kenneth O.~Stanley
Department EECS (Computer Science Divison)
University of Central Florida
Orlando
FL 32816 USA
\{pszerlip
jpugh
kstanley\
%By learning to reconstruct their own inputs, %unsupervised training approaches such as autoencoders can learn %features that describe key dimensions of variation in the training data. %However, the benefit of such features for later learning supervised classifications %is not assured. Unlike unsupervised approaches such as autoencoders that learn to reconstruct their inputs, this paper introduces an alternative approach to unsupervised feature learning called divergent discriminative feature accumulation (DDFA) that instead continually accumulates features that make novel discriminations among the training set. Thus DDFA features are inherently discriminative from the start even though they are trained without knowledge of the ultimate classification problem. Interestingly, DDFA also continues to add new features indefinitely (so it does not depend on a hidden layer size), is not based on minimizing error, and is inherently divergent instead of convergent, thereby providing a unique direction of research for unsupervised feature learning. In this paper the quality of its learned features is demonstrated on the MNIST dataset, where its performance confirms that indeed DDFA is a viable technique for learning useful features.
arxivDDFA.tex
\documentclass[hidelinks]{article} % For LaTeX2e
\pdfoutput=1
\usepackage{nips14submit_e,times}
%\usepackage{hyperref}
\usepackage{url}
\usepackage[sort&compress]{natbib} 
%\documentstyle[nips14submit_09,times,art10]{article} % For LaTeX 2.09
\usepackage{graphicx}

%\hypersetup{
%    colorlinks=false,
%    pdfborder={0 0 0},
%}

%\input{nnmacros}

%\PassOptionsToPackage{draft}{hyperref}


\title{Unsupervised Feature Learning through Divergent Discriminative Feature Accumulation}


%\title{A Neuroevolutionary Alternative for Learning Features without Supervision}


\author{
Paul A.~Szerlip, Gregory Morse, Justin K.~Pugh, and Kenneth O.~Stanley \\
Department EECS (Computer Science Divison)\\
University of Central Florida\\
Orlando, FL 32816 USA\\
\texttt{\{pszerlip,jpugh,kstanley\}@eecs.ucf.edu, gregorymorse07@gmail.com}
}

% The \author macro works with any number of authors. There are two commands
% used to separate the names and addresses of multiple authors: \And and \AND.
%
% Using \And between authors leaves it to \LaTeX{} to determine where to break
% the lines. Using \AND forces a linebreak at that point. So, if \LaTeX{}
% puts 3 of 4 authors names on the first line, and the last on the second
% line, try using \AND instead of \And before the third author name.

\newcommand{\fix}{\marginpar{FIX}}
\newcommand{\new}{\marginpar{NEW}}

\nipsfinalcopy % Uncomment for camera-ready version

\begin{document}


\maketitle

\begin{abstract}
%By learning to reconstruct their own inputs, 
%unsupervised training approaches such as autoencoders can learn 
%features that describe key dimensions of variation in the training data.
%However, the benefit of such features for later learning supervised classifications 
%is not assured.  
Unlike unsupervised approaches such as autoencoders
that learn to reconstruct their inputs,
this paper introduces an alternative approach to 
unsupervised feature learning called \emph{divergent discriminative
feature accumulation} (DDFA)
that instead continually accumulates features that
make novel discriminations among the training set.  Thus DDFA features
are inherently discriminative from the start even though they are
trained without knowledge of the ultimate classification problem.
Interestingly, DDFA also continues to add new features indefinitely
(so it does not depend on a hidden layer size),
is not based on minimizing error, and is inherently divergent instead
of convergent, thereby providing a unique direction of research
for unsupervised feature learning.  In this paper the quality of
its learned features is demonstrated on the MNIST dataset,
where its performance confirms that indeed DDFA 
is a viable technique for learning 
useful features.                     
\end{abstract}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Introduction}

%TODO: spellcheck

%graves:nips09,ciresan:nc10,schmidhuber:agi11,ciresan:cvpr12

The increasing realization in recent years that artificial neural
networks (ANNs) can learn many layers of features
\citep{bengio:nips07,hinton:nc06,marcaurelio:nips07,ciresan:nc10} 
has reinvigorated the study of representation learning in ANNs 
\citep{bengio:ieeetpam13}.  While the beginning of this
renaissance focused on the sequential unsupervised
training of individual layers one upon 
another \citep{bengio:nips07,hinton:nc06}, the number of approaches
and variations that have proven effective at training in such
\emph{deep learning} has since 
exploded \citep{schmidhuber:agi11,bengio:ieeetpam13}.  
This explosion has in
turn raised the question of what makes a good representation,
and how it is best learned \citep{bengio:ieeetpam13}.
%While the ambitious goal of 
%a comprehensive theory of good representation remains
%unmet, a recent review from \citet{bengio:ieeetpam13} provides
%a summary of ideas drawing from much recent research in the field.
%These include priors such as smoothness, multiple 
%explanatory factors, a hierarchical organization of explanatory factors,
%natural clustering, temporal and spatial coherence, 
%sparsity, and the simplicity of factor dependencies \citep{bengio:ieeetpam13}.
%Augmenting this list,
The main contribution of this paper is to advance our understanding
of good representation learning by suggesting a new principle
for obtaining useful representations that is accompanied by
a practical algorithm embodying the principle. 

The feature representation obtained through learning
algorithms is often impacted by the nature of the training.
For example, supervised approaches 
such as stochastic gradient descent \citep{ciresan:nc10}
that aim to minimize the error in a classification problem 
in effect encourage the exclusive discovery of features that 
help to discriminate
among the target classifications.  
In contrast, unsupervised approaches, which include
both generative representations such as 
restricted Boltzmann machines (RBMs) \citep{hinton:nc06}
and autoencoders that are trained to reproduce
their inputs \citep{bengio:nips07}, yield a more 
general feature set that captures dimensions of variation
that may or may not be essential to the classification objective.
The hope of course is that such a set would nevertheless be
useful for classification in any case, and the pros and cons 
of e.g.\ generative versus discriminative features have
proven both subtle and complex \citep{ng:nips02,jaakkola:nips98}. 
Nevertheless, one benefit of unsupervised training is that it does
not require labeled data to gain traction.  

An important insight in this paper is that there is an 
unrecognized option
outside this usual unsupervised versus supervised (or generative versus
discriminative) dichotomy. 
In particular, there is an alternative kind of discriminative learning
that is unsupervised rather than supervised. 
In this proposed alternative 
approach, 
called  \emph{divergent discriminative
feature accumulation} (DDFA),
instead of searching for features constrained by
the objective of solving the discriminative classification problem,
a learning algorithm can instead attempt to \emph{collect} as many
features that discriminate strongly among training examples 
as possible, without regard to any particular classification problem.
%regardless of 
%whether any particular discriminative 
%feature is aligned with
%any particular classification problem or not.  

The approach in such unsupervised discriminative learning is thus to search 
continually for novel features that discriminate among
training examples in new ways.  Interestingly,
unlike conventional algorithms in deep
learning, such a search is explicitly 
divergent by design and therefore continues to
accumulate new features without converging.   
%During the course of the search, a candidate feature is
%added to the collection if it \emph{discriminates} 
%among examples in the training
%set significantly differently from other feature
%detectors already collected.   
%In other words the successful candidate 
%\emph{discriminates} among the examples in a new way that has
%not been observed before, 
%The implication is that 
In effect, a high-scoring feature is therefore
relevant to discriminating among the examples,
even though the ultimate discrimination problem is not known.
%The goal then is to collect as many such unique discriminators as possible. 
%The insight is that 
A comprehensive set of such 
features that discriminate
among the training set in fundamental ways is thereby suitable 
in principle for
later supervised training \emph{from those collected features} for
any particular discrimination task.
%Unlike a generative feature accumulator like a RBM, the collected
%features are retained for their ability to discriminate among examples
%rather then their ability to regenerate examples. 
This idea is intuitive in the sense that even for humans,
distinctions among experiences
can be learned before we know how we will apply such distinctions.
%The ability to aggregate such distinctions is then potentially useful 
%in the future when a prediction or classification needs to be made.
Furthermore, as the results will show, if the search gradually shifts
from simple to more complex distinctions, only a small subset
of all possible distinctions (many of which are obscure) needs to be 
discovered.

%While it may seem initially that the exponential possible distinctions 
%across a set of 
%examples foreshadows a computationally prohibitive search for a 
%set of features exponential in the size of the training set,  
%in practice if the search algorithm begins with initially simple
%feature detectors that gradually increase in complexity,
%then it will tend to discover the most salient distinctions first
%and gradually shift towards increasingly obscure distinctions.
%Thus the feature accumulating search algorithm can be naturally
%run until the added value of new features begins to
%drop (which is well before all
%possible discriminators are discovered).

In fact,                
this perspective on feature learning has the advantage 
over more conventional approaches in deep learning that
learning does not depend on a fixed a priori number
of features.
Rather, it simply continues to accumulate new features 
as long as the algorithm is run.  
Furthermore, unlike in other unsupervised approaches, the
accumulated features are known explicitly to be discriminative,
suiting them well to later discriminative learning.   
Another potentially advantageous property of such a feature accumulator
is its lack of convergence (thereby avoiding the problem of local optima), 
which stems from the fact that it is inherently divergent  
because it is not based on 
minimizing an error.
In these ways DDFA is
uniquely flexible and autonomous.   

The driving force behind the feature accumulator is the 
imperative of finding novel features.
Thus a well-suited algorithm for implementing this idea in
practice is the recent \emph{novelty search} algorithm
\citep{lehman:ecj11}, which is 
a divergent evolutionary algorithm that is rewarded
for moving \emph{away} in the search space of candidate behaviors 
(such as discriminations)
from where it has already visited to where it has not. 
%Thus while conventional evolutionary algorithms have focused
%on coverging to a solution, novelty search follows the
%insight that evolution may be put to better 
%and more natural use as
%an accumulator of innovative and novel behavioral diversity.
By accumulating features that are themselves
ANNs, novelty search in this paper enables 
DDFA.
As with other unsupervised pretraining approaches such as autoencoders,
once DDFA determines that sufficient features are collected,
a classifier is trained above them for the classification
task (through backpropagation in this paper). 
To demonstrate the potential of DDFA to collect useful features,
it is tested in this paper by collecting single-layer
features for the MNIST digital handwriting recognition 
benchmark \citep{lecun:mnist98}.
Even with the consequent two-layer shallow classifier network, its   
testing performance rivals more conventional training techniques. 
%in this paper it is tasked to collect \emph{single-layer} 
%features for the full MNIST training set.
%Then when set of classifier nodes is added on top of the collected
%features, the aggregate classifier is a two-layer ANN.
%Interestingly, while best result for ANNs of this shallow size
%and without any special training help like elastic distortions
%is 1.6\% error, DDFA achieves 1.51\% performance 
%without almost any of the usual
%regularizations or adjustments customary in deep learning.
%The implication is that the process of accumulating novel
%features indeed finds \emph{useful} features during its
%collection.

This initial proof of concept establishes the efficacy of
accumulating features as a basis for representation learning.
While the simple one-layer accumulated discriminative features
from DDFA perform well,
DDFA can conceivably improve further through layering
(e.g.\ accumulating multilayer features or
searching for novel 
features that are built above already-discovered
features) and convolution \citep{lecun:hbtnn95}, 
just like other deep learning
algorithms.  
%In effect DDFA offers a new way to think about
%feature discovery and representation learning that 
%complements existing deep learning frameworks. 
Most importantly,
based on the novel representational principle 
that discriminators make 
good features
for classification problems, DDFA opens up
a new class of learning approaches.

\section{Background}

This section reviews the two algorithms, novelty search
and HyperNEAT, that underpin the DDFA approach. 

\subsection{Novelty Search} 

The problem of \emph{collecting} novel instances of a class 
is different from the 
more familiar problem of minimizing error.  
While error minimization aims at converging towards 
minima in the search space, collecting novelty requires
diverging away from past discoveries and fanning out
across the search space in all directions that appear to lead 
towards further novelty.  
%That is,
%as soon as a novel
%discovery is made future candidates 
%are rewarded for moving \emph{away} from it in behavioral
%space.
This fanning-out process is thus
well-suited to a population-based approach that accumulates
and remembers novel discoveries to help 
push the search continually to even more novelty as it progresses.
The novelty search algorithm \citep{lehman:ecj11} 
implements such a process in practice through an evolutionary
approach, which naturally provides the population-driven context
appropriate for finding novelty.  However, it is important to note
that novelty search is unlike even traditional evolutionary 
algorithms (EAs), which themselves are usually driven to
converge to higher fitness. 
In fact, while EAs are often viewed as 
an alternative approach to optimization, their natural capacity
to \emph{diversify and collect} may instead better capture their
practical potential to contribute to problems in learning. 

The idea in novelty search is to reward candidates (by increasing
their probability of reproduction) who are behaviorally novel.  If the
candidates are ANNs as in the present study, then the word
``behaviorally'' becomes critical because it refers to what the
discovered ANNs \emph{actually do} (e.g.\ how they discriminate)
as opposed to just their underlying
genetic representations (i.e.\ genomes), which may or may not do
anything interesting.  Thus discovering novel behaviors 
requires search (as opposed to just enumerating random sets of genes),
thereby instantiating a nontrivial 
alternative to the traditional objective gradient.

This point is particularly important in the context of deep learning,
where researchers have commented on the potential 
long-term limitations of optimization gradients 
and the need for a broader and less convergent approaches for learning
representations.  For example, when discussing the
future of representation learning, \citet{bengio:slsp13} notes:

\begin{quotation}
The basic idea is that humans (and current learning algorithms) are
limited to ``local descent'' optimization methods, that make small
changes in the parameter values with the effect of reducing the
expected loss in average. This is clearly prone to the presence of
local minima, while a more global search (in the spirit of both
genetic and cultural evolution) could potentially reduce this
difficulty. 
%One hypothesis is that more abstract learning tasks
%involve more challenging optimization difficulties, which would make
%such global optimization algorithms necessary if we want computers to
%learn such abstractions from scratch.
\end{quotation}
   
Novelty search \citep{lehman:ecj11} can be viewed as an embodiment of just such 
a ``genetic evolution'' that is suited to accumulating discoveries
free from the pitfalls of ``local descent.''  In fact,
while novelty search was originally shown sometimes to 
find the objective of an optimization
problem more effectively than objective-based optimization 
\citep{lehman:ecj11}, 
%which spawned a 
%wide breadth of 
%investigations \citep{risi:ab11,
%risi:gecco09,
%lehman:gecco11,
%martinez:cec13,
%naredo:gecco13,
%kistemaker:gecco11,
%lehman:gecco10a,lehman:gecco10b,lehman:gptp11,
%soltoggio:gecco09,
%graening:ppsn10,
%woolley:gecco11,
%woolley:2012arXiv1207.6682W,
%gomes:aamas14,
%gomes:si13,
%gomes:gecco13,
%mouret:ecj12,
%krcah:isda10,
%ollion:gecco2011,
%joachimczak:alife12,
%cuccu:cec11,
%krcah:cjs10,
%goldsby:mdels10,
%doucette:gp10,
%mouret:nh11,
%morse:gecco13,risi:gecco13,
%liapis:gecco13,
%liapis:iccc13},
\citet{cully:gecco13} recently raised the intriguing 
notion of novelty search as a \emph{repertoire collector}.
That is, instead of searching for a \emph{solution} to a problem,
novelty search can collect a set of novel skills (each one a point in 
the search space) intended for later aggregation by a higher-level mechanism.
This repertoire-collecting idea aligns elegantly with 
the problem of accumulating features for deep learning, wherein each
feature detector can be viewed as a ``skill'' within the repertoire of
a classifier.

In practice, novelty search maintains an \emph{archive} of previously
novel discoveries as part of the algorithm.  Future candidates are
then compared to the archive to determine whether they too are novel.
A random sampling of candidates is entered into the archive,
which implies that more frequently-visited areas will be
more densely covered. 
%Entry into the archive is 
%The probability of entry into the archive is based on 
%difference of an individual's behavior
%(which is characterized as a vector $b$)  
%from those already in the archive.
%This decision is based on comparing the candidate to its ${k}$-nearest neighbors in behavior
%space.
Intuitively, if the
average distance to the nearest neighbors of a given behavior $b$ 
is large then it
is in a sparse area; it is in a dense region if the average distance
is small. The sparseness ${\rho}$ at point ${b}$ is given by
%\vspace{-0.20in}
\begin{eqnarray} \label{eq1}
\rho(x) = \frac{1}{k}\sum_{i=0}^k \mathrm{dist}(b,{\mu_{i}}),
%\vspace{-0.15in}
\end{eqnarray}
where ${\mu_{i}}$ is the ${i}$th-nearest neighbor of ${b}$ with
respect to the distance metric \emph{dist}, which is a
domain-dependent measure of behavioral difference between two
individuals in the search space.
The nearest neighbors calculation
must take into consideration individuals from the current population
and from the permanent archive of novel individuals.
Candidates from
more sparse regions of this behavioral search space then receive
higher novelty scores, which lead to a higher chance of reproduction.  
It is important to note that this novelty space
cannot be explored purposefully, that is, it is not known \emph{a
  priori} how to enter areas of low density just as it is not known a
priori how to construct a solution close to the objective. Thus,
moving through the space of novel behaviors requires exploration. 
The gradient of novelty is interesting in particular because novel
discoveries lead to other novel discoveries, which means that a search algorithm
following gradients of novelty is likely to make many interesting discoveries. 
%Also, because novelty is measured \emph{relative} to other individuals in
%evolution, it is driven by a coevolutionary dynamic.

Novelty search in effect runs as a regular EA wherein novelty replaces fitness
as the criterion for selection, and an expanding archive of past novel discoveries              
is maintained.  This simple idea 
%whose effectiveness has been demonstrated
%in many applications in recent years (as cited above), 
will empower DDFA in this paper to accumulate a collection of novel features.

\subsection{HyperNEAT}  

The term for algorithms that search for ANNs through an
evolutionary process is \emph{neuroevolution}
\citep{stanley:ec02,floreano:ei08}.  It is important to note
that modern neuroevolution algorithms are not like conventional EAs
based on bit strings, but instead implement a variety of sophisticated
heuristics and encodings that enable the discovery of large and
well-organized networks.  This section is designed to introduce the
particular neuroevolution algorithm (called HyperNEAT) that is
combined with novelty search to search for features in this
paper.  Because neuroevolution is an independent field 
that may be unfamiliar to many in deep learning,
this section is written to emphasize the main ideas that make it
appealing for the purpose of feature learning, without including details
that are unnecessary to understand the operation of the proposed
DDFA algorithm.  
The complete details of HyperNEAT can be found in its primary 
sources \citep{stanley:alife09,gauci:nc10,gauci:aaai08,verbancsics:jmlr10}.
%Those interested in more technical details can refer
%to Appendix \ref{appendix}.
  
In a domain like visual recognition, 
the pattern of weights in useful features 
can be expected to exhibit a degree of contiguity and perhaps regularity.
For example, it is unlikely that an entirely random pattern of largely 
unconnected pixels corresponds to a useful or interesting feature.  
It has accordingly long been
recognized in neuroevolution that entirely random perturbations of
weight patterns, which are likely to emerge for example from random
mutations, are unlikely to maintain contiguity or regularity.  While
stochastic gradient descent (SGD) algorithms at least justify their 
trajectory through the search space through descent, a completely
random perturbation of weights is arguably less principled and therefore
perhaps less effective.  Nevertheless, SGD still suffers to some extent
from the same problem that even a step that reduces error may not
maintain contiguity or regularity in the feature geometry.  
Neuroevolution algorithms have responded to this concern with
a class of representations called 
\emph{indirect encodings} \citep{stanley:alife03},
wherein the weight pattern is generated by an evolving genetic encoding
that is biased towards contiguity and regularity by design.             
That way, when a mutation is applied to a feature, the feature 
deforms in a systematic though still randomized fashion 
(figure \ref{fg:deformation}).

\begin{figure}
\begin{center}
  \begin{minipage}{1.6in}  %space of picture including whitespace
     \centering
     \includegraphics[height=0.4in]{feature_mutation/fakefeature1.pdf}

     (a) Original Weight Pattern
  \end{minipage}
  \begin{minipage}{1.6in}  %space of picture including whitespace
     \centering
     \includegraphics[height=0.4in]{feature_mutation/fakefeature2.pdf}

     (b) HyperNEAT Mutation 
  \end{minipage}
  \begin{minipage}{1.6in}  %space of picture including whitespace
     \centering
     \includegraphics[height=0.4in]{feature_mutation/fakefeature3.pdf}

     (c) Uniform Mutation
  \end{minipage}
\end{center}
%\vspace{-0.2in}
\caption{\textbf{Systematic Deformation in HyperNEAT-style Mutation.}
Each image depicts the pattern of weights projecting from a
single $28\times 28$ input field to a single output node.
The weight of a hypothetical feature (a) exhibits contiguity and 
some symmetry.  The HyperNEAT style of mutation (b) perturbs
the pattern of weights while still preserving the geometric
regularities of the original feature.  In contrast, simply
randomly mutating weights with uniform probability (c) leads
to an incoherent corruption of the original feature.   
}
\label{fg:deformation}
\end{figure}

HyperNEAT, which stands for \emph{Hypercube-based NeuroEvolution of
Augmenting 
Topologies} \citep{stanley:alife09,gauci:nc10,gauci:aaai08,verbancsics:jmlr10}
is a contemporary neuroevolution algorithm
based on such an indirect encoding.  In short, HyperNEAT evolves an
encoding network called a compositional pattern producing network
(CPPN; \citealt{stanley:gpem07}) 
that \emph{describes} the pattern of connectivity within
the ANN it encodes.
Therefore, mutations in HyperNEAT
happen to the \emph{CPPN}, 
which then transfers their effects to the
encoded ANN.  In this way the CPPN is like DNA, which transfers
the effects of its own mutations to the structures it encodes,
such as the brain.  
%Because the CPPN in HyperNEAT computes the
%pattern of weights of the substrate as a function of the
%substrate's geometry (e.g.\ taking a two-dimensional field as 
%input for a vision problem), 
Because the CPPN encoding is designed to describe patterns of
weights across the geometry of the encoded network,
the weights in HyperNEAT ANNs
tend to deform in contiguity-preserving and regularity-preserving ways
(as seen in figure \ref{fg:deformation}),
thereby providing a useful bias \citep{stanley:alife09,gauci:nc10}. 
Furthermore, CPPNs in HyperNEAT grow over evolution
(i.e.\ their structure is augmented over learning), which means that the 
pattern of weights in the ANN they describe (which is fixed in size)
can become more intricate and complex over time.
For unfamiliar readers, 
it is worth noting that this
HyperNEAT style of representation for ANNs
is well-established and has appeared in mainstream venues such as 
AAAI \citep{gauci:aaai08}, Neural Computation \citep{gauci:nc10},
and JMLR \citep{verbancsics:jmlr10}. 

%permutation issue here
An important observation is that HyperNEAT's tendency to preserve
geometric properties in its weights means that it is
not invariant to permutations in the input vector.  In effect 
(in e.g.\ the case of MNIST) it
is exploiting the known two-dimensional geometry of the problem.
However, at the same time, while it does exploit geometry, 
its use in this paper is
not convolutional either: its input field is never
broken into receptive fields and is 
rather projected in whole directly (without intervening layers)    
to a single-feature output node.  Thus the powerful advantage 
of convolution for visual problems is \emph{not available in
this investigation}, making the problem more challenging.
As a consequence, the DDFA implementation in this paper does not
fit neatly into the permutation-invariant-or-not dichotomy,
and may be considered somewhere closer to typical
permutation-invariant scenarios.        

This overview of HyperNEAT is left brief because 
its other details (which are widely disseminated in the venues above)
are not essential to the main idea in this paper, which is
to accumulate feature detectors through novelty search.  
%In principle,
%any underlying representation could be integrated with this approach
%and HyperNEAT itself is not essential;
%HyperNEAT is chosen here for this purpose because of it respect
%for geometry within the features it encodes, which is relevant 
%in a two-dimensional visual domain.  

\vspace{-0.1in}
\section{Approach: Divergent Discriminative Feature Accumulation (DDFA)}

%POSSIBLE CUTS: ISSUES WITH AUTOENCODERS
Unsupervised pretraining in deep learning has historically focused
on approaches such as autoencoders and RBMs \citep{bengio:nips07,hinton:nc06}
that attempt to \emph{reconstruct} training examples
by first translating them into a basis of features different
from the inputs, and then from those features regenerating the inputs.
This idea is appealing because the imperative of reconstruction
demands that the learned features must ultimately reflect some aspect of the
underlying structure of the training set.
Of course, one potential problem with this approach is that there
is no assurance that the learned features actually align with
any particular classification or discrimination problem for which they  
might be used in the future.    
Yet this conventional approach to learning features also raises some
interesting deeper questions.  For example, is there any \emph{other}
way to extract meaningful features and thereby learn representations
from a set of examples without explicit supervision?   
    
There are some well-known simple alternatives, though they are
not usually characterized as feature-learning algorithms.
For example clustering algorithms such as K-means or Gaussian
mixture models in effect extract structure from data that can
then assist in classification; in fact at least one study has
shown that such clustering algorithms can yield features as
effective or more so for classification than autoencoders or RBMs
\citep{coates:icais11}.  This result highlights that 
reconstruction is not the only effective incentive for extracting
useful structure from the world.  

%POSSIBLE CUTS: tree example
The approach introduced here goes beyond simple clustering by
emphasizing the general ability to learn diverse \emph{distinctions}.
That is, while one can learn how to \emph{describe} the world,
one can also learn how different aspects of the world
\emph{relate} to each other.  Importantly, there is no
single ``correct'' view of such relations.  Rather,
a rich set of learned relationships can support drawing 
important distinctions later.  For example, in one view
palm trees and ``regular'' trees share properties that distinguish them
from other plants.  However, in another view, palm trees
are in fact distinct from regular trees.  \emph{Both} such views can be useful
in understanding nature, and one can hold both simultaneously
with no contradiction.  When an appropriate question comes
up, such as which plants are tall and decorative, the feature
\emph{tall} becomes available because it was learned to
help make such general distinctions about the world 
in the past.        
  
The idea in DDFA is to continually accumulate such distinctions
systematically through novelty search, thereby building an
increasingly rich repertoire of features that help divide
and relate observations of the world.  
Specifically, suppose there are $n$ training examples
$\{x^{(1)},...,x^{(n)}\}$;
whether or not they are labeled will not matter because  
feature learning will be unsupervised.
Suppose also that any single \emph{feature detector}
$h_i$ (i.e.\ a single hidden node that detects a particular feature)
outputs a real number whose intensity represents the degree to which
that feature is present in the input.  
It follows that $h_i$ will assign a real number $h_i^{(t)}$ to
every example $x^{(t)}$ depending on the degree to which $x^{(t)}$ contains
the feature of interest for $h_i$.
The output of $h_i$ for all features $x^{(t)}$ 
where $t = 1,\dots,n$
thereby
forms a vector $\{h_i^{(1)},\dots,h_i^{(n)}\}$ that can be 
interpreted as the \emph{signature} of feature detector $h_i$
across the entire training set.  
%In a more simplified view,
%if each $h_i$ were binary, then the output of $h_i$ for all
%features $x^{(t)}$ could be thought of as a \emph{partitioning}
%of the training examples.  
In effect the aim is 
to continually discover new such signatures.

This problem of continually discovering novel signatures is naturally
captured by novelty search, which can be set up explicitly to
evolve feature detectors $h_i$, each of which takes a training
example as input and returns a single output.  The signature
$\{h_i^{(1)},...,h_i^{(n)}\}$ of $h_i$ over all training examples
is then its \emph{behavior characterization} for novelty search.
The novelty of the signature is then measured by comparing it
to the $k$-nearest signatures in the novelty archive, following
equation \ref{eq1}.  Novelty search then dictates that more novel features
are more likely to reproduce, which means that gradients of
novel signatures will be followed in parallel by the evolving
population.  Those features whose sparseness ${\rho}$
(i.e.\ novelty) exceeds a minimum threshold 
${\rho}_{\textrm{\footnotesize min}}$ 
are stored in the growing novel feature collection for later 
classifier training.
%the expanding archive of novel features.     

A likely source of confusion is the question of whether DDFA is a kind
of exhaustive search over signatures, which would not tend to 
discover \emph{useful} features in a reasonable runtime. 
After all, the number of theoretically possible distinctions is
exponential in the number of training examples.   
However, a critical facet of novelty-based searches that are combined
with HyperNEAT-based neuroevolution
is that the complexity of features (and hence distinctions) 
tends to increase 
over the 
run \citep{lehman:ecj11}.
As a result, the initial features discovered encompass
simple principles (e.g.\ is the left side of the image dark?) that
gradually increase in complexity.  For this reason, the most
arbitrary and incoherent features (e.g.\ are there 17 particular dots at 
specific non-contiguous coordinates in the image?) are possible
to discover only late in the search.  
%Furthermore, this tendency interacts with the
%with the training set itself because after all a feature's 
%novelty signature is over the 
%training set.  
Furthermore, because the novelty signature is measured over the training set,
features that make broad separations 
\emph{relevant}
to the training set itself  
are more likely to be discovered early.  In effect, over the course of DDFA,
the feature discoveries increasingly shift from simple
principles to intricate minutia.  Somewhere along this road are likely 
diminishing returns, well before all possible signatures are discovered.     
Empirical results reported here support this point.

%At any time during DDFA, the current set of features can be
%fed into a classifier that then learns the separations
%for the particular problem based on those collected
%discriminative features.  
Interestingly, because DDFA does not
depend on the minimization of error, in principle it can continue
to collect features virtually indefinitely, but in practice
at some point its features are fed into a classifier that is
trained from the collected discriminative features.

\vspace{-0.1in}
\section{Experiment}
\vspace{-0.1in} 

The key question addressed in this paper
is whether a divergent discriminative feature accumulator 
can learn \emph{useful} features, which means they should
aid in effective generalization on the test set.
If that is possible, the implication is that DDFA is a viable
alternative to other kinds of unsupervised pretraining.
To investigate this question DDFA is trained and tested on the MNIST
handwritten digit recognition dataset \citep{lecun:mnist98}, 
which consists of 60,000 training images and 10,000 test images.
Therefore, the signature of each candidate feature discovered by
DDFA during training is a vector of 60,000 real values.    

%Recall that features are being generated through novelty search
%by HyperNEAT.  
Because the structure of the networks that
are produced by HyperNEAT can include as many hidden layers as
the user chooses,
the question arises how many hidden layers should be allowed
in \emph{individual features} $h_i$ learned by HyperNEAT.
This consideration is substantive because in principle
DDFA can learn arbitrarily-deep individual features all at once,
which is unlike e.g.\ the layer-by-layer 
training of a deep stack of autoencoders.  
%Deep features can in
%principle encompass more complex concepts than shallow ones,
%which have less representational capacity.  It is also possible
%that one layer of features could be generated by DDFA that is
%then fed into a higher-level DDFA that generates higher-level features,
%similarly to the layer-by-layer training of stacked autoencoders.
However, the explicit choice was made in this introductory experiment to
limit DDFA to single-layer features (i.e.\ without hidden nodes)
to disentangle the key question of whether the DDFA process represents
a useful principle from other questions of representation such
as the implications of greater depth.  Therefore,
feature quality is addressed straightforwardly in this study by
observing the quality of classifier produced based only on 
single-layer DDFA features.  As a result, 
the final classifier ANN has just two layers: the layer
of collected features and the ten-unit output layer for classifying
MNIST digits.       
    
The single-layer DDFA approach with novelty search and HyperNEAT is 
difficult to align directly with common deep learning approaches
in part because of its lack of permutation invariance even though
it is not convolutional in any sense (thereby lacking the
representational power of such networks), 
and its lack of depth in this initial test.  
Thus to get a fair sense of whether DDFA
learns useful features it is most illuminating to contrast it with
the leading result on an equivalently shallow two-layer architecture 
(which are rare in recent years) 
that similarly avoided special preprocessing like elastic distortions 
or deskewing.
In particular,  \citet{simard:icdar03} obtained one of the best
such results of 1.6\% test error performance.  
Thus a significant improvement on that result would suggest
that DDFA generates useful features that help to stretch the capacity
of such a shallow network to generalize. 
DDFA's further ability to approach the performance of conventional vanilla deep
networks, such as the original 1.2\% result from \citet{hinton:nc06}
on a four-layer network pretrained by a RBM,
would hint at DDFA's potential utility in the future for pretraining 
deeper networks. 

%One useful consequence of restricting the experiment to collecting such
%single-layer features is that it is known that the best shallow two-layer ANNs
%was trained through gradient descent 
%(without any special preprocessing like elastic distortions or deskewing)
%to a test error of 1.6\% on MNIST \citep{simard:icdar03}, as also noted by
%\citet{hinton:nc06}.  Thus if the features of a two-layer ANN can alternatively
%be trained by DDFA to achieve a similar or better error without any
%preprocessing, then the implication is that DDFA generates features
%at least as useful or more so than the best ANN optimization technique
%reported for such a network, providing a useful baseline.  While of course
%the focus of the field and ultimately this line of research is on
%learning deeper networks, the clarity provided by first exposing the
%method in such a controlled comparison thereby provides concrete support
%for more ambitious future investigations.    
    
During the course of evolution, features are selected for reproduction
based on their signature's novelty score (sparseness $\rho$)
calculated as the sum of the distances to the k-nearest neighbors
($k=20$), where neighbors include other members of the population as
well as the historical novelty archive. At the end of each generation,
each individual in the population ($\textrm{size}=100$) has a $1\%$ chance of
being added to the novelty archive, resulting in an average of 1
individual added to the novelty archive on each generation.
Separately, a list of individuals called the \emph{feature list} is
maintained.  At the end of each generation, each member of the
population is scored against the current feature list by finding the
distance to the nearest neighbor ($k=1$), where neighbors are members
of the feature list. Those individuals that score above a threshold
${\rho}_{\textrm{\footnotesize min}} = 2{,}000$ are added to the feature list.  In effect, the
feature list is constructed in such a way that all collected features
have signatures that differ by at least ${\rho}_t$ from all others in
the collection. This threshold-based collection process protects
against collecting redundant features.
A simple variant of HyperNEAT called HyperNEAT-LEO \citep{verbancsics:gecco11}
(which leads to less connectivity) was the main neuroevolution engine.  
The HyperNEAT setup and 
parameters can be easily reproduced in full because they are simply
the default parameters of the SharpNEAT 2.0 publicly-available package
\citep{green:SharpNEAT}.
%In this HyperNEAT component, 
%mutations to the CPPN (which \emph{encodes} the fixed-sized single-layer ANN) 
%have a 0.1\% probability of adding a new node,
%a 1\% probability of adding a new connection, and a 0.1\% probability of
%deleting a connection.  Weight mutation in the CPPN is based 
%on a Gaussian perturbation with $\sigma=1.0$ half the time and
%$\sigma=0.2$ the rest of the time. 

To observe the effect of collecting different numbers of features,  
DDFA was run separately until both 1,500 and 3,000
features were collected.  After collection concludes,
a set of ten classification nodes
is added on top of the collected features, and
simple backpropagation training commences.
The training and validation procedure mirrors that followed by
\citet{hinton:nc06}: first training is run on 50,000 examples
for 50 epochs
to find the network that performs best on a 10,000-example
validation set.  Then training shifts to the full 60,000-example
set, which is trained until it reaches the same training error as
in the best validation epoch.  The resulting network is finally tested
on the full 10,000-example test set.        
This whole procedure is similar to how autoencoders are trained before
gradient descent in deep learning \citep{bengio:nips07}.

%In preliminary experiments, about 1,500 features were found sufficient for
%good performance.
%Therefore, once 1,500 features are collected, the unsupervised pretraining
%phase ends, a set of ten classification nodes are added on top of the 
%collected features, and simple backpropagation training is applied
%until convergence on the training set. 
%This procedure is similar to how autoencoders are trained before 
%gradient descent in deep learning \citep{bengio:nips07}.

% and the features are 
%While in principle any number of features could be collected, the
%list size is capped at $5{,}000$, allowing later perceptron learning
%from the feature set to be reasonably efficient.  Once the number of
%features rises above the cap, every $30$ generations, the current 
%feature set is constructed into a multiclass perceptron, which is then 
%partially trained (with learning initial rate $0.01$, decaying $10\%$ per iteration) 
%on the $60{,}000$ MNIST training images.              
%Training the classifier with a perceptron ensures that the 
%%Such partial training consists of a higher than normal learning rate (initial rate $0.01$, 
%%decay $10\%$ per iteration) and a smaller number of iterations ($25$ iterations).
%%Each collected feature is then assigned a 
%%quality according to the sum of the ten weight magnitudes 
%%linking the feature to the ten outputs in the partially trained multi-perceptron. 
%%If the feature list contains more than $5{,}000$ features, the lowest quality 
%Features with the lowest sum of outgoing weights (i.e.\ those
%contributing least to the perceptron) are then discarded to bring the
%feature list size back down to $5{,}000$.  
%The final feature set is
%then constructed into a multi-class perceptron of ten outputs and trained to
%convergence (TODO iterations) with an initial learning rate of
%$0.0008$ and a learning rate decay of $0.1\%$ per iteration.

\section{Results}


%a total of $12{,}735$ features was considered (based on
%exceeding the threshold ${\rho}_t = 1{,}000$). 
%The set of $5{,}000$ features with the highest partial
%training accuracy (99.7\% in generation 840) was saved as the final 
%feature set.  A perceptron with this set as input was then finally trained   
%to an accuracy of $100\%$ on the training set.
%The main result is that  
%the testing error rate of this final DDFA-based ANN was 1.5%, which is 
%below the best-reported score of 1.6\% on two-layer ANNs
%without preprocessing.  The conclusion is that DDFA-collected features   
%are at least as useful (if not more) 
%for generalization as those collected
%through stochastic gradient descent.


The main results are shown in Table \ref{fig:testingResults}.
DDFA was able to achieve test errors of 1.42\% and 1.25\% 
from collections of 1,500 and 3,000 features, respectively,
which are both well below the 1.6\% error of the similar 
shallow network trained without preprocessing from \citet{simard:icdar03}.
In fact, the result for the 3,000-feature network even approaches
the 1.2\% error of the significantly deeper network of \citet{hinton:nc06},
showing that shallow networks can generalize surprisingly well
by finding sufficiently high-quality feature sets, even despite 
a lack of exposure to distortions during training.    
It also appears that more collected features lead to better generalization, at
least at these sizes.  
It took 338 and 676 generations of feature collection
to obtain the 1,500 and 3,000 features, respectively.  Collecting 3,000 
features took about 36 hours of computation on 12 3.0 GHz cores.  
%parallel cluster without GPUs.

\begin{table}
\centering
    \begin{tabular}{| l | c | c | c |}
    \hline   
    \textbf{Features} & \textbf{DDFA Test Error} & \textbf{Random CPPNs Control} & \textbf{Random Weights Control} \\ \hline 
%    & \textbf{error} & \textbf{CPPNs} & \textbf{weights}\\ \hline
    \textbf{1,500} & 1.42\% & 1.63\% & 2.21\% \\ \hline
    \textbf{3,000} & \textbf{1.25\%} & 1.61\% & 2.00\% \\ \hline
    \end{tabular}
\caption{\textbf{MNIST Testing Error Rates of DDFA and Controls.}
}
\label{fig:testingResults}
\end{table}

Figure \ref{fg:features} shows a typical set of features 
collected by DDFA.  Interestingly, unlike the bottom layer of
deep learning networks that typically exhibit various line-orientation
detectors, DDFA also collects more complex features because newer features
of increasing complexity evolve from older features.   

\begin{figure}
\begin{center}
     \includegraphics[height=0.4in]{found_features/sorted/featureSample1.jpg}
     \includegraphics[height=0.4in]{found_features/sorted/featureSample2.jpg}
     \includegraphics[height=0.4in]{found_features/sorted/featureSample3.jpg}
     \includegraphics[height=0.4in]{found_features/sorted/featureSample4.jpg}
     \includegraphics[height=0.4in]{found_features/sorted/featureSample5.jpg}
     \includegraphics[height=0.4in]{found_features/sorted/featureSample6.jpg}
     \includegraphics[height=0.4in]{found_features/sorted/featureSample7.jpg}
     \includegraphics[height=0.4in]{found_features/sorted/featureSample8.jpg}
     \includegraphics[height=0.4in]{found_features/sorted/featureSample9.jpg}
     \includegraphics[height=0.4in]{found_features/sorted/featureSample10.jpg}
\end{center}
%\vspace{-0.2in}
\caption{\textbf{Example Collected Features.}
Each square is a weight pattern for an actual feature discovered by DDFA 
in which white is negative and black is positive.    
As these examples show, features range from simple line orientation detectors
reminiscent of those found in the lower levels of conventional deep
networks (towards left) to more complex shapes (towards right).        
%white negative black positive
}
\label{fg:features}
\end{figure}

To rule out the possibility that the reason for the testing performance 
is simply the HyperNEAT-based encoding of features,    
a \textbf{random CPPN control} was also run.  It follows an identical
procedure for training and testing, except that novelty scores
and adding to the feature list
during the feature accumulation phase are decided 
\emph{randomly}, which means the final collection in effect
contained random features with a range of CPPN complexity similar to the 
normal run.  To further investigate the value of the HyperNEAT
representation, an additional 
\textbf{random weights control} was tested
whose weights were assigned from a uniform random distribution, 
bypassing
HyperNEAT entirely.  
As the results in Table \ref{fig:testingResults} show,
the CPPN encoding in HyperNEAT provides a surprisingly good basis
for training even when the features are entirely randomly-generated.
However, they are still inferior to the features collected by 
normal DDFA.  As shown in the last column, without HyperNEAT, testing
performance with a collection of random features is unsurprisingly poor. 
In sum these controls show that the pretraining in DDFA is essential
to priming the later classifier for the best possible performance.        
      
%as a control, DDFA was also attempted for TODO generations
%in which novelty scores were intentionally assigned
%randomly.  The random scores lead to a random final set of 
%HyperNEAT-encoded features
%with a range of complexity similar to the normal run.  
%About 5 features were randomly selected for the feature list in each generation
%to yield $TODO$ like in the normal run. 
%The best result exhibited a TODO error (sd=TODO), confirming
%that the novelty search is necessary for useful features
%to emerge.

\section{Discussion and Future Work}

The results suggest that DDFA can indeed collect useful features
and thereby serve as an alternative unsupervised feature learner.
While it may ultimately
lead to better training performance in some 
cutting-edge problems, 
future work with more layers and on larger problems 
is clearly necessary to investigate its full potential for
exceeding top results.

However, it is important to recognize that significantly more
than performance is at stake in the dissemination of
alternative unsupervised training techniques based on
new principles.  Deep learning faces several fundamental
challenges that are not only about testing performance.       
For example, recent surprising 
results from \citet{szegedy:corr13} 
show that very small yet anomalous perturbations of 
training images that are imperceptible to the human eye
can fool several different kinds of deep networks 
that nevertheless ominously score well on the 
test set.  The implications of these anomalies are
not yet understood. 
At the same time, as \citet{bengio:slsp13}
points out, local descent on its own will not 
ultimately be enough to tackle the most challenging problems,
suggesting the need for radical new kinds of optimization
that are more global.  These kinds of 
considerations suggest that simply scoring well on
a test set in the short run may not necessarily foreshadow
continuing success for the field in the long run.      

Therefore, the expanded possibilities that a validated
new principle  
can inspire are essential to the health
of an evolving field, whether or not it ultimately
breaks a particular benchmark record.
For example, DDFA shows that unsupervised
discriminative learning is possible and can be
effective, bringing with it several
intriguing corrollaries.  Among those,
it is possible to conceive training methods that
act as continual feature accumulators that do not
require a fixed ``hidden layer size.''   
Furthermore, it is possible to learn useful features
without any kind of error minimization
(which is even used in conventional unsupervised 
techniques).  Relatedly, an interesting question is whether
anomalous results are sometimes a side effect of the
very idea that all useful knowledge ultimately 
must come from minimizing error.   
The divergent dynamics of novelty search also
mean that the search is inherently more global
than local descent for the very reason that  
it is continually 
diverging, thereby offering a hint
of how more expansive feature sets can be
collected.
Thus, in addition to the many possibilities
for training multilayer deep features in DDFA,
another important path for future work is 
to investigate the long-term implications of 
these more
subtle differences from conventional techniques,
and to determine whether similar such unique 
properties can be introduced to deep learning through 
non-evolutionary techniques that follow gradients other than error. 
   
%(available at \texttt{http://www.cambrianlabs.com/mattias/GeneticArt/}) based on his DelphiNEAT implementation
%(available at \texttt{http://www.cambrianlabs.com/mattias/DelphiNEAT/}, to Holger Ferstl for creating the second NEAT-based Genetic
%Art Program based on SharpNEAT (available at \texttt{http://sephi.axinom.de/Blog/PermaLink.aspx\\?guid=a0affef8-eb07-4ad8-8942-be5d481dbc0f}, and to Colin Green for creating 
%SharpNEAT (available at \texttt{http://sharpneat.sourceforge.net//}).

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% Change the title of \thebibliography section if required
%%\renewcommand\refname{Bibliography}
%%
%% Squeeze bibliography together
%%\renewcommand{\baselinestretch}{0.7}
\bibliographystyle{nnapalike}
%%\setlength{\bibhang}{0.125in}  % To match CogSci97 specs
\bibliography{nnstrings,nn,newentries,mybib,ucf}

\appendix

%\section{Appendix: HyperNEAT Technical Details} \label{appendix}

%Hi

%\subsubsection*{References}
%
%References follow the acknowledgments. Use unnumbered third level heading for
%the references. Any choice of citation style is acceptable as long as you are
%consistent. It is permissible to reduce the font size to `small' (9-point) 
%when listing the references. {\bf Remember that this year you can use
%a ninth page as long as it contains \emph{only} cited references.}
%
%\small{
%[1] Alexander, J.A. \& Mozer, M.C. (1995) Template-based algorithms
%for connectionist rule extraction. In G. Tesauro, D. S. Touretzky
%and T.K. Leen (eds.), {\it Advances in Neural Information Processing
%Systems 7}, pp. 609-616. Cambridge, MA: MIT Press.
%
%[2] Bower, J.M. \& Beeman, D. (1995) {\it The Book of GENESIS: Exploring
%Realistic Neural Models with the GEneral NEural SImulation System.}
%New York: TELOS/Springer-Verlag.
%
%[3] Hasselmo, M.E., Schnell, E. \& Barkai, E. (1995) Dynamics of learning
%and recall at excitatory recurrent synapses and cholinergic modulation
%in rat hippocampal region CA3. {\it Journal of Neuroscience}
%{\bf 15}(7):5249-5262.
%}

\end{document}

nnmacros.tex
%% $Id: nnmacros.tex,v 1.28 2005/09/28 21:18:37 jbednar Exp $
%%
%% Standard set of LaTeX macros and declarations for the NN group
%%
%% Originally based on risto's com.tex
%%
%% Because of \usepackage{hyperref}, this file should be included
%% *after* all other packages.


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% Add ability to check for PDFTeX

\newif\ifpdf
\ifx\pdfoutput\undefined
  \pdffalse
\else
  \pdfoutput=1
  \pdftrue
\fi

%% Macro useful for figuring out if a TeX command has been defined
%%
\def\ifundefined#1{\expandafter\ifx\csname#1\endcsname\relax}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% Packages required


% Because of this call, the "graphics" interface can't be used in
% conjunction with this file.  
%
\ifpdf\PassOptionsToPackage{pdftex}{graphicx}\fi
\usepackage{graphicx}

%% If you want to supply options to the hyperref package, you can add :
%% 
%% \PassOptionsToPackage{...}{hyperref} 
%%
%% in your source file before any \usepackage commands.  To turn
%% off hyperlinks altogether, you can use: 
%%
%% \PassOptionsToPackage{draft}{hyperref} %% Turn off hyperref links
%%
\ifpdf
%% Highlights links in PDF files
\usepackage{hyperref}
%% Highlights bibliography links
\newcommand{\biblink}[2]{\htmladdnormallink{#1}{#2}}
\fi

% From the LaTeX2HTML distribution.
\usepackage{html}%   
%
% If you don't have or want LaTeX2HTML when running this at home, just
% copy /u/nn/tex/latex2html/texinputs/html.sty to somewhere   
% in your TEXINPUTS path and ignore the rest of LaTeX2HTML.


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% From risto's com.tex:

%%% Generates a \tt quotation without indentation
%\def\quotation{\list{}{\listparindent 0.0em
%    \setlength{\parindent}{0.0in}
%    \renewcommand{\baselinestretch}{1}\small\normalsize\tt
%    \renewcommand{\arraystretch}{1}
%%    \itemindent\listparindent
%    \setlength{\leftmargin}{0.0in}
%    \parsep 0pt plus 1pt}\item[]}
%\let\endquotation=\endlist

%% I like to tighten up my captions and tables
\newcommand{\mycapdefs}{\renewcommand{\baselinestretch}{1}\small}
\newcommand{\mytabdefs}{\renewcommand{\arraystretch}{1}\small}

%% Get it?
\hyphenation{Miik-ku-lai-nen}

%% I don't like pages with only figures on them
\renewcommand{\topfraction}{0.9}
\renewcommand{\floatpagefraction}{0.8}
\renewcommand{\dbltopfraction}{0.9}
\renewcommand{\dblfloatpagefraction}{0.8}
\renewcommand{\textfraction}{.15}

%% Simple PostScript picture inclusion macros using the graphicx package
%%
%% These three commands have been provided for backwards compatibility
%% only.  You should not use them in new documents, because the 
%% non-standard argument format causes trouble with packages that expect
%% LaTeX-style  arguments, such as AucTeX, emacs font-lock, and
%% LaTeX2HTML.  Instead use the \pspic or \pdfpic macros (below).
%%
\begin{latexonly}
\def\putpicture   #1 (#2)\centerline{\includegraphics[totalheight=#2]ps/#1.ps}
\def\hputpicture  #1 (#2){\centerline{\includegraphics[width=#2]{ps/#1.ps}}}
\def\hvputpicture #1 (#2 by #3){\centerline{\includegraphics[width=#2,totalheight=#3]{ps/#1.ps}}}
\end{latexonly}


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% \scribbledraft: Provides a single-column mode with lots of space
%% to write comments on, e.g. for giving to a coauthor.
%%
%% To use, add \scribbledraft before \begin{document} but after
%% all your \usepackage commands.
%%
%% Tips:
%%
%% If the formatting doesn't look right, you may have to comment out
%% your document style (e.g. \usepackage{<confname>}) and instead use
%% \usepackage{nntwocolumn}.  Usually it will only be the first 
%% page that has problems, though; that's where style files usually
%% mess with the page widths and lengths.
%%
%% If you get lines sticking past the edge of the column, try adding
%% \sloppy just after \scribbledraft.
%%
%% If your document has figure or figure* environments that come out much
%% too small in \scribbledraft mode, you may be able to fix them by
%% changing them to fullfigure or fullfigure* environments, which are
%% defined below.  However, you may find that the result is no longer
%% centered properly or is the wrong width.  If so, you might need to add
%% a \setlength\originalwidth{6.5in} or \setlength\marginoffset{0in}
%% after \scribbledraft, adjusting those two values to fix the width and
%% left margin.  
%%
\newcommand{\scribbledraft}{
  \newlength\marginoffset\setlength\marginoffset{-0.5in}
  \newlength\nominalwidth\setlength\nominalwidth{3.5in}
  \newlength\nominalmargin\setlength\nominalmargin{0.5in}
  \setlength\topmargin{0.5in}
%% \setlength\oddsidemargin{0.5in}
%% \setlength\evensidemargin{0.5in}
\setlength\oddsidemargin{\nominalmargin}
\setlength\evensidemargin{\nominalmargin}
\setlength\textheight{8in}
\setlength\textwidth{\nominalwidth}
\setlength\footskip{20pt}
\setlength\marginparwidth{2.5in}
\pagestyle{plain}
\onecolumn
% Some styles (e.g. cogsci, nntwocolumn) call \twocolumn in \maketitle,
% so here we define it as a no-op
\renewcommand{\twocolumn}[1][]{##1\bigskip\bigskip} 
%%
\renewcommand{\forcedoublecolumnfigurestart}
{\setlength\textwidth{\originalwidth}%
\hspace*{\marginoffset}%
\begin{minipage}{\originalwidth}}
%%
\renewcommand{\forcedoublecolumnfigureend}
{\end{minipage}\setlength\textwidth{\nominalwidth}}
}
%% Save original value here for later
\newlength{\originalwidth}\setlength{\originalwidth}{\textwidth}
%%
\newcommand{\forcedoublecolumnfigurestart}{}
\newcommand{\forcedoublecolumnfigureend}{}
%%
\newenvironment{fullfigure}[1][tbp]
{\begin{figure}[#1]\forcedoublecolumnfigurestart}
{\forcedoublecolumnfigureend\end{figure}}
%%
\newenvironment{fullfigure*}[1][tbp]
{\begin{figure*}[#1]\forcedoublecolumnfigurestart}
{\forcedoublecolumnfigureend\end{figure*}}



%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% Conditional picture inclusion macros using the graphicx package
%%
%% If you use \pdfpic (below) when specifying a picture, you
%% can use \psdraftlevel (also below) to specify which ones to include
%% for a printout (e.g. to save print time or file size, or to print
%% color pages separately).  It's also handy to have the type of picture
%% declared in the source file this way, as a clue to what type of
%% picture is involved.  Just ignore this if it is not needed.

%% First, declare some symbolic constants representing the different
%% classes of images.  These definitions shouldn't ever need changing, 
%% but new ones can be added in between if needed.
%%  
\def\psnone{100}        % No picture should be included
\def\psline{80}         % Black and white line drawings only
\def\psmono{60}         % Monochrome images only
\def\pscolororbw{40}    % Images for which color is optional
\def\pscolor{20}        % Images for which color is required
\def\psall{0}           % All pictures should be included


%% The \psdraftlevel command determines which pictures should actually
%% be included, while other pictures merely have their bounding
%% box drawn in to save time or file size, or to print on printers
%% with insufficient memory.  It is optional, but if used, should
%% presumably be in the preamble. Each of the above levels is
%% inclusive of the preceding ones (those with lower numbers), so
%% \mono includes line drawings too, and \color includes line, mono,
%% and color pictures. 
%%
%% Example:  To include all pictures (the default),
%%   \psdrafttype{\psall}
%%
%% Example:  To include all pictures which make sense in monochrome,
%%   \psdrafttype{\pscolororbw}
%%
%% Example:  To include only line drawings, i.e. pictures likely to be
%% very small in file size, 
%%   \psdrafttype{\psline}
%%
%% Below, 0 is hardcoded instead of \psall because of problems 
%% with some versions of latex2html
\newcount\psdraftlevel \psdraftlevel=0
\newcommand{\psdrafttype}[1]{\psdraftlevel=#1}


%% Including PDF, JPG, PNG, or TIF pictures in PDFLaTeX documents,
%% (as well as EPS pictures in LaTeX documents)
%%
%% \pdfpic[category]{width}{height}{[dir/]filename}{filetype}
%%
%%
%% Arguments:
%%
%% The "category" argument is optional; if present it should indicate the
%% type of image the file represents, i.e. \psline for a line drawing,
%% \pscolor for a color photo, \pscolororbw for an image that is readable
%% either in color or monochrome, \psnone for a blank space, etc.  For 
%% a given run, the actual image is included only if the the
%% \psdrafttype is low enough.  For example, if \psdrafttype}{\psmono}
%% has been specified, all line and monochrome pictures are included,
%% but color pictures will be represented only by a bounding box.  If
%% "category" is omitted then the image will always be displayed,
%% regardless of draftlevel.
%%
%% Either or both of the length arguments "length" and "width" can be 
%% omitted if they are replaced by "!", which indicates that that
%% dimension should scale proportionally.  (If both are "!", then the
%% image is left at the natural size of the image, whatever that might
%% be for a given filetype.)  A dummy image (type \psnone) must have
%% both "length" and "width" specified, since they cannot be read from
%% a file.
%%
%% The argument "[dir/]filename" should be the complete path to your
%% picture file, minus the final extension, which in turn goes in the 
%% {filetype} argument.  For instance, to access "./images/pretty.jpg",
%% use \pdfpic{!}{!}{images/pretty}{jpg}.  
%%
%% 
%% Tips and examples:
%% 
%% Filetypes (i.e., extensions) of jpg, png, pdf and tif are currently
%% supported. Of these you should usually use pdf for line drawings,
%% jpg for photographs, and tif or png for non-photograph bitmap images.
%% 
%% Usually, it is best to specify that a picture be fixed in size
%% horizontally if it extends across the page, so that it still fits
%% on the page properly if the margins, number of columns, etc. are
%% changed.  
%%
%% Example: To insert a monochrome PNG picture images/sample.png, scaled
%% to be as wide as the current page,        
%%   \pdfpic[\psmono]{\textwidth}{!}{images/sample}{png}
%%
%% Example: To insert a line drawing images/sample.pdf, scaled to be as
%% half as wide as the page, rotated 90 degrees counterclockwise, and
%% framed with a thin, snug outline,
%%   \frame{\rotatebox{90}{\pdfpic[\psline]{!}{0.5\textwidth}{images/sample.pdf}{pdf}}}
%%
%% Example: To insert a dummy 3" x 2" space for pasting in a picture 
%% later, when no image file actually exists on disk,
%%   \pdfpic[\psnone]{3in}{2in}{Paste feller:science96 figure 3 here}{}
%%
%% If this command is used under LaTeX, the filetype is ignored and 
%% one of type eps and ending in .ps is assumed.  If you do use such EPS 
%% files, make sure that they are truly EPS (i.e., have a bounding box)
%% and not just PostScript.
%%
%%
\newcommand{\pdfpic}[5][\psall]{\includepdfpic[#1]{#2}{#3}{#4}{#5}{}}%


%% Same as \pdfpic but clips to boundary of box
\newcommand{\pdfpicclip}[5][\psall]{\includepdfpic[#1]{#2}{#3}{#4}{#5}{clip=True,}}%



%% Old-style interface for inserting PostScript pictures into LaTeX
%% documents:
%%
%%
%% This interface only supports PDF and EPS filetypes, and is included
%% for backwards compatibility with older documents developed in LaTeX.
%% Such documents can be converted to work with PDFLaTeX simply by
%% converting all of the EPS files they include to PDF.  For instance,
%% (in csh):
%%
%% foreach psfile (ps/*.ps) 
%%   epstopdf "${psfile}"
%% end
%%
%% That way PDFLaTeX will find the ps/*.pdf files, and LaTeX will find
%% the ps/*.ps files.  Be sure to check the generated .pdf figures
%% carefully; sometimes the translation process generates visible
%% artifacts.  For instance, sometimes you can see fringes around
%% sharp, high-contrast borders in images, like those when converting
%% bitmaps to JPEG.
%%
%% 
%% This macro is used just like \pdfpic, except that you should omit the
%% filetype (because only one filetype is supported for each platform)
%% and the directory (which is assumed to be ./ps/):
%%
%% \pspic[category]{width}{height}{filename}
%%
\newcommand{\pspic}[4][\psall]{\pdfpic[#1]{#2}{#3}{ps/#4}{pdf}}
\newcommand{\pspicclip}[4][\psall]{\pdfpicclip[#1]{#2}{#3}{ps/#4}{pdf}}




%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%
%% Implementation for \pdfpic, etc.; shouldn't usually 
%% be called directly.  Arguments are same as for \pdfpic
%% except that a 6th argument is accepted, and passed
%% directly to \includegraphics
%%
\newcommand{\includepdfpic}[6][\psall]{%
\def\pdfpicFileType{#5}%
\def\pdfpicFileExt{#5}%
%%
%% Assume .ps under LaTeX
\ifpdf\else% 
\def\pdfpicFileType{eps}%
\def\pdfpicFileExt{ps}%
\fi%
%%
%% Dummy picture: empty box (not even a frame).
%% Unspecified heights default to \unitlength
\ifx#1\psnone%
\resizebox{#2}{#3}{\begin{picture}(1,1)\end{picture}}%
%%
%% Draft picture: framed box with filename inside.
%% This looks fine with dvips, but it confuses xdvi
%% for some reason, which draws an extra box with the
%% natural size of the image. 
\else{\ifnum#1<\psdraftlevel%
\frame{\resizebox{#2}{#3}{\includegraphics[#6draft=true,type=\pdfpicFileType,ext=.\pdfpicFileExt,read=*,command=##1]{#4}}}%
\else%
%%
%% Normal picture: image in the specified format.
\resizebox{#2}{#3}{\includegraphics[#6type=\pdfpicFileType,ext=.\pdfpicFileExt,read=*,command=##1]{#4}}\fi}\fi%
}

%% Simpler version to keep LaTeX2HTML from getting confused
\begin{imagesonly}
\renewcommand{\pspic}[4][\psall]{\resizebox{#2}{#3}{\includegraphics{ps/#4.ps}}}
\end{imagesonly}



%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%>> To do (6/2002): 
%% It would be nice to have some sort of facility for automatically 
%% handling conditional text for different document types.  Here's
%% a sketch, but it is not legal LaTeX code:
%%
%% \newcommand{\definedocumenttype}[2]{
%%   \expandafter\def\csname dt#1\endcsname{#2}%
%%   \newcommand{not#1}[1]{}%
%%   \newcommand{#1only}[1]{}%
%% }
%% \definedocumenttype{book}{100}          % Book
%% \definedocumenttype{thesis}{80}         % PhD. or Master's thesis
%% \definedocumenttype{techreport}{60}     % Technical report
%% \definedocumenttype{preprint}{45}       % Journal preprint (to be passed around, not submitted)
%% \definedocumenttype{journal}{40}        % Journal paper (as published, or to be submitted)
%% \definedocumenttype{conf}{20}           % Conference paper
%% \definedocumenttype{article}{0}         % Default; generic article
%% 
%% \newcount\documenttype \documenttype=0
%% \newcommand{\setdocumenttype}[1]{%
%%   \documenttype=#1%
%%   \newcommand{not#1}[1]{#1}%
%%   \newcommand{#1only}[1]{}%
%% }
%%
%% Usage:
%%  \setdocumenttype{\book}  % Or any other
%%  \bookonly{Text for the book version}
%%  \notbook{Text for all versions except the book}
%%
%% All 'only' and 'not' comparisons should be available
%% even if only a single document type is available.  It should
%% be legal to define multiple document types.  However, it's not
%% clear how to implement any of this.


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%  Standard lengths, used to keep documents uniform throughout
 
%% Set size of windowpane to fit one, two, etc. square panes per line  
%%
\newlength{\oneplotwidth}           \setlength{\oneplotwidth}         {\textwidth}
\newlength{\twoplotwidth}           \setlength{\twoplotwidth}         {0.46\textwidth}
\newlength{\twoplotheight}          \setlength{\twoplotheight}        {\twoplotwidth}
\newlength{\threeplotwidth}         \setlength{\threeplotwidth}       {0.31\textwidth}
\newlength{\threeplotheight}        \setlength{\threeplotheight}      {\threeplotwidth}
\newlength{\fourplotwidth}          \setlength{\fourplotwidth}        {0.22\textwidth}
\newlength{\fourplotheight}         \setlength{\fourplotheight}       {\fourplotwidth}
\newlength{\fiveplotwidth}          \setlength{\fiveplotwidth}        {0.18\textwidth}
\newlength{\fiveplotheight}         \setlength{\fiveplotheight}       {\fiveplotwidth}
 
%% Set size of windowpane to fit the specified configuration of square
%% panes per page  
%%
\newlength{\twobytwoplotwidth}      \setlength{\twobytwoplotwidth}    {0.46\textwidth}
\newlength{\threebyfourplotwidth}   \setlength{\threebyfourplotwidth} {0.25\textwidth}
\newlength{\fourbythreeplotwidth}   \setlength{\fourbythreeplotwidth} {0.171875\textheight}
%% Note that the portrait page above with 12 square panes is named as 
%% a "width", but it is actually limited in the vertical direction
%% for typical US paper proportions, assuming there is a 1-inch caption.
 
 
%% Plot width that xmgr graphs should use
%%
%% This value is appropriate for output that has been run through
%% ps2epsi to fix the bounding boxes.  Xmgr computes them too
%% generously, so this would need to be some smaller value otherwise.
%%
\newlength{\xmgrplotwidth}          \setlength{\xmgrplotwidth}{\textwidth}
 
 
%% Blank variable available for various purposes
%%   Must use \setlength or \settowidth to give it 
%%   a value before using it
%%   E.g.: \settowidth{\templength}{Some text}
%%
\newlength{\templength}

 

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% Support for captioned subfigures
%%
%% To enable these commands, add "\usepackage[tight,center]{subfigure}"
%% before loading this file.  Then define subfigures with \sfobject, \sfpic,
%% or \sfpicframe, and refer to them with \sfref or \sfrefnp.
%%
%% Requires subfigure package version 2.1.2 or higher.  See the subfigure 
%% documentation for more details, such as how to change the caption 
%% separation, fonts, etc.:
%% http://www-2.cs.cmu.edu/afs/cs/usr/sdc/www/latex/subfigure.html
%%
\makeatletter
  \@ifpackageloaded{subfigure}{%

  %% Set up the subfigure package to italicize only the subfigure letters,
  %% not the parentheses, and to make \ref show only the bare letters,
  %% without any parentheses and without the main figure number.
  \let\p@subfigure\relax
  \renewcommand{\thesubfigure}{\emph{\alph{subfigure}}}
  \renewcommand*{\@thesubfigure}{(\thesubfigure)\hskip\subfiglabelskip}
  \renewcommand*{\@@thesubfigure}{(\thesubfigure)}


  %% Define a subfigure of any type with a caption underneath it
  %% Usage:   \sfobject[width]{item}{caption-text}{subfigure-label}
  %% Example: \sfobject{0.3\textwidth}{... a picture of some sort ...}{Above is a fancy picture}{fig}
  %% The subfigure-label must be unique for the whole document.
  %%
  \newcommand{\sfobject}[4]{%
    \subfigure[\label{sf:#4}#3]{\begin{minipage}[t]{#1}#2\end{minipage}}}
  \newcommand{\sfobjectuncaptioned}[2]{% Same spacing but no caption
    \subfigure{\begin{minipage}[t]{#1}#2\end{minipage}\addtocounter{subfigure}{-1}}}


  %% Define a subfigure consisting of a captioned \pdfpic
  %% Usage:   \sfpic{width}{filenamebase}{imagetype}{caption text}{subfigure-label}
  %% Example: \sfpic{0.3\textwidth}{picture}{jpg}{A picture}{renoir-pic}
  %%
  \newcommand{\sfpic}[5]{%
    \sfobject{#1}{\pdfpic{\textwidth}{!}{#2}{#3}}{#4}{#5}}%
  \newcommand{\sfpicuncaptioned}[3]{% Same but no caption
    \sfobjectuncaptioned{#1}{\pdfpic{\textwidth}{!}{#2}{#3}}}%

  %% Same as \sfpic but with a frame around the picture
  \newcommand{\sfpicframe}[5]{%
    \sfobject{#1}{\frame{\pdfpic{\textwidth}{!}{#2}{#3}}}{#4}{#5}}%
  \newcommand{\sfpicframeuncaptioned}[3]{%
    \sfobjectuncaptioned{#1}{\frame{\pdfpic{\textwidth}{!}{#2}{#3}}}}%


  %% Expand a reference to a subfigure, e.g. in caption or the main text
  %% \sfref{subfigure-label} yields (\emph{a}), etc.
  \newcommand{\sfref}[1]{(\ref{sf:#1})}


  %% Same as \sfref without the parentheses
  %% \sfrefnp{subfigure-label} yields \emph{a}, etc.
  \newcommand{\sfrefnp}[1]{\ref{sf:#1}}


  %% If producing overhead slides, call this command in the preamble
  %% so that (a), (b), etc. will be omitted from subfigure captions
  \newcommand{\sfstriplabels}{\renewcommand*{\@thesubfigure}{\mbox{}}}
}
\makeatother



%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%  Misc

%% Puts a tiny note in the margin; useful for notes on drafts.
%%
%% To disable all of them uncomment the second def.
%% When you use it in the middle of a paragraph, you should follow 
%% the last brace with a comment character (%) to avoid getting
%% a double space in the output.
\newcommand{\draftnote}[1]{{\marginpar{\tiny\em\raggedright #1}}}
%%\renewcommand{\draftnote}[1]{}

%% The \draftnote command is based on \marginpar, which doesn't work 
%% in floating  bodies like figures and tables because the notes also
%% float.  So we provide a non-floating version of marginpar
%% (\rawmarginpar), and then use that to create a version of
%% \draftnote that works with figures (\rawdraftnote).  Note that
%% \rawdraftnotes might overlap, go off the page, etc.; they are not
%% juggled automatically like \draftnotes are.  \rawdraftnote
%% usually works best right before the figure caption.
%%
\newcommand{\rawmarginpar}[1]{%
\vadjust{\smash{\llap{\parbox[t]{0.7\marginparwidth}{#1}\kern\marginparsep}}}}
\newcommand{\rawdraftnote}[1]{{\rawmarginpar{\tiny\em\raggedright #1}}}

%% Pretends that its argument takes zero horizontal space, e.g. to 
%% prevent a caption from having a linebreak by letting it 
%% extend slightly into the space on either side.
%%
\providecommand{\zerospace}[1]{\mbox{\makebox[0in]{#1}}}

%% Same as \zerospace except that the result is left-
%% justified, as if it had zero length out to the right.
\providecommand{\zerolength}[1]{\mbox{\makebox[0in][l]{#1}}}


%% Centers the given object vertically and horizontally
%% within a square box of the specified size, ignoring 
%% the size of the object.  
%%
%% Usage: \squarebox[position]{width}{object}
%%
%% Examples:
%%   Center some text in the left 1/10 page and draw a border around it:
%%     \frame{\squarebox{0.1\textwidth}{Some text}}
%%   Center an image 1/8 the size of the page in a box twice its width:
%%     \frame{\squarebox{0.25\textwidth}{\pdfpic{0.5\textwidth}{!}{somefile}{pdf}}
\newcommand{\squarebox}[3][c]{%
    \begin{minipage}[#1]{#2}%
      \setlength{\unitlength}{\textwidth}%
      \begin{picture}(1.0,1.0)%
        \put(0.5,0.5){\makebox(0,0)[c]{#3}}%
      \end{picture}%
    \end{minipage}}

%% Command to add a degree symbol following a number
%%
%% When using it, you need to follow it with a "\ " if the next
%% character should be a space.  
%%
%% Examples: "23\degree\ line"    "the orientation was 23\degree."
%%
%% Used to be called \deg but there is now a LaTeX2e command by that
%% name. 
\newcommand{\degree}{\ensuremath{^\circ}}


%% Non-math-mode equivalents of math-mode characters, etc.
%% These are useful for preventing LaTeX2HTML from converting
%% tiny in-text expressions like these into GIFs, and can also 
%% make text a little more readable overall.
\newcommand{\superscript}[1]{\ensuremath{^#1}}
\newcommand{\subscript}[1]{\ensuremath{_#1}}
\newcommand{\mtimes}[2]{\ensuremath{#1 \times #2}}
\newcommand{\madd}[2]{\ensuremath{#1 + #2}}
\newcommand{\mplus}{\ensuremath{+}}
\newcommand{\mpm}{\ensuremath{\pm}}
\newcommand{\mlt}{\ensuremath{<}}
\newcommand{\mgt}{\ensuremath{>}}

%% Given a color type as argument, e.g. \psmono or \pscolor, returns the note 
%% that should be placed in the caption for that figure.
%% This will make figures that require color say so; I can't think of anything
%% useful for the rest to say.
%% Maybe \pscolororbw should say something like "(Figure looks better in color)"?
\newcommand{\figurenote}[1]{%
\ifx#1\pscolor{\ ({\em color figure\/})}\fi}


%% Dummy command marking a section of text as something for latex2html
%% to typeset as a single left-justified block  
\newcommand{\htmlblock}[1]{#1}


%% Constructs a titled figure caption.  Takes four arguments:
%%
%% \titledcaption[category]{label}{title}{captiontext}
%%
%% The "category" argument is optional; if present it should indicate the
%% type of image the file represents, i.e.  \psline for a line drawing,
%% \pscolor for a color photo, \pscolororbw for an image that is readable
%% either in color or monochrome, \psnone for a blank space, etc.  At
%% present, "category" is only used to see whether or not to
%% add a descriptive note for the title and list of figures entry.
%% See the \figurenote command for the cases where a note will be
%% added.  The default will never add any note.
%% 
%% The "label" is a name used to construct a label for the figure.
%%
%% The "title" should be a short heading, which will go in bold at the
%% top of the caption, and also goes into the list of figures.
%%
%% The "captiontext" will go into the caption only, not the list of
%% figures, after the title.  Just use an empty last argument {} if 
%% there is no caption text.
%% 
%% Complete figure example:
%% \begin{figure}
%%   \centering 
%%   \pspic[\psline]{\oneplotwidth}{!}{some-figure-filename-wo-ps-extension}
%%   \titledcaption[\psline]{fg:some-figure-label}
%%   {Title of the figure}{ 
%%     Some long text describing what this figure means and how it
%%     should be interpreted.}
%% \end{figure}
%%
%% jbednar040510: For some reason, I had replaced both "#1"s with 
%% "\protect{#1}", probably to fix some problem with the separate
%% versions for typesetters (below).  However, that broke the color
%% figure labels for the normal case, so I have removed it.  Next time
%% the typesetter versions are used, they will probably need to be
%% debugged.
%%
\newcommand{\titledcaption}[4][\psall]{%
    \caption[#3\figurenote{#1}]{%
      \label{#2}\mycapdefs%
      \textbf{\mathversion{bold}#3\figurenote{#1}.} %
      \htmlblock{#4}}}


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% Support for defining figure graphics separately from captions
%%
%% To use this, just pick a label you want to refer to the figure by,
%% and define the meat of the figure:
%%
%% \definefigbody{fg:some-fig}{%
%%  \sfpicframe{\twoplotwidth}{picture1}{jpg}{Case 1}{case-1-label}\hfill
%%  \sfpicframe{\twoplotwidth}{picture2}{jpg}{Case 2}{case-2-label}}
%%
%% Then wherever you want the figure to appear, insert it with a caption:
%%
%%  \insertfig[\pscolor]{fg:some-fig}{%
%%    Figure title}{%
%%    This figure conclusively demonstrates that...}%
%%
%% Leave off [\pscolor] if you don't want the "(color figure)" note
%% in the caption.
%%
%% Everything else will be handled automatically.  If you want fine
%% control over the figure (instead of automatically using
%% titledcaption, fullfigure*, etc.), just use 
%% \insertfigbody{fg:some-fig} to insert the meat of the figure into 
%% your own figure skeleton.
%%
%% If you also want to define the captions separately from where they
%% are first inserted into the text, then use \definefigcaption
%% instead of \insertfig.  That way the caption will be defined but
%% not yet inserted into the text.  Then just use \firstfigref
%% instead of \ref the first time you refer to the figure, and the
%% figure will automatically be placed into the text at that point.
%%
%% If you want the captioned figure to be placed somewhere other
%% than the first reference, just put an \insertfighere{fg-some-fig} 
%% command where you want the figure to appear.
%%
%%
%% Defines the body of the given figure
\newcommand{\definefigbody}[2]{%
  \ifundefined{figbody--#1}\expandafter\def\csname figbody--#1\endcsname{#2}%
  \else\errmessage{Figure #1 already defined}\fi}%
%%
%% Rarely used by end users:
\newcommand{\insertfigbody}[1]{\csname figbody--#1\endcsname}%
\newcommand{\insertfigtype}[1]{\csname figtype--#1\endcsname}%
\newcommand{\insertfigtitle}[1]{\csname figtitle--#1\endcsname}%
\newcommand{\insertfigcaption}[1]{\csname figcaption--#1\endcsname}%
%%
%% Defines the type, title, and caption of the given figure
\newcommand{\definefigcaption}[4][\psmono]{%
  \ifundefined{figtype--#2}\expandafter\def\csname figtype--#2\endcsname{#1}%
  \else\errmessage{Type for #2 already defined}\fi%
  \ifundefined{figtitle--#2}\expandafter\def\csname figtitle--#2\endcsname{#3}%
  \else\errmessage{Title for #2 already defined}\fi%
  \ifundefined{figcaption--#2}\expandafter\def\csname figcaption--#2\endcsname{#4}%
  \else\errmessage{Caption for #2 already defined}\fi}%
%%
%% Inserts the given figure given the type, title, and caption
\newcommand{\insertfig}[4][\psmono]{%
  \begin{fullfigure*}\insertfigbody{#2}\titledcaption[#1]{#2}{#3}{#4}\end{fullfigure*}}%
%%
%% Rarely used by end users:
\newcommand{\insertfighere}[1]{%
  \insertfig%
  [\insertfigtype{#1}]{#1}%
  {\insertfigtitle{#1}}%
  {\insertfigcaption{#1}}}%
%%
%% Refers to and inserts the given captioned figure
\newcommand{\firstfigref}[1]{\ref{#1}\insertfighere{#1}}%


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% Support for figures at end of document for journal paper
%% submissions
%% 
%% None of these commands are used in a normal camera-ready paper
%%
%% For these commands to work, you should use \firstfigref
%% consistently (which allows the actual caption to be inserted 
%% wherever).  
%%
%% If you use subfigures, you should also add a line:
%%   \PassOptionsToPackage{normalsize}{subfigure}
%% before the \usebpackage{subfigure}, to make the fonts the right 
%% size.


%% Declare that figures should be put at end of paper for typesetting
\newcommand{\figuresatend}[1]{%
  \renewcommand{\firstfigref}[1]{\begin{center}--- Insert
      figure~\ref{##1} about here ---\end{center}\ref{##1}}%
  %% Hack to make subfigure package work outside of a float environment
  \makeatletter\newcommand{\@captype}{figure}\makeatother%
}

%% Override this if you need to add your name or manuscript number
%% Accepts one argument, the figure key (suitable for \ref)
\newcommand{\submissionfigdescription}[1]{Figure~\ref{#1}}

% Format a figure for typesetting
% Usage: \insertsubmissionfig[separation]{reference}
\newcommand{\insertsubmissionfig}[2][1in]{%
\newpage\setcounter{subfigure}{0}
\centering
\makebox[0in]{%
\resizebox{7.5in}{!}
{\begin{minipage}{\textwidth}
    \centering%
    \insertfigbody{#2}\\[#1]
    \submissionfigdescription{#2}
  \end{minipage}}}%
}


% Format a caption for typesetting
% Usage: \insertsubmissioncaption{reference}
\newcommand{\insertsubmissioncaption}[1]{%
  \newpage%
  \begin{figure}[h]%
    \renewcommand{\baselinestretch}{1.5}\small\normalsize
    \caption[\insertfigtitle{#1}\figurenote{\protect{\insertfigtype{#1}}}]{%
      \label{#1}%
      \textbf{\insertfigtitle{#1}\figurenote{\protect{\insertfigtype{#1}}}}.
      \raggedright%
      \htmlblock{\insertfigcaption{#1}}}
  \end{figure}%
}



%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% Tech report title pages
%%
%% Default values; you can (and should) override these using
%% \renewcommand within your own paper.  Often you will need to add
%% different linebreaks to the \techreporttitle and
%% \techreportauthor since they need to fit in such a narrow window.
%%
\newcommand{\techreporttitle}{\title}
\newcommand{\techreportauthor}{\author}
\newcommand{\techreportnumber}{}
\newcommand{\techreportemailaddress}{}
\newcommand{\techreporthttpaddress}{http://www.cs.utexas.edu/users/nn}
\newcommand{\techreportdepartment}{Artificial Intelligence Laboratory}
\newcommand{\universityname}{The University of Texas at Austin}
\newcommand{\universityaddress}{Austin, TX 78712}

%% Call just after your \begin{document} to actually generate the tech
%% report title page.
%%
%% Takes two arguments -- the offset in x and y to correct for the
%% left and top margins, respectively, in the current style.  The
%% values should both be zero for the default 1-inch margins in
%% nnonecolumn.sty; adjust other margins as needed.
%%
\newcommand{\techreporttitlepage}[2]{%
  \newpage
  \thispagestyle{empty}
  \setlength{\unitlength}{1in}%
  \begin{picture}(0,0)(#1,#2)
    \put(1,-2.3){\parbox{4in}{%
      \begin{center}%
        {\textbf{\Large \techreporttitle}}\\[8mm]
        {\textbf{\large \techreportauthor}}\\[8mm]
        \textbf{\large \techreportnumber}
      \end{center}}}
    \put(1,-7.8){\parbox{4in}{%
      \begin{center}%
        {\tt \techreportemailaddress}\\
        {\tt \techreporthttpaddress}\\[8mm]
        \techreportdepartment\\
        \universityname\\
        \universityaddress
      \end{center}}}
  \end{picture}
  \newpage
}


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% For emacs:
%Local Variables:
%TeX-command-default:"Check"
%End:

arxivDDFA.bbl
\begin{thebibliography}{26}
\expandafter\ifx\csname natexlab\endcsname\relax\def\natexlab#1{#1}\fi
\providecommand{\biblink}[2]{#1}

\bibitem[{Bengio(2013)}]{bengio:slsp13}
Bengio, Y. (2013).
\newblock Deep learning of representations: Looking forward.
\newblock In \emph{Statistical Language and Speech Processing}, 1--37.
  Springer.

\bibitem[{Bengio et~al.(2013)Bengio, Courville, and
  Vincent}]{bengio:ieeetpam13}
Bengio, Y., Courville, A., and Vincent, P. (2013).
\newblock Representation learning: A review and new perspectives.
\newblock \emph{IEEE transactions on pattern analysis and machine
  intelligence}, 1798--1928.

\bibitem[{Bengio et~al.(2007)Bengio, Lamblin, Popovici, and
  Larochelle}]{bengio:nips07}
Bengio, Y., Lamblin, P., Popovici, D., and Larochelle, H. (2007).
\newblock Greedy layer-wise training of deep networks.
\newblock In \emph{Advances in Neural Information Processing Systems 19
  (NIPS)}. Cambridge, MA: MIT Press.

\bibitem[{Cire{\c{s}}an et~al.(2010)Cire{\c{s}}an, Meier, Gambardella, and
  Schmidhuber}]{ciresan:nc10}
Cire{\c{s}}an, D., Meier, U., Gambardella, L., and Schmidhuber, J. (2010).
\newblock Deep, big, simple neural nets for handwritten digit recognition.
\newblock \emph{Neural computation}, 22(12):3207--3220.

\bibitem[{Coates et~al.(2011)Coates, Ng, and Lee}]{coates:icais11}
Coates, A., Ng, A.~Y., and Lee, H. (2011).
\newblock An analysis of single-layer networks in unsupervised feature
  learning.
\newblock In \emph{International Conference on Artificial Intelligence and
  Statistics}, 215--223.

\bibitem[{Cully and Mouret(2013)}]{cully:gecco13}
Cully, A., and Mouret, J.-B. (2013).
\newblock \biblink{Behavioral repertoire learning in robotics}
  {{http://doi.acm.org/10.1145/2463372.2463399}}.
\newblock In \emph{Proceeding of the fifteenth annual conference on Genetic and
  evolutionary computation conference}, GECCO '13, 175--182. New York, NY, USA:
  ACM.

\bibitem[{Floreano et~al.(2008)Floreano, D\"{u}rr, and
  Mattiussi}]{floreano:ei08}
Floreano, D., D\"{u}rr, P., and Mattiussi, C. (2008).
\newblock Neuroevolution: from architectures to learning.
\newblock \emph{Evolutionary Intelligence}, 1:47--62.

\bibitem[{Gauci and Stanley(2008)}]{gauci:aaai08}
Gauci, J., and Stanley, K.~O. (2008).
\newblock \biblink{A case study on the critical role of geometric regularity in
  machine learning} {{http://eplex.cs.ucf.edu/papers/gauci_aaai08.pdf}}.
\newblock In \emph{Proceedings of the Twenty-Third AAAI Conference on
  Artificial Intelligence (AAAI-2008)}. Menlo Park, CA: AAAI Press.

\bibitem[{Gauci and Stanley(2010)}]{gauci:nc10}
Gauci, J., and Stanley, K.~O. (2010).
\newblock \biblink{Autonomous evolution of topographic regularities in
  artificial neural networks}
  {{http://eplex.cs.ucf.edu/publications/2010/gauci.nc10.html}}.
\newblock \emph{Neural Computation}, 22(7):1860--1898.

\bibitem[{Green(2003--2014)}]{green:SharpNEAT}
Green, C. (2003--2014).
\newblock Sharp{NEAT} homepage.
\newblock \url{http://sharpneat.sourceforge.net/}.

\bibitem[{Hinton et~al.(2006)Hinton, Osindero, and Teh}]{hinton:nc06}
Hinton, G.~E., Osindero, S., and Teh, Y.-W. (2006).
\newblock A fast learning algorithm for deep belief nets.
\newblock \emph{Neural Computation}, 18(7):1527--1554.

\bibitem[{Jaakkola et~al.(1999)Jaakkola, Haussler et~al.}]{jaakkola:nips98}
Jaakkola, T., Haussler, D., et~al. (1999).
\newblock Exploiting generative models in discriminative classifiers.
\newblock \emph{Advances in neural information processing systems}, 487--493.

\bibitem[{LeCun and Bengio(1995)}]{lecun:hbtnn95}
LeCun, Y., and Bengio, Y. (1995).
\newblock Convolutional networks for images, speech, and time-series.
\newblock In Arbib, M.~A., editor, \emph{The Handbook of Brain Theory and
  Neural Networks}. MIT Press.

\bibitem[{LeCun and Cortes(1998)}]{lecun:mnist98}
LeCun, Y., and Cortes, C. (1998).
\newblock The mnist database of handwritten digits.

\bibitem[{Lehman and Stanley(2011)}]{lehman:ecj11}
Lehman, J., and Stanley, K.~O. (2011).
\newblock \biblink{Abandoning objectives: Evolution through the search for
  novelty alone}
  {{http://www.mitpressjournals.org/doi/pdf/10.1162/EVCO_a_00025}}.
\newblock \emph{Evolutionary Computation}, 19(2):189--223.

\bibitem[{Marc'Aurelio et~al.(2007)Marc'Aurelio, Boureau, and
  LeCun}]{marcaurelio:nips07}
Marc'Aurelio, R., Boureau, L., and LeCun, Y. (2007).
\newblock Sparse feature learning for deep belief networks.
\newblock In \emph{Advances in Neural Information Processing Systems 20
  (NIPS)}, 1185--1192. Cambridge, MA: MIT Press.

\bibitem[{Ng and Jordan(2002)}]{ng:nips02}
Ng, A.~Y., and Jordan, M.~I. (2002).
\newblock On discriminative vs. generative classifiers: {A} comparison of
  logistic regression and naive bayes.
\newblock \emph{Advances in neural information processing systems}, 2:841--848.

\bibitem[{Schmidhuber et~al.(2011)Schmidhuber, Cire{\c{s}}an, Meier, Masci, and
  Graves}]{schmidhuber:agi11}
Schmidhuber, J., Cire{\c{s}}an, D., Meier, U., Masci, J., and Graves, A.
  (2011).
\newblock On fast deep nets for agi vision.
\newblock In \emph{The Fourth Conference on Artificial General Intelligence
  (AGI)}, 243--246. New York, NY: Springer.

\bibitem[{Simard et~al.(2003)Simard, Steinkraus, and Platt}]{simard:icdar03}
Simard, P., Steinkraus, D., and Platt, J.~C. (2003).
\newblock Best practices for convolutional neural networks applied to visual
  document analysis.
\newblock In \emph{International Conference on Document Analysis and Recogntion
  ({ICDAR})}, vol.~3, 958--962.

\bibitem[{Stanley(2007)}]{stanley:gpem07}
Stanley, K.~O. (2007).
\newblock Compositional pattern producing networks: A novel abstraction of
  development.
\newblock \emph{Genetic Programming and Evolvable Machines Special Issue on
  Developmental Systems}, 8(2):131--162.

\bibitem[{Stanley et~al.(2009)Stanley, D'Ambrosio, and Gauci}]{stanley:alife09}
Stanley, K.~O., D'Ambrosio, D.~B., and Gauci, J. (2009).
\newblock \biblink{A hypercube-based indirect encoding for evolving large-scale
  neural networks}
  {{http://eplex.cs.ucf.edu/publications/2009/stanley.alife09.html}}.
\newblock \emph{Artificial Life}, 15(2):185--212.

\bibitem[{Stanley and Miikkulainen(2002)}]{stanley:ec02}
Stanley, K.~O., and Miikkulainen, R. (2002).
\newblock \biblink{Evolving neural networks through augmenting topologies}
  {{http://nn.cs.utexas.edu/keyword?stanley:ec02}}.
\newblock \emph{Evolutionary Computation}, 10:99--127.

\bibitem[{Stanley and Miikkulainen(2003)}]{stanley:alife03}
Stanley, K.~O., and Miikkulainen, R. (2003).
\newblock \biblink{A taxonomy for artificial embryogeny}
  {{http://nn.cs.utexas.edu/keyword?stanley:alife03}}.
\newblock \emph{Artificial Life}, 9(2):93--130.

\bibitem[{Szegedy et~al.(2013)Szegedy, Zaremba, Sutskever, Bruna, Erhan,
  Goodfellow, and Fergus}]{szegedy:corr13}
Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow,
  I.~J., and Fergus, R. (2013).
\newblock Intriguing properties of neural networks.
\newblock \emph{CoRR}, abs/1312.6199.

\bibitem[{Verbancsics and Stanley(2010)}]{verbancsics:jmlr10}
Verbancsics, P., and Stanley, K.~O. (2010).
\newblock Evolving static representations for task transfer.
\newblock \emph{Journal of Machine Learning Research (JMLR)}, 11:1737--1769.

\bibitem[{Verbancsics and Stanley(2011)}]{verbancsics:gecco11}
Verbancsics, P., and Stanley, K.~O. (2011).
\newblock Constraining connectivity to encourage modularity in {H}yper{NEAT}.
\newblock In \emph{GECCO '11: Proceedings of the 13th annual conference on
  Genetic and evolutionary computation}, 1483--1490. Dublin, Ireland: ACM.

\end{thebibliography}
