Towards Closing the Energy Gap Between HOG and CNN Features for Embedded Vision
\IEEEauthorblockN { Amr Suleiman*
Yu-Hsin Chen*
Joel Emer
Vivienne Sze
%Computer vision enables a wide range of applications in robotics/drones, self-driving cars, smart Internet of Things, and portable/wearable electronics. For many of these applications, local embedded processing is preferred due to privacy and/or latency concerns. Accordingly, energy-efficient embedded vision hardware delivering real-time and robust performance is crucial. While deep learning is gaining popularity in several computer vision algorithms, a significant energy consumption difference exists compared to traditional hand-crafted approaches. In this paper, we provide an in-depth analysis of the computation, energy and accuracy trade-offs between learned features such as deep Convolutional Neural Networks (CNN) and hand-crafted features such as Histogram of Oriented Gradients (HOG). This analysis is supported by measurements from two chips that implement these algorithms. Our goal is to understand the source of the energy discrepancy between the two approaches and to provide insight about the potential areas where CNNs can be improved and eventually approach the energy-efficiency of HOG while maintaining its outstanding performance. Computer vision enables a wide range of applications in robotics/drones, self-driving cars, smart Internet of Things, and portable/wearable electronics. For many of these applications, local embedded processing is preferred due to privacy and/or latency concerns. Accordingly, energy-efficient embedded vision hardware delivering real-time and robust performance is crucial. While deep learning is gaining popularity in several computer vision algorithms, a significant energy consumption difference exists compared to traditional hand-crafted approaches. In this paper, we provide an in-depth analysis of the computation, energy and accuracy trade-offs between learned features such as deep Convolutional Neural Networks (CNN) and hand-crafted features such as Histogram of Oriented Gradients (HOG). This analysis is supported by measurements from two chips that implement these algorithms. Our goal is to understand the source of the energy discrepancy between the two approaches and to provide insight about the potential areas where CNNs can be improved and eventually approach the energy-efficiency of HOG while maintaining its outstanding performance accuracy.
0_abstract.tex

\begin{abstract}

%Computer vision enables a wide range of applications in robotics/drones, self-driving cars, smart Internet of Things, and portable/wearable electronics. For many of these applications, local embedded processing is preferred due to privacy and/or latency concerns. Accordingly, energy-efficient embedded vision hardware delivering real-time and robust performance is crucial. While deep learning is gaining popularity in several computer vision algorithms, a significant energy consumption difference exists compared to traditional hand-crafted approaches. In this paper, we provide an in-depth analysis of the computation, energy and accuracy trade-offs between learned features such as deep Convolutional Neural Networks (CNN) and hand-crafted features such as Histogram of Oriented Gradients (HOG). This analysis is supported by measurements from two chips that implement these algorithms. Our goal is to understand the source of the energy discrepancy between the two approaches and to provide insight about the potential areas where CNNs can be improved and eventually approach the energy-efficiency of HOG while maintaining its outstanding performance.

Computer vision enables a wide range of applications in robotics/drones, self-driving cars, smart Internet of Things, and portable/wearable electronics. For many of these applications, local embedded processing is preferred due to privacy and/or latency concerns. Accordingly, energy-efficient embedded vision hardware delivering real-time and robust performance is crucial. While deep learning is gaining popularity in several computer vision algorithms, a significant energy consumption difference exists compared to traditional hand-crafted approaches. In this paper, we provide an in-depth analysis of the computation, energy and accuracy trade-offs between learned features such as deep Convolutional Neural Networks (CNN) and hand-crafted features such as Histogram of Oriented Gradients (HOG). This analysis is supported by measurements from two chips that implement these algorithms. Our goal is to understand the source of the energy discrepancy between the two approaches and to provide insight about the potential areas where CNNs can be improved and eventually approach the energy-efficiency of HOG while maintaining its outstanding performance accuracy.

\end{abstract}

1_introduction.tex

%% ====================================================================
\section{Introduction}
\label{sec:introduction}
%% ====================================================================

Computer vision (CV) is a critical technology to numerous smart embedded systems, such as advanced driver assistant systems, autonomous cars/drones, and robotics. It extracts meaningful information from visual data for further decision making. However, many modern CV algorithms require high computational complexity, which makes their deployment on battery-powered devices challenging due to the tight energy constraints. Near-sensor visual data processors should consume under 1nJ/pixel with a logic gate counts of around 1000 kgates and a memory capacity of few hundred kBytes in order to be comparable with video codecs, which are present in most cameras~\cite{7063063}. For many applications, offloading computation to the cloud is also undesirable because of latency, connectivity, and security limitations. Thus, dedicated energy-efficient CV hardware becomes very crucial.

Feature extraction is the first processing step in most CV tasks, such as object classification and detection (Fig.~\ref{fig:cv_processing_pipeline}). It transforms the raw pixels into a high-dimensional space, where only meaningful and distinctive information is kept. Traditionally, features are designed by experts in the field through a hand-crafted process. For instance, many well-known \emph{hand-crafted features} use image gradients, such as histogram of oriented gradients (HOG)~\cite{dalal2005histograms} and scale invariant feature transform (SIFT)~\cite{lowe1999object}, based on the fact that human eyes are sensitive to edges. In contrast, \emph{learned features} learn a representation with the desired characteristics directly from data using deep convolutional neural networks (CNNs)~\cite{iscas2010-lecun}. Learned features are gaining popularity, as they are outperforming hand-crafted features in many CV tasks~\cite{nips2012-krizhevsky, cvpr2014-girshick}.

The differences in design between hand-crafted and learned features lead to not only different performance in applications, but also different hardware implementation considerations, which have a strong implication for energy efficiency. In general, hardware implementations for hand-crafted features are widely understood to be more energy-efficient than learned features. However, there is no analysis that explains the energy gap between the two types of features. Also, an open question is whether the energy gap can be closed in the future.

In this paper, we will provide an in-depth analysis on the causes for the energy gap between hand-crafted and learned features. We use results from two actual chip designs:~\cite{suleiman201658} implements the hand-crafted feature using HOG, and~\cite{isscc2016-chen} implements the learned feature using CNN. Both chips use 65nm CMOS technology and have similar hardware resource utilization in terms of logic gate count and memory capacity. Based on the insights derived from the two implementations, we will discuss techniques to help close the energy gap.

% The rest of the paper is organized as follows. Section~\ref{sec:overview} describes briefly the two algorithms implemented in our reference chips. Section~\ref{sec:hardware_considerations} presents different considerations when implementing feature extraction hardware. Then a detailed performance comparison is presented in Section~\ref{sec:performance_comparison}. Finally, before conclusion, Section~\ref{sec:closing_the_gap} shows some suggested approaches to close the gap between the two designs.

\begin{figure}
    \begin{center}
        \includegraphics[width=0.9\linewidth]{figures/pipeline_CNN.pdf}
        \caption{General processing pipeline for object classification and detection.
                }
        \vspace{-10pt}                
        \label{fig:cv_processing_pipeline}
    \end{center}
\end{figure}

% One of the most fundamental tasks in CV is object detection~\cite{szeliski2010computer}, which localizes and classifies objects in provided images (i.e., the outputs are the class and coordinates of each object in the images). It is also used as a building block for more complicated tasks like object segmentation and scene understanding. Object detection consists of two processing steps (Fig.~\ref{fig:detection_approach}): feature extraction and classification. Feature extraction transforms the raw pixels into a higher dimension space, where only meaningful and distinctive information are kept. Classification then uses machine learning techniques, such as support vector machine~\cite{cristianini2000introduction} and Adaboost~\cite{schapire2012boosting}, to map the extracted features to a score

% In this paper, we will focus on the feature extraction process. Traditionally, features were designed by experts in the field to mimic a physical property. For instance, human eyes are sensitive to edges (i.e., gradients). As a result, many well-known features use image gradients such as Histogram of Oriented Gradients (HOG)~\cite{dalal2005histograms} and Scale Invariant Feature Transform (SIFT)~\cite{lowe1999object}. These are called \emph{hand-crafted} features. The other approach is to learn the features directly from the data. These \emph{learned} features are used in a popular form of machine learning called deep neural networks (DNN), also known as deep learning~\cite{nature2015-lecun}. The two reference chips are implementing these two types of features, namely HOG and convolution neural networks (CNN)~\cite{iscas2010-lecun}.

% There are many different artificial intelligence (AI) tasks in computer vision~\cite{szeliski2010computer}. Object detection is one of the fundamental algorithms in computer vision. It is also one of the main building blocks of other complicated vision tasks like object segmentation and scene understanding. In object detection, the entire image is provided and the task is to localize an object and determine its class. To provide in depth analysis of object detection, 

% Fig.~\ref{fig:detection_approach} shows a conventional approach of object detection, which consists of two processes: feature extraction and classification. Feature extraction transforms the raw pixels into a higher dimension space, where only meaningful and distinctive information are kept. While classification maps the output of feature extraction to a score using machine learning techniques. Depending on the application, the score is either compared to a threshold to determine if an object is \emph{present}, or compared to the other scores to determine the object \emph{class}. Techniques often used for classification include linear methods such as support vector machine (SVM)~\cite{cristianini2000introduction} and Softmax, and non-linear methods such as  kernel-SVM~\cite{cristianini2000introduction} and Adaboost~\cite{schapire2012boosting}.

% The other approach is to learn the features directly from the data. These \emph{learned} features are used in a popular form of machine learning called deep neural networks (DNN), also known as deep learning~\cite{nature2015-lecun}. The two reference chips are implementing these two types of features, namely HOG and convolution neural networks (CNN)~\cite{iscas2010-lecun}.



2_hardware_implementation.tex

%% ====================================================================
\section{Feature Extraction Hardware Implementations}
\label{sec:hardware_implementation}
%% ====================================================================

%% ====================================================================
\subsection{Hardware for Hand-crafted Feature: HOG}

The chip presented in~\cite{suleiman201658} implements the entire object detection pipeline based on deformable parts models (DPM)~\cite{girshick2012discriminatively} for high throughput and low power embedded vision applications. DPM extracts HOG features from the input image, and localizes objects by sweeping the features with support vector machine (SVM) classifiers~\cite{cristianini2000introduction}. Fig.~\ref{fig:hog_simple} shows the features extraction process using HOG: the image is divided into non-overlapping 8$\times$8 pixel cells. A 9-bin histogram is then generated for each cell using gradients orientations. The histograms are further normalized with respect to the surrounding cells for robustness against illumination and texture variations. Since HOG features are not scale invariant, the features are extracted over an image pyramid to achieve multi-scale detection.

\begin{figure}
    \begin{center}
        \includegraphics[width=0.85\linewidth]{figures/hog_simple.pdf}
        \caption{Feature extraction with histogram of oriented gradients (HOG).
                }
        \vspace{-10pt}                
        \label{fig:hog_simple}
    \end{center}
\end{figure}

%% ====================================================================
\subsection{Hardware for Learned Feature: CNN}

The second chip presented in~\cite{isscc2016-chen}, called Eyeriss, is an energy-efficient accelerator for deep CNNs. Fig.~\ref{fig:DNN} shows a general CNN processing pipeline, consisting mainly of a series of convolutional (CONV) layers. In each CONV layer, a collection of 3D filters are applied to the input images or feature maps to generate output feature maps, which are then used as the input to the next CONV layer. Eyeriss is programmable in terms of the size and shape of filters as well as number of layers. Therefore, it can accommodate different CNN architectures, such as AlexNet~\cite{nips2012-krizhevsky} and VGG-16~\cite{iclr2015-simonyan}.

\begin{figure}
    \begin{center}
        \includegraphics[width=0.9\linewidth]{figures/DNN.pdf}
        \caption{General processing pipeline of CNN.
                }
        \vspace{-10pt}                
        \label{fig:DNN}
    \end{center}
\end{figure}

%% ====================================================================
\subsection{Performance Comparison}

Table~\ref{tab:compare_chips} shows the hardware specification and measured performance of the two designs for feature extraction. For CNN, Eyeriss is programmed to run two CNN models (five CONV layers of AlexNet and thirteen CONV layers of VGG-16) to demonstrate the hardware performance variations of running different CNN features. Both chip designs use around 1000 kgates and 150 kB memory. While Eyeriss achieves approximately the same computation throughput (i.e., GOPS) as the HOG design when running AlexNet features, HOG processes 35$\times$ more input pixels per second; the gap is even larger between HOG and VGG-16 features. This is due to the differences in computational complexity (i.e., operations per pixel) between different features as shown in Table~\ref{tab:feat_comp}. However, the HOG design also consumes around 10$\times$ less power than Eyeriss. Thus, the HOG hardware consumes 311$\times$ and 13,486$\times$ less energy per pixel than Eyeriss running AlexNet and VGG-16 features, respectively. In terms of energy per operation, the HOG hardware is 10$\times$ less than Eyeriss. In Section~\ref{sec:hardware_considerations}, we will discuss the cause of this energy gap.

% Tables~\ref{tab:feat_comp} and~\ref{tab:feat_energy} show the computation requirements and the measured energy consumption to extract the two types of features. For learned features, two CCN architectures (i.e., AlexNet and VGG) are implemented on Eyeriss chip to show its flexibilty to adapt to different CNN architectures. Comparing between hand-crafted and learned features, Table~\ref{tab:feat_comp} shows a one to two orders of magnitude computation difference. This difference is justified with the higher dimensionality of the learned features. However, a much larger two to four orders of magnitude energy consumption difference is shown in Table~\ref{tab:feat_energy}. The two types of programmability, i.e., PoP and PoHP, discussed in Section~\ref{sec:hardware_considerations} are believed to be the main reason for the gap between computation and energy. The HOG features extraction architecture is hardwired with no PoP and PoHP in order to achieve the highest possible energy efficiency. On the contrary, the CNN accelerator has both PoP and PoHP, so it can be programmed to implement different CNN architectures with user-defined parameters.

% have similar area and on-chip memory requirements, and performing the same amount of computation per second. However, an order of magnitude difference exists in power and throughput, and this leads to two orders of magnitude difference in energy efficiency. These gaps exist because of the large amount of computation required by CNNs, with hundreds of filtering operations, compared to HOG. Note that unlike Eyeriss chip that has one computation resolution of 16-bit, the resolution of each computation differs largely in the HOG hardware (e.g., multiplications resolution ranges from 5-bit$\times$11-bit to 22-bit$\times$22-bit). In the following sections, we will analyze the algorithms trying to close this gap.

\begin{table}
    \begin{center}
        % \normalsize
        \scriptsize
        \caption{Hardware specification and measured performance of hand-crafted feature HOG~\cite{suleiman201658} and learned feature CNN~\cite{isscc2016-chen}.}
        \begin{tabular}{|l|c|c|c|} 
            \hline
                                            & \cite{suleiman201658}                 & \multicolumn{2}{c|}{\cite{isscc2016-chen}}             \\
            \hline
            \multirow{2}{*}{\textbf{Implemented Feature}}    & \multirow{2}{*}{HOG} & CNN               & CNN \\ 
                                            &                                       & (AlexNet)         & (VGG-16) \\
            \hline
            \textbf{Technology}             & 65nm                                  & \multicolumn{2}{c|}{65nm} \\ 
            \hline
            \textbf{Gate Counts (kgates)}   & 893.0                                 & \multicolumn{2}{c|}{1176.0} \\ 
            \hline
            \textbf{Memory (kB)}            & 159.0                                 & \multicolumn{2}{c|}{181.5} \\ 
            \hline
            \textbf{Multiplier Bitwidth}    & 5$\times$11 -- 22$\times$22           & \multicolumn{2}{c|}{16$\times$16} \\
            \hline
            \textbf{Throughput (Mpixels/s)} & 62.5                                  & 1.8               & 0.04 \\ 
            \hline
            \textbf{Throughput (GOPS)}      & 46.0                                  & 46.2              & 21.4 \\ 
            \hline
            \textbf{Power (mW)}             & 29.3                                  & 278.0             & 236.0 \\ 
            \hline
            \textbf{DRAM access (B/pixel)}  & 1.0                                   & 74.7              & 2128.6 \\ 
            \hline
            \textbf{Energy Efficiency (nJ/pixel)}      & 0.5                        & 155.5             & 6742.9 \\ 
            \hline
            \textbf{Energy Efficiency (GOPS/W)} & 1570.0                            & 166.2             & 90.7 \\
            \hline
        \end{tabular}
        \label{tab:compare_chips}
    \end{center}
    \vspace{-10pt}
\end{table}

\begin{table}
    \begin{center}
        \small
        \caption{Computational complexity comparison between features}
        \begin{tabular}{|l|c|c|c|} 
            \hline
            \multicolumn{2}{|c|}{\textbf{Feature}}      & \textbf{GOP/Mpixel}   & \textbf{Ratio}\\ 
            \hline
            Hand-crafted                & HOG           & 0.7                   & 1.0$\times$   \\ 
            \hline
            \multirow{2}{*}{Learned}    & CNN (AlexNet) & 25.8                  & 36.9$\times$  \\ 
            \cline{2-4}
                                        & CNN (VGG-16)  & 610.3                 & 871.9$\times$ \\ 
            \hline
        \end{tabular}
        \label{tab:feat_comp}
    \end{center}
    \vspace{-10pt}
\end{table}

%% ====================================================================
\subsection{Accuracy vs. Energy Efficiency}

While there is a large energy gap between the hardware for hand-crafted and learned features, the performance differences in applications, e.g., accuracy, between the features should also be taken into account. To measure accuracy, we use the features for object detection, which localizes and classifies objects in provided images (i.e., the outputs are the class and coordinates of each object in the images). The mean average precision (mAP) metric is used to quantify detection accuracy.

Fig.~\ref{fig:hand_vs_dnn} shows the trade-off between detection accuracy and energy consumption. Note that the vertical axis is logarithmic. All reported detection accuracy numbers are measured on PASCAL VOC 2007 dataset~\cite{pascal-voc-2007}, which is a widely used image dataset containing 20 different object classes in 9,963 images. In order to achieve the same detection accuracy of HOG features, it only requires the features extracted from the first three CONV layers of AlexNet instead of five~\cite{zou2014dnp}, but it comes at the cost of 100$\times$ higher energy consumption per pixel to generate the features. Fortunately, mAP can be nearly doubled by using the ouput of all five layers of AlexNet with minimal increase in energy consumption (only 22\%). Even higher mAP has been demonstrated by using VGG-16 features~\cite{girshick2015frcnn}. However, the accompanying energy consumption per pixel becomes four orders of magnitude higher than using HOG features. If we look at the minimum hardware energy consumption required for a given accuracy in Fig.~\ref{fig:hand_vs_dnn}, we roughly see that a linear increase in detection accuracy requires an exponential increase in energy consumption per pixel.

% The same accuracy of HOG based object detection can be achieved using the output of the third convolution layer of AlexNet with the object detection pipeline in~\cite{zou2014dnp}, but a 100$\times$ more energy is required to generate the learned features. Fortunately, mAP can be doubled by using the output of the fifth and final convolution layer of AlexNet, with minimal increase in the energy consumption (only 22\%). For much higher mAP, authors in~\cite{girshick2015frcnn} use VGG CNN which has more convolution layers compared to AlexNet. However, the energy consumption of VGG is four orders of magnitudes higher than HOG features. Fig.~\ref{fig:hand_vs_dnn} shows that for a linear increase in the detection accuracy, an exponential increase in energy consumption is required.

\begin{figure}
    \begin{center}
        \includegraphics[width=0.9\linewidth]{figures/hand_vs_dnn_new.pdf}
        \caption{    Energy versus accuracy trade-off comparison for hand-crafted and learned features.
                }
        \vspace{-10pt}                
        \label{fig:hand_vs_dnn}
    \end{center}
\end{figure}

%The other approach is to learn the features directly from the data. These \emph{learned} features are used in a popular form of machine learning called deep neural networks (DNN), also known as deep learning~\cite{nature2015-lecun}. Lately, DNN has been widely used in a variety of tasks where it delivers higher accuracy than hand-crafted features~\cite{ijcv2015-russakovsky}. DNN's superior performance comes from its ability in learning features with millions of trained parameters~\cite{nips2012-krizhevsky}; however, it comes at the cost of high computational complexity. 

%For the classification process, a model is trained to map the output of feature extraction to a score. Depending on the application, the score is either compared to a threshold to determine if an object is \emph{present}, or compared to the other scores to determine the object \emph{class}. Techniques often used for classification include linear methods such as support vector machine (SVM)~\cite{cristianini2000introduction} and Softmax, and non-linear methods such as  kernel-SVM~\cite{cristianini2000introduction} and Adaboost~\cite{schapire2012boosting}. %In many of these classifiers, the computation of the score is effectively a dot product of the features ($\vec{x}$) and the weights ($\vec{w}$) (i.e., $\sum_i w_{i}x_{i}$). As a result, much of the hardware research has been focused on reducing the cost of a multiply and accumulate (MAC).


%The remaining sections focus on comparisons between hand-crafted and learned features from computation, energy and accuracy points of view. HOG is selected as the hand-crafted features since it provides a favorable trade-off between detection accuracy and complexity compared to alternative richer features~\cite{suleiman2014energy}. 

%For learned features, DNNs are composed of multiple convolution (CONV) layers~\cite{iscas2010-lecun}, where in each layer a higher-level abstraction of the input data is extracted to preserve essential yet unique information. Fig.~\ref{fig:DNN} shows a general DNN pipeline with convolution layers, which is called convolutional neural network (CNN). Different popular CNNs architectures are used like AlexNet~\cite{nips2012-krizhevsky} and VGG-16~\cite{iclr2015-simonyan}. The difference between architectures is in numbers of CONV layers, number and shape of filters in each layer.

3_hardware_considerations.tex

%% ====================================================================
\section{Hardware Implementation Considerations}
\label{sec:hardware_considerations}
%% ====================================================================

% The differences in design between hand-crafted and learned features lead to not only different performance in applications, but also different hardware implementation considerations, which have a strong implication for energy efficiency. In this section, we will qualitatively analyze how do these differences affect the hardware energy efficiency.

One of the key factors that differentiates the two types of features in hardware implementation is \emph{programmability}, which defines the flexibility of a feature to be configured to deal with different data statistics or different tasks. While hand-crafted features can achieve a certain degree of invariance to data variations (e.g., images with different exposures), they are mostly designed for very specific tasks with known data statistics, leaving little to be programmed when deployed. In contrast, learned features isolate algorithm design from learning the data statistics, which leaves room for programmability to take advantage of the flexibility. We categorize programmability into two types: Programmability of Hyper-Parameters (PoHP) and Programmability of Parameters (PoP).

%One of the key factors that differentiates the two types of features in hardware implementation is \emph{programmability}, which defines how flexible a feature can be configured to deal with different data statistics or even different tasks. While hand-crafted features can achieve a certain degree of invariance to data variations (e.g., images with different exposures), they are mostly designed for very specific tasks with known data statistics, leaving little to be programmed when deployed. In contrast, learned features isolate algorithm design from learning the data statistics, which leaves room for programmability to take advantage of the flexibility. To understand how programmability affects hardware implementation, and therefore impacts the energy efficiency, we further categorize it into two types: Programmability of Hyper-Parameters (PoHP) and Programmability of Parameters (PoP).


%% ====================================================================
\subsection{Programmability of Hyper-Parameters (PoHP)}

Hyperparameters refer to the number and/or dimensionality of parameters of a feature such as the number of layers and size of filters in a CNN, or the number of histogram bins in HOG. They are usually determined at design time by using heuristics or through experiments. 

\smallskip\noindent\textbf{Advantages}: For learned features, changes in hyperparameters with proper (re-)training can result in significant performance improvements. As a result, hardware that supports PoHP can easily trade-off computational complexity for higher accuracy. For instance, a CNN object detection system with PoHP can choose between lower-complexity, lower-accuracy, e.g., AlexNet, and higher-complexity, higher-accuracy, e.g., VGG-16, according to the use cases. For hand-crafted features, however, the impact of PoHP on performance is usually limited, since it can be hard to capture the changes in parameter dimensionality without redesign. As a result, PoHP are not commonly supported for the hardware implementation of hand-crafted features.

\smallskip\noindent\textbf{Energy Cost}: Although PoHP can greatly benefits the performance of learned features, it comes at the price of lowered energy efficiency, since the hardware implementation must accommodates a wide range of possible configurations. First, it introduces \emph{irregular data movement}. For example, a CNN processor that supports PoHP has to deal with filters of many shapes and sizes, which complicates the control logic, synchronization schemes between parallel blocks, and data tiling optimization. Second, it usually requires \emph{higher dynamic range and resolution for data representation}, which have a negative impact on both computation and data movement. The bitwidth of datapath and the bandwidth requirement of memory all need to be designed for the worst-case scenario, which penalize the average energy efficiency.

%% ====================================================================
\subsection{Programmability of Parameters (PoP)}

Parameters are the actual coefficients, such as the filter weights, that are used in the computation. For learned features, parameters are learned through training; for hand-crafted features, they are usually carefully designed according to the data statistics or pre-determined based on the desired properties. For instance, in HOG, the filter mask for the gradient extraction is simply $[-1~0~1]$.

\smallskip\noindent\textbf{Advantages}: With PoP, learned features can adapt to new data by simply retraining the parameters. For example, AlexNet has been shown to work for both RGB images and depth images~\cite{guptaECCV14}. For hand-crafted features, however, there is no PoP since all parameters are fixed at design time.

\smallskip\noindent\textbf{Energy Cost}: PoP also negatively impacts the hardware energy efficiency since the parameters have to be treated as data instead of fixed values during hardware implementation. This not only increases the required memory capacity and data movement, but also complicates the datapath design. In the case of CNN, the amount of parameters is usually too large to fit on-chip, which increases the accesses to energy-consuming off-chip memory, such as DRAM. In contrast, hand-crafted features can be greatly optimized for energy efficiency by hard-wiring the fixed parameters in datapaths or ROM. In the case of HOG, multiplications between the input images and gradient filter can be completely avoided. 
4_performance_comparison.tex
%% ====================================================================
\section{Performance Comparison}
\label{sec:performance_comparison}
%% ====================================================================

In this section, complete object detection systems are analyzed with object detection accuracy numbers. As shown below, different classification techniques lead to different accuracy numbers. The mean Average Precision (mAP) metric is used to measure the accuracy.

%% ====================================================================
%\subsection{Different Object Detection Approaches}
%\label{sec:different_detection_approaches}
%% ====================================================================
%For HOG hand-crafted features, a simple detection approach is to train an SVM classifier that captures the characteristics of the object of interest, and sweep the entire images at all position to locate the object. This is the method used in the original HOG paper~\cite{dalal2005histograms}. To increase the detection accuracy, a more complex approach is used in~\cite{girshick2012discriminatively}, called deformable parts models (DPM), where few extra SVM classifier are trained to detect moving parts and allow object deformation. With its flexibility, the DPM object detection achieves 2$\times$ higher detection accuracy compared to rigid body models. However, this comes at a 35$\times$ computation overhead~\cite{suleiman201658}.

%With learned CNN features, some examples of object detection with CNN features are presented in~\cite{zou2014dnp, girshick2015dpmcnn, cvpr2014-girshick, girshick2015frcnn}. In these approaches, the output of the convolution layers is used as input features to the classifiers. Different classification approaches are used with CNN features. For instance, boosting classifier cascades are used in~\cite{zou2014dnp}, while a separate convolution layer is used in~\cite{girshick2015dpmcnn} as a classifier. The most common classification approach with CNN is using fully connected layers with Softmax as done in~\cite{cvpr2014-girshick, girshick2015frcnn}.

%% ====================================================================
\subsection{Complexity}
\label{sec:complexity}
%% ====================================================================
Tables~\ref{tab:feat_comp} and~\ref{tab:feat_energy} show the computation requirements and the measured energy consumption to extract the two types of features. For learned features, two CNN architectures (i.e., AlexNet and VGG) are implemented on Eyeriss chip to show its flexibilty to adapt to different CNN architectures. Comparing between hand-crafted and learned features, Table~\ref{tab:feat_comp} shows a one to two orders of magnitude computation difference. This difference is justified with the higher dimensionality of the learned features. However, a much larger two to four orders of magnitude energy consumption difference is shown in Table~\ref{tab:feat_energy}. The two types of programmability, i.e., PoP and PoHP, discussed in Section~\ref{sec:hardware_considerations} are believed to be the main reason for the gap between computation and energy. The HOG features extraction architecture is hardwired with no PoP and PoHP in order to achieve the highest possible energy efficiency. In contrast, the CNN accelerator has both PoP and PoHP, so it can be programmed to implement different CNN architectures with user-defined parameters.

\begin{table}
    \begin{center}
        \normalsize
        \caption{Features computation.}
        \begin{tabular}{|l|c|c|c|} \hline
            \multicolumn{2}{|c|}{\textbf{Feature}}&\textbf{GOP/Mpixel}&\textbf{Ratio}\\ \hline
            \multicolumn{2}{|c|}{Hand-crafted HOG~\cite{suleiman201658}}&0.7&1.0$\times$\\ \hline
            \multirow{2}{*}{Learned~\cite{isscc2016-chen}}&AlexNet&25.8&36.9$\times$\\ \cline{2-4}
            &VGG&610.3&871.9$\times$\\ \hline
        \end{tabular}
        \label{tab:feat_comp}
    \end{center}
    %\vspace{-5pt}
\end{table}

\begin{table}
    \begin{center}
        \normalsize
        \caption{Features extraction energy consumption.}
        \begin{tabular}{|l|c|c|c|} \hline
            \multicolumn{2}{|c|}{\textbf{Feature}}&\textbf{Energy {\small (nJ/pixel)}}&\textbf{Ratio}\\ \hline
            \multicolumn{2}{|c|}{Hand-crafted HOG~\cite{suleiman201658}}&0.5&1.0$\times$\\ \hline
            \multirow{2}{*}{Learned~\cite{isscc2016-chen}}&AlexNet&155.5&311.0$\times$\\ \cline{2-4}
            &VGG&6,742.9&13,485.8$\times$\\ \hline
        \end{tabular}
        \label{tab:feat_energy}
    \end{center}
    %\vspace{-5pt}
\end{table}

%% ====================================================================
\subsection{Detection Accuracy vs. Energy Efficiency}
\label{sec:accuracy_vs_energy}
%% ====================================================================
Fig.~\ref{fig:hand_vs_dnn} shows the trade-off between detection accuracy and energy consumption. Note that the vertical axis is in logarithmic scale. All reported detection accuracy numbers are measured on PASCAL VOC 2007 dataset~\cite{pascal-voc-2007}, which is a widely used image dataset containing 20 different object classes in 9,963 images. The same accuracy of HOG based object detection can be achieved using the output of the third convolution layer of AlexNet with the object detection pipeline in~\cite{zou2014dnp}, but a 100$\times$ more energy is required to generate the learned features. Fortunately, mAP can be doubled by using the output of the fifth and final convolution layer of AlexNet, with minimal increase in the energy consumption (only 22\%). For much higher mAP, authors in~\cite{girshick2015frcnn} use VGG CNN which has more convolution layers compared to AlexNet. However, the energy consumption of VGG is four orders of magnitudes higher than HOG features. Fig.~\ref{fig:hand_vs_dnn} shows that for a linear increase in the detection accuracy, an exponential increase in energy consumption is required.


\begin{figure}
    \begin{center}
        \includegraphics[width=0.9\linewidth]{figures/hand_vs_dnn_new.pdf}
        \caption{    Energy versus accuracy trade-off comparison for hand-crafted and learned features.
                }
        \vspace{-10pt}                
        \label{fig:hand_vs_dnn}
    \end{center}
\end{figure}

5_closing_the_gap.tex

%% ====================================================================
\section{Closing the Energy Gap}
\label{sec:closing_the_gap}
%% ====================================================================

%In Section~\ref{sec:hardware_implementation}, we have shown the energy efficiency results of hardware implementations from two extremes: the HOG hand-crafted feature with no PoP or PoHP, and the DNN learned feature with both PoP and PoHP, and the latter consumes over two orders of magnitude higher energy per pixel than the former. In this section, we will discuss the techniques that can be applied to reduce the energy efficiency gap.

%% ====================================================================
%\smallskip\noindent\textbf{Removing Programmability}:
%While learned features can take advantage of programmability, it is still possible to remove all programmability for the hardware implementation. For example, the CONV layer weights from AlexNet can either be hard-wired in the multipliers, or stored in on-chip SRAM or ROM. However, it is still very challenging if the available hardware resources are constrained to the level of the HOG design~\cite{suleiman201658}, i.e., 1000 kgates with 150 kB SRAM. Assuming each input and weight value take 1 byte, only 10k multipliers with fixed weights can be implemented in 1000 kgates, and only 150k weight values can fit in the SRAM. However, the first 3 CONV layers of AlexNet already have 1227k weights, and all five CONV layers have 2334k weights. The number of multipliers cannot even fit 1\% of AlexNet, and 15$\times$ larger memory is required to store all weights. Additional optimizations are needed to make this approach feasible.
%so neither SRAM nor ROM can fit into the resource constraint.

In Section~\ref{sec:hardware_implementation}, we have shown the energy efficiency results of hardware implementations from two extremes: the HOG hand-crafted feature with no PoP or PoHP, and the CNN learned feature with both PoP and PoHP, and the latter consumes an order of magnitude higher normalized energy per pixel than the former. A simple approach to reduce this energy gap is to remove all programmability in the hardware implementation of CNN, but this is not straightforward.

For example, the CONV layer weights from AlexNet can either be hard-wired in the multipliers, or stored in on-chip SRAM or ROM. This is not feasible if the available hardware resources are constrained to the level of the HOG design~\cite{suleiman201658}, i.e., 1000 kgates with 150 kB SRAM. Assuming each input and weight value take 1 byte, only 10k multipliers with fixed weights can be implemented in 1000 kgates, and only 150k weight values can fit in the SRAM. This number of multipliers cannot even fit 1\% of AlexNet 2334k weights in the CONV layers, and 15$\times$ larger memory is required to store all weights.

In this section, we will discuss some techniques that can be applied to reduce the energy efficiency gap.

%% ====================================================================
\smallskip\noindent\textbf{Reduced Precision}:
Reducing data precision is an active research area for DNN~\cite{gysel2016hardware}, which directly reduces the worst-case bitwidth requirement when supporting PoHP. Specifically, 8-bit integer precision has become popular in recent DNN hardware~\cite{tpu}. Non-uniform quantization~\cite{han2016eie} and bitwidth reduction to 1-bit~\cite{nips2015-courbariaux-binaryconnect, courbariaux2016binarynet,eccv2016-rastegari-xnor_net} have also been demonstrated. Energy efficiency can also be improved if the hardware can adapt to the need of actual data. Custom datapath designs that adapt to the lower data precision show 2.56$\times$ energy savings in~\cite{moons20160}.

% , which uses a precision of lower bitwidth, e.g., 4 bits, to represent a subset of data at higher bitwidth, e.g., 16 bits, has also been demonstrated~\cite{han2016eie}. 

%% ====================================================================
\smallskip\noindent\textbf{Sparsity}:
Increasing data sparsity reduces the intrinsic amount of computation and data movement, which improve energy efficiency. For CNN, the number of weights can be greatly reduced without reducing accuracy through pruning~\cite{nips1990-lecun-opt_brain_damage, nips2015-han, yang2016}. Specifically,~\cite{yang2016} has shown that the number of CONV layer weights in AlexNet can be reduced from 2334k to 352k, which is only double the memory capacity used in HOG; furthermore it reduces energy by 3.7$\times$. Specialized hardware designs can be used to exploit sparsity for increased speed or reduced energy consumption~\cite{isscc2016-chen,albericio2016cnvlutin, han2016eie}.
%within an order of magnitude of

% Specialized hardware have also been proposed in to exploit data sparsity for increased speed or reduced energy consumption. 
%Specifically, Eyeriss~\cite{isscc2016-chen} is designed to skip zero inputs, which saves the processing element energy consumption by 45\%. However, maximally reducing weights does not necessarily result in lower energy consumption, since each weight can have a different impact on energy consumption. Instead, the pruning process should focus on the weights that have the most impact on the overall energy consumption~\cite{yang2016}.

%% ====================================================================
\smallskip\noindent\textbf{Data Compression}:
Sparsity also suggests opportunities for compression, which saves memory space and data transfer bandwidth. Many lightweight lossless compression schemes, such as run-length coding~\cite{isscc2016-chen} and Huffman coding~\cite{han2016eie}, are proposed to reduce the amount of off-chip data~\cite{isscc2016-chen,moons20160,han2016eie}. %Directly processing compressed data on-chip has also become a promising direction to further improve energy efficiency.

% For example, Eyeriss~\cite{isscc2016-chen} uses run-length compression to achieve 1.9$\times$ off-chip bandwidth reduction. In general, the cost of performing compression and decompression is negligible compared to the accompanied energy savings.

%% ====================================================================
\smallskip\noindent\textbf{Energy Optimized Dataflow}:
PoHP incurs irregular data movement, preventing memory access optimization during hardware design. Therefore, designing hardware architecture that can adapt to the irregular data movement becomes critical to high energy efficiency, since data movement often consumes more energy than computation. Eyeriss demonstrates a reconfigurable architecture that can optimize data movement for various CNNs with a row stationary dataflow, and achieves 1.4$\times$ to 2.5$\times$ higher energy efficiency than existing designs~\cite{isca2016-chen}. 

% Since data movement can be more energy consuming than computation, designing a hardware architecture that can adapt to the irregular data movement becomes the key to high energy efficiency. In Eyeriss~\cite{isscc2016-chen}, a reconfigurable dataflow, called row-stationary (RS), that can optimize the data movement under various DNN configurations was presented to address this issue. It has shown 1.4$\times$ to 2.5$\times$ higher energy efficiency than the existing dataflows. However, the RS dataflow was proposed based on the dense DNN model, i.e., without sparsity, with uniform data precision across all data types. It becomes a new challenge for future work to come up with a more adaptive dataflow that can address sparsity and reduced data precision.

%% ====================================================================
%\smallskip\noindent\textbf{Adaptive Processing}:
%PoHP requires overhead on bitwidth, bandwidth, and memory capacity to accommodate the worst-case setup. However, energy efficiency can be improved if the hardware can adapt to the need of actual data. For example, custom datapath designs that adapt to the data precision show 2.56$\times$ energy savings in~\cite{moons20160}.
% or 2.24$\times$ throughput improvement~\cite{judd2016stripes}. \cite{han2016eie} has also demonstrated using buffering queue to even out imbalanced workload caused by data sparsity.


Table~\ref{tab:techniques} summarizes the energy and memory savings using the discussed techniques. Combining them all can potentially deliver an order of magnitude reduction. Taking into account the fundamental computation gap discussed in Section~\ref{sec:hardware_implementation}, this reduction has the potential of closing the gap between HOG and CNN features.

\begin{table}
\centering
\small
\caption{CNN energy and memory savings using different techniques. *Measured on AlexNet **Assuming a 16-bit baseline}
\begin{tabular}{|c|c|c|} \hline
\textbf{Method}&\textbf{Energy}&\textbf{Memory Size}\\\hline
Reduced precision~\cite{tpu,moons20160}&2.56$\times$&2.0$\times$\\ \hline
Sparsity by pruning~\cite{yang2016}&3.7$\times$&6.6$\times$*\\ \hline
Data Compression~\cite{moons20160}&-&$>$2.0$\times$**\\ \hline
Energy optimized dataflow~\cite{isca2016-chen}&1.4--2.5$\times$&-\\ \hline
\end{tabular}
\label{tab:techniques}
%\vspace{-5pt}
\end{table}
6_conclusion.tex

%% ====================================================================
\section{Conclusion}
\label{sec:conclusion}
%% ====================================================================

%The CNN learned features outperform the HOG hand-crafted features in visual object classification and detection tasks. This paper compares two chip designs (one for CNN and the other for HOG) to better understand the energy discrepancy between the two approaches and provide insight about the potential optimizations. Although learned features achieves more than 2$\times$ higher detection accuracy, it comes at a large 311$\times$ to 13,486$\times$ overhead in energy consumption. A fundamental computation overhead exists, but another order of magnitude gap is mainly caused by the fact that CNN architecture is programmable. Some techniques are highlighted in the paper to reduce the energy gap. A simple approach of removing all programmability in CNN and hard-wire all multiplications doesn't work, as only 1\% of the required computation for AlexNet already exceeds the area cost of a typical embedded processor. However, other optimizations such as increasing data sparsity, using lossless data compression and reducing the multiplication precision can help close the energy gap.

The CNN learned features outperform the HOG hand-crafted features in visual object classification and detection tasks. This paper compares two chip designs for CNN and HOG to better understand the energy discrepancy between the two approaches and provide insight about the potential optimizations. Although learned features achieve more than 2$\times$ accuracy, it comes at a large 311$\times$ to 13,486$\times$ overhead in energy consumption. While a fundamental computation overhead exists, another order of magnitude gap is mainly caused by the fact that CNN architecture is programmable. A simple approach of removing all programmability in CNN and hard-wiring all multiplications doesn't work due to significant area cost (i.e., logic gates and on-chip memory).  Combing the techniques highlighted in the paper can potentially reduce the energy and memory sizes by an order of magnitude, and help reduce the gap between learned and hand-crafted features.
main.tex

%% ------------------------------------------------------------------------------------------------
%%  style setup
%% ------------------------------------------------------------------------------------------------

\documentclass[conference]{IEEEtran}

\usepackage{fancyhdr}
\usepackage{setspace}
\usepackage[normalem]{ulem}
\usepackage[hyphens]{url}
\usepackage[caption=false]{subfig}
\usepackage{amsmath}
\usepackage{booktabs}
\usepackage{multirow}
\usepackage{threeparttable}
\usepackage{array,multirow}
\usepackage{graphicx}
\usepackage{makecell}
\usepackage{epsfig}
\usepackage{enumitem}
\usepackage{cleveref}
\usepackage[square,sort&compress,comma,numbers]{natbib}

\usepackage{footnote}
\makesavenoteenv{tabular}
\makesavenoteenv{table}

% correct bad hyphenation here
\hyphenation{op-tical net-works semi-conduc-tor hyper-parameters}

%% ------------------------------------------------------------------------------------------------
%%  Title & Authors
%% ------------------------------------------------------------------------------------------------


% paper title
\title
{
    Towards Closing the Energy Gap Between HOG and CNN Features for Embedded Vision
}

\author
{
    \IEEEauthorblockN   {   Amr Suleiman*, Yu-Hsin Chen*, Joel Emer, Vivienne Sze}
    \IEEEauthorblockA   {   Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology\\ 
                            \{suleiman, yhchen, jsemer, sze\}@mit.edu\\
                \textit{*These authors contributed equally to this work}}
}



% use for special paper notices
\IEEEspecialpapernotice{(Invited Paper)}

\begin{document}
% make the title area
\maketitle

%% ------------------------------------------------------------------------------------------------
%%  Abstract 
%% ------------------------------------------------------------------------------------------------

\input{0_abstract}

%% ------------------------------------------------------------------------------------------------
%%  Text Body
%% ------------------------------------------------------------------------------------------------

\input{1_introduction}

\input{2_hardware_implementation}

\input{3_hardware_considerations}

% \input{4_performance_comparison}

\input{5_closing_the_gap}

\input{6_conclusion}

%% ------------------------------------------------------------------------------------------------
%%  Acknowledgment
%% ------------------------------------------------------------------------------------------------

% \section*{Acknowledgment}
% 
% The authors would like to thank...

%% ------------------------------------------------------------------------------------------------
%%  References
%% ------------------------------------------------------------------------------------------------

\bibliographystyle{ieeetr}
%\small
%\footnotesize
\scriptsize
\bibliography{__references}

%% ------------------------------------------------------------------------------------------------

\end{document}



main.bbl
\begin{thebibliography}{10}

\bibitem{7063063}
C.~C. Ju and et~al., ``{A 0.5nJ/pixel 4K H.265/HEVC codec LSI for multi-format
  smartphone applications},'' in {\em ISSCC}, pp.~1--3, Feb 2015.

\bibitem{dalal2005histograms}
N.~Dalal and B.~Triggs, ``{Histograms of oriented gradients for human
  detection},'' in {\em CVPR}, 2005.

\bibitem{lowe1999object}
D.~G. Lowe, ``{Object recognition from local scale-invariant features},'' in
  {\em {ICCV}}, 1999.

\bibitem{iscas2010-lecun}
Y.~LeCun, K.~Kavukcuoglu, and C.~Farabet, ``{Convolutional networks and
  applications in vision},'' in {\em ISCAS}, 2010.

\bibitem{nips2012-krizhevsky}
A.~Krizhevsky, I.~Sutskever, and G.~E. Hinton, ``{ImageNet Classification with
  Deep Convolutional Neural Networks},'' in {\em {NIPS}}, 2012.

\bibitem{cvpr2014-girshick}
R.~Girshick, J.~Donahue, T.~Darrell, and J.~Malik, ``Rich {Feature}
  {Hierarchies} for {Accurate} {Object} {Detection} and {Semantic}
  {Segmentation},'' in {\em CVPR}, 2014.

\bibitem{suleiman201658}
A.~Suleiman, Z.~Zhang, and V.~Sze, ``{A 58.6 mW real-time programmable object
  detector with multi-scale multi-object support using deformable parts model
  on 1920$\times$ 1080 video at 30fps},'' in {\em {Sym. on VLSI}}, 2016.

\bibitem{isscc2016-chen}
Y.-H. Chen, T.~Krishna, J.~Emer, and V.~Sze, ``{Eyeriss: An Energy-Efficient
  Reconfigurable Accelerator for Deep Convolutional Neural Networks},'' in {\em
  ISSCC}, 2016.

\bibitem{girshick2012discriminatively}
R.~B. Girshick, P.~F. Felzenszwalb, and D.~McAllester, ``{Discriminatively
  trained deformable part models, release 5},'' 2012.

\bibitem{cristianini2000introduction}
N.~Cristianini and J.~Shawe-Taylor, {\em An introduction to support vector
  machines and other kernel-based learning methods}.
\newblock {Cambridge University Press}, 2000.

\bibitem{iclr2015-simonyan}
K.~Simonyan and A.~Zisserman, ``{Very Deep Convolutional Networks for
  Large-Scale Image Recognition},'' in {\em ICLR}, 2015.

\bibitem{pascal-voc-2007}
M.~Everingham, L.~Van~Gool, C.~K.~I. Williams, J.~Winn, and A.~Zisserman, ``The
  {PASCAL} {V}isual {O}bject {C}lasses {C}hallenge 2007 {(VOC2007)}
  {R}esults.''
  http://www.pascal-network.org/challenges/VOC/voc2007/workshop/index.html.

\bibitem{zou2014dnp}
W.~Zou, X.~Wang, M.~Sun, and Y.~Lin, ``{Generic Object Detection with Dense
  Neural Patterns and Regionlets},'' in {\em BMVC}, 2014.

\bibitem{girshick2015frcnn}
R.~Girshick, ``{Fast R-CNN},'' in {\em ICCV}, 2015.

\bibitem{guptaECCV14}
S.~Gupta, R.~Girshick, P.~Arbelaez, and J.~Malik, ``Learning rich features from
  {RGB-D} images for object detection and segmentation,'' in {\em ECCV}, 2014.

\bibitem{gysel2016hardware}
P.~Gysel, M.~Motamedi, and S.~Ghiasi, ``{Hardware-oriented Approximation of
  Convolutional Neural Networks},'' in {\em ICLR}, 2016.

\bibitem{tpu}
S.~Higginbotham, ``{Google Takes Unconventional Route with Homegrown Machine
  Learning Chips},'' May 2016.

\bibitem{han2016eie}
S.~Han, X.~Liu, H.~Mao, J.~Pu, A.~Pedram, M.~A. Horowitz, and W.~J. Dally,
  ``{EIE: efficient inference engine on compressed deep neural network},'' in
  {\em ISCA}, 2016.

\bibitem{nips2015-courbariaux-binaryconnect}
M.~Courbariaux, Y.~Bengio, and J.-P. David, ``{Binaryconnect: Training deep
  neural networks with binary weights during propagations},'' in {\em {NIPS}},
  2015.

\bibitem{courbariaux2016binarynet}
M.~Courbariaux and Y.~Bengio, ``{Binarynet: Training deep neural networks with
  weights and activations constrained to+ 1 or-1},'' {\em arXiv preprint
  arXiv:1602.02830}, 2016.

\bibitem{eccv2016-rastegari-xnor_net}
M.~Rastegari, V.~Ordonez, J.~Redmon, and A.~Farhadi, ``{XNOR-Net: ImageNet
  Classification Using Binary Convolutional Neural Networks},'' in {\em
  {ECCV}}, 2016.

\bibitem{moons20160}
B.~Moons and M.~Verhelst, ``{A 0.3--2.6 TOPS/W precision-scalable processor for
  real-time large-scale ConvNets},'' in {\em {Sym. on VLSI}}, 2016.

\bibitem{nips1990-lecun-opt_brain_damage}
Y.~LeCun, J.~S. Denker, and S.~A. Solla, ``{Optimal Brain Damage},'' in {\em
  {NIPS}}, 1990.

\bibitem{nips2015-han}
S.~Han, J.~Pool, J.~Tran, and W.~Dally, ``{Learning both Weights and
  Connections for Efficient Neural Network},'' in {\em NIPS}, 2015.

\bibitem{yang2016}
T.-J. Yang, Y.-H. Chen, and V.~Sze, ``{Designing Energy-Efficient Convolutional
  Neural Networks using Energy-Aware Pruning},'' {\em CVPR}, 2017.

\bibitem{albericio2016cnvlutin}
J.~Albericio, P.~Judd, T.~Hetherington, T.~Aamodt, N.~E. Jerger, and
  A.~Moshovos, ``{Cnvlutin: ineffectual-neuron-free deep neural network
  computing},'' in {\em {ISCA}}, 2016.

\bibitem{isca2016-chen}
Y.-H. Chen, J.~Emer, and V.~Sze, ``{Eyeriss: A Spatial Architecture for
  Energy-Efficient Dataflow for Convolutional Neural Networks},'' in {\em
  {ISCA}}, 2016.

\end{thebibliography}
