The Ciona17 Dataset for Semantic Segmentation \\ of Invasive Species in a Marine Aquaculture Environment
\IEEEauthorblockN{Angus Galloway
\IEEEauthorblockN { Angus Galloway\IEEEauthorrefmark{1
\IEEEauthorblockN{Michael Shell\IEEEauthorrefmark{1
An original dataset for semantic segmentation, ``Ciona17'', is introduced, which to the best of the authors' knowledge, is the first dataset of its kind with pixel-level annotations pertaining to invasive species in a marine environment. Diverse outdoor illumination, a range of object shapes, colour, and severe occlusion provide a significant real world challenge for the computer vision community. An accompanying ground-truthing tool for superpixel labeling, ``Truth and Crop'', is also introduced. Finally, we provide a baseline using a variant of Fully Convolutional Networks, and report results in terms of the standard mean intersection over union (mIoU) metric.
crv2017_ciona_dataset.tex

%% bare_conf.tex
%% V1.3
%% 2007/01/11
%% by Michael Shell
%% See:
%% http://www.michaelshell.org/
%% for current contact information.
%%
%% This is a skeleton file demonstrating the use of IEEEtran.cls
%% (requires IEEEtran.cls version 1.7 or later) with an IEEE conference paper.
%%
%% Support sites:
%% http://www.michaelshell.org/tex/ieeetran/
%% http://www.ctan.org/tex-archive/macros/latex/contrib/IEEEtran/
%% and
%% http://www.ieee.org/

%%*************************************************************************
%% Legal Notice:
%% This code is offered as-is without any warranty either expressed or
%% implied; without even the implied warranty of MERCHANTABILITY or
%% FITNESS FOR A PARTICULAR PURPOSE!
%% User assumes all risk.
%% In no event shall IEEE or any contributor to this code be liable for
%% any damages or losses, including, but not limited to, incidental,
%% consequential, or any other damages, resulting from the use or misuse
%% of any information contained here.
%%
%% All comments are the opinions of their respective authors and are not
%% necessarily endorsed by the IEEE.
%%
%% This work is distributed under the LaTeX Project Public License (LPPL)
%% ( http://www.latex-project.org/ ) version 1.3, and may be freely used,
%% distributed and modified. A copy of the LPPL, version 1.3, is included
%% in the base LaTeX documentation of all distributions of LaTeX released
%% 2003/12/01 or later.
%% Retain all contribution notices and credits.
%% ** Modified files should be clearly indicated as such, including  **
%% ** renaming them and changing author support contact information. **
%%
%% File list of work: IEEEtran.cls, IEEEtran_HOWTO.pdf, bare_adv.tex,
%%                    bare_conf.tex, bare_jrnl.tex, bare_jrnl_compsoc.tex
%%*************************************************************************

% *** Authors should verify (and, if needed, correct) their LaTeX system  ***
% *** with the testflow diagnostic prior to trusting their LaTeX platform ***
% *** with production work. IEEE's font choices can trigger bugs that do  ***
% *** not appear when using other class files.                            ***
% The testflow support page is at:
% http://www.michaelshell.org/tex/testflow/



% Note that the a4paper option is mainly intended so that authors in
% countries using A4 can easily print to A4 and see how their papers will
% look in print - the typesetting of the document will not typically be
% affected with changes in paper size (but the bottom and side margins will).
% Use the testflow package mentioned above to verify correct handling of
% both paper sizes by the user's LaTeX system.
%
% Also note that the "draftcls" or "draftclsnofoot", not "draft", option
% should be used if it is desired that the figures are to be displayed in
% draft mode.
%
\documentclass[10pt, conference, compsocconf]{IEEEtran}
% Add the compsocconf option for Computer Society conferences.
%
% If IEEEtran.cls has not been installed into the LaTeX system files,
% manually specify the path to it like:
% \documentclass[conference]{../sty/IEEEtran}





% Some very useful LaTeX packages include:
% (uncomment the ones you want to load)


% *** MISC UTILITY PACKAGES ***
%
%\usepackage{ifpdf}
% Heiko Oberdiek's ifpdf.sty is very useful if you need conditional
% compilation based on whether the output is pdf or dvi.
% usage:
% \ifpdf
%   % pdf code
% \else
%   % dvi code
% \fi
% The latest version of ifpdf.sty can be obtained from:
% http://www.ctan.org/tex-archive/macros/latex/contrib/oberdiek/
% Also, note that IEEEtran.cls V1.7 and later provides a builtin
% \ifCLASSINFOpdf conditional that works the same way.
% When switching from latex to pdflatex and vice-versa, the compiler may
% have to be run twice to clear warning/error messages.






% *** CITATION PACKAGES ***
%
\usepackage{cite}
% cite.sty was written by Donald Arseneau
% V1.6 and later of IEEEtran pre-defines the format of the cite.sty package
% \cite{} output to follow that of IEEE. Loading the cite package will
% result in citation numbers being automatically sorted and properly
% "compressed/ranged". e.g., [1], [9], [2], [7], [5], [6] without using
% cite.sty will become [1], [2], [5]--[7], [9] using cite.sty. cite.sty's
% \cite will automatically add leading space, if needed. Use cite.sty's
% noadjust option (cite.sty V3.8 and later) if you want to turn this off.
% cite.sty is already installed on most LaTeX systems. Be sure and use
% version 4.0 (2003-05-27) and later if using hyperref.sty. cite.sty does
% not currently provide for hyperlinked citations.
% The latest version can be obtained at:
% http://www.ctan.org/tex-archive/macros/latex/contrib/cite/
% The documentation is contained in the cite.sty file itself.






% *** GRAPHICS RELATED PACKAGES ***
%
\ifCLASSINFOpdf
    \usepackage[pdftex]{graphicx}
  % declare the path(s) where your graphic files are
  % \graphicspath{{../pdf/}{../jpeg/}}
  % and their extensions so you won't have to specify these with
  % every instance of \includegraphics
  % \DeclareGraphicsExtensions{.pdf,.jpeg,.png}
\else
  % or other class option (dvipsone, dvipdf, if not using dvips). graphicx
  % will default to the driver specified in the system graphics.cfg if no
  % driver is specified.
  % \usepackage[dvips]{graphicx}
  % declare the path(s) where your graphic files are
  % \graphicspath{{../eps/}}
  % and their extensions so you won't have to specify these with
  % every instance of \includegraphics
  % \DeclareGraphicsExtensions{.eps}
\fi
% graphicx was written by David Carlisle and Sebastian Rahtz. It is
% required if you want graphics, photos, etc. graphicx.sty is already
% installed on most LaTeX systems. The latest version and documentation can
% be obtained at:
% http://www.ctan.org/tex-archive/macros/latex/required/graphics/
% Another good source of documentation is "Using Imported Graphics in
% LaTeX2e" by Keith Reckdahl which can be found as epslatex.ps or
% epslatex.pdf at: http://www.ctan.org/tex-archive/info/
%
% latex, and pdflatex in dvi mode, support graphics in encapsulated
% postscript (.eps) format. pdflatex in pdf mode supports graphics
% in .pdf, .jpeg, .png and .mps (metapost) formats. Users should ensure
% that all non-photo figures use a vector format (.eps, .pdf, .mps) and
% not a bitmapped formats (.jpeg, .png). IEEE frowns on bitmapped formats
% which can result in "jaggedy"/blurry rendering of lines and letters as
% well as large increases in file sizes.
%
% You can find documentation about the pdfTeX application at:
% http://www.tug.org/applications/pdftex





% *** MATH PACKAGES ***
%
%\usepackage[cmex10]{amsmath}
% A popular package from the American Mathematical Society that provides
% many useful and powerful commands for dealing with mathematics. If using
% it, be sure to load this package with the cmex10 option to ensure that
% only type 1 fonts will utilized at all point sizes. Without this option,
% it is possible that some math symbols, particularly those within
% footnotes, will be rendered in bitmap form which will result in a
% document that can not be IEEE Xplore compliant!
%
% Also, note that the amsmath package sets \interdisplaylinepenalty to 10000
% thus preventing page breaks from occurring within multiline equations. Use:
%\interdisplaylinepenalty=2500
% after loading amsmath to restore such page breaks as IEEEtran.cls normally
% does. amsmath.sty is already installed on most LaTeX systems. The latest
% version and documentation can be obtained at:
% http://www.ctan.org/tex-archive/macros/latex/required/amslatex/math/





% *** SPECIALIZED LIST PACKAGES ***
%
%\usepackage{algorithmic}
% algorithmic.sty was written by Peter Williams and Rogerio Brito.
% This package provides an algorithmic environment fo e.g.~ algorithms.
% You can use the algorithmic environment in-text or within a figure
% environment to provide for a floating algorithm. Do NOT use the algorithm
% floating environment provided by algorithm.sty (by the same authors) or
% algorithm2e.sty (by Christophe Fiorio) as IEEE does not use dedicated
% algorithm float types and packages that provide these will not provide
% correct IEEE style captions. The latest version and documentation of
% algorithmic.sty can be obtained at:
% http://www.ctan.org/tex-archive/macros/latex/contrib/algorithms/
% There is also a support site at:
% http://algorithms.berlios.de/index.html
% Also of interest may be the (relatively newer and more customizable)
% algorithmicx.sty package by Szasz Janos:
% http://www.ctan.org/tex-archive/macros/latex/contrib/algorithmicx/




% *** ALIGNMENT PACKAGES ***
%
%\usepackage{array}
% Frank Mittelbach's and David Carlisle's array.sty patches and improves
% the standard LaTeX2e array and tabular environments to provide better
% appearance and additional user controls. As the default LaTeX2e table
% generation code is lacking to the point of almost being broken with
% respect to the quality of the end results, all users are strongly
% advised to use an enhanced (at the very least that provided by array.sty)
% set of table tools. array.sty is already installed on most systems. The
% latest version and documentation can be obtained at:
% http://www.ctan.org/tex-archive/macros/latex/required/tools/


%\usepackage{mdwmath}
%\usepackage{mdwtab}
% Also highly recommended is Mark Wooding's extremely powerful MDW tools,
% especially mdwmath.sty and mdwtab.sty which are used to format equations
% and tables, respectively. The MDWtools set is already installed on most
% LaTeX systems. The lastest version and documentation is available at:
% http://www.ctan.org/tex-archive/macros/latex/contrib/mdwtools/


% IEEEtran contains the IEEEeqnarray family of commands that can be used to
% generate multiline equations as well as matrices, tables, etc., of high
% quality.


%\usepackage{eqparbox}
% Also of notable interest is Scott Pakin's eqparbox package for creating
% (automatically sized) equal width boxes - aka "natural width parboxes".
% Available at:
% http://www.ctan.org/tex-archive/macros/latex/contrib/eqparbox/





% *** SUBFIGURE PACKAGES ***
%\usepackage[tight,footnotesize]{subfigure}
% subfigure.sty was written by Steven Douglas Cochran. This package makes it
% easy to put subfigures in your figures. e.g., "Figure 1a and 1b". For IEEE
% work, it is a good idea to load it with the tight package option to reduce
% the amount of white space around the subfigures. subfigure.sty is already
% installed on most LaTeX systems. The latest version and documentation can
% be obtained at:
% http://www.ctan.org/tex-archive/obsolete/macros/latex/contrib/subfigure/
% subfigure.sty has been superceeded by subfig.sty.



%\usepackage[caption=false]{caption}
%\usepackage[font=footnotesize]{subfig}
% subfig.sty, also written by Steven Douglas Cochran, is the modern
% replacement for subfigure.sty. However, subfig.sty requires and
% automatically loads Axel Sommerfeldt's caption.sty which will override
% IEEEtran.cls handling of captions and this will result in nonIEEE style
% figure/table captions. To prevent this problem, be sure and preload
% caption.sty with its "caption=false" package option. This is will preserve
% IEEEtran.cls handing of captions. Version 1.3 (2005/06/28) and later
% (recommended due to many improvements over 1.2) of subfig.sty supports
% the caption=false option directly:
%\usepackage[caption=false,font=footnotesize]{subfig}
%
% The latest version and documentation can be obtained at:
% http://www.ctan.org/tex-archive/macros/latex/contrib/subfig/
% The latest version and documentation of caption.sty can be obtained at:
% http://www.ctan.org/tex-archive/macros/latex/contrib/caption/




% *** FLOAT PACKAGES ***
%
%\usepackage{fixltx2e}
% fixltx2e, the successor to the earlier fix2col.sty, was written by
% Frank Mittelbach and David Carlisle. This package corrects a few problems
% in the LaTeX2e kernel, the most notable of which is that in current
% LaTeX2e releases, the ordering of single and double column floats is not
% guaranteed to be preserved. Thus, an unpatched LaTeX2e can allow a
% single column figure to be placed prior to an earlier double column
% figure. The latest version and documentation can be found at:
% http://www.ctan.org/tex-archive/macros/latex/base/



%\usepackage{stfloats}
% stfloats.sty was written by Sigitas Tolusis. This package gives LaTeX2e
% the ability to do double column floats at the bottom of the page as well
% as the top. (e.g., "\begin{figure*}[!b]" is not normally possible in
% LaTeX2e). It also provides a command:
%\fnbelowfloat
% to enable the placement of footnotes below bottom floats (the standard
% LaTeX2e kernel puts them above bottom floats). This is an invasive package
% which rewrites many portions of the LaTeX2e float routines. It may not work
% with other packages that modify the LaTeX2e float routines. The latest
% version and documentation can be obtained at:
% http://www.ctan.org/tex-archive/macros/latex/contrib/sttools/
% Documentation is contained in the stfloats.sty comments as well as in the
% presfull.pdf file. Do not use the stfloats baselinefloat ability as IEEE
% does not allow \baselineskip to stretch. Authors submitting work to the
% IEEE should note that IEEE rarely uses double column equations and
% that authors should try to avoid such use. Do not be tempted to use the
% cuted.sty or midfloat.sty packages (also by Sigitas Tolusis) as IEEE does
% not format its papers in such ways.





% *** PDF, URL AND HYPERLINK PACKAGES ***
%
\usepackage{url}
% url.sty was written by Donald Arseneau. It provides better support for
% handling and breaking URLs. url.sty is already installed on most LaTeX
% systems. The latest version can be obtained at:
% http://www.ctan.org/tex-archive/macros/latex/contrib/misc/
% Read the url.sty source comments for usage information. Basically,
% \url{my_url_here}.


% **** ANGUS MISC IMPORTS ***
\usepackage{booktabs}       % professional-quality tables
\usepackage{nicefrac}       % compact symbols for 1/2, etc.



% *** Do not adjust lengths that control margins, column widths, etc. ***
% *** Do not use packages that alter fonts (such as pslatex).         ***
% There should be no need to do such things with IEEEtran.cls V1.6 and later.
% (Unless specifically asked to do so by the journal or conference you plan
% to submit to, of course. )


% correct bad hyphenation here
\hyphenation{op-tical net-works semi-conduc-tor}


\begin{document}
%
% paper title
% can use linebreaks \\ within to get better formatting as desired
\title{The Ciona17 Dataset for Semantic Segmentation \\
of Invasive Species in a Marine Aquaculture Environment}


% author names and affiliations
% use a multiple column layout for up to two different
% affiliations

\iffalse
\author{\IEEEauthorblockN{Angus Galloway}
\IEEEauthorblockA{School of Engineering\\
University of Guelph\\
Guelph ON, Canada\\
Email: gallowaa@uoguelph.ca}
\and
\IEEEauthorblockN{Graham Taylor}
\IEEEauthorblockA{School of Engineering\\
University of Guelph\\
Guelph ON, Canada\\
Email: gwtaylor@uoguelph.ca}
\and
\IEEEauthorblockN{Medhat Moussa}
\IEEEauthorblockA{School of Engineering\\
University of Guelph\\
Guelph ON, Canada\\
Email: mmoussa@uoguelph.ca}
}
\fi

\author{
	\IEEEauthorblockN
	{
		Angus Galloway\IEEEauthorrefmark{1},
		Graham W. Taylor\IEEEauthorrefmark{1},
		Aaron Ramsay\IEEEauthorrefmark{2},
		Medhat Moussa\IEEEauthorrefmark{1}
	}
	\IEEEauthorblockA
	{
		\IEEEauthorrefmark{1}
		School of Engineering\\
		University of Guelph\\
		Guelph, ON, Canada\\
	    \{gallowaa, gwtaylor, mmoussa\}@uoguelph.ca\\
	    \\\IEEEauthorrefmark{2}
	    Department of Agriculture and Fisheries\\
	    Government of PEI\\
	    Montague, PEI, Canada\\
	    \{apramsay@gov.pe.ca\}\\
	}
%	\and
%	\IEEEauthorblockN
%	{
%		Aaron Ramsay \IEEEauthorrefmark{2}
%	}
%	\IEEEauthorblockA
%	{
%		\IEEEauthorrefmark{2}
%		Department of Agriculture and Fisheries\\
%		Government of PEI\\
%		Montague, PEI, Canada\\
%		\\\ apramsay@gov.pe.ca
%	}
}


% conference papers do not typically use \thanks and this command
% is locked out in conference mode. If really needed, such as for
% the acknowledgment of grants, issue a \IEEEoverridecommandlockouts
% after \documentclass

% for over three affiliations, or if they all won't fit within the width
% of the page, use this alternative format:
%
%\author{\IEEEauthorblockN{Michael Shell\IEEEauthorrefmark{1},
%Homer Simpson\IEEEauthorrefmark{2},
%James Kirk\IEEEauthorrefmark{3},
%Montgomery Scott\IEEEauthorrefmark{3} and
%Eldon Tyrell\IEEEauthorrefmark{4}}
%\IEEEauthorblockA{\IEEEauthorrefmark{1}School of Electrical and Computer Engineering\\
%Georgia Institute of Technology,
%Atlanta, Georgia 30332--0250\\ Email: see http://www.michaelshell.org/contact.html}
%\IEEEauthorblockA{\IEEEauthorrefmark{2}Twentieth Century Fox, Springfield, USA\\
%Email: homer@thesimpsons.com}
%\IEEEauthorblockA{\IEEEauthorrefmark{3}Starfleet Academy, San Francisco, California 96678-2391\\
%Telephone: (800) 555--1212, Fax: (888) 555--1212}
%\IEEEauthorblockA{\IEEEauthorrefmark{4}Tyrell Inc., 123 Replicant Street, Los Angeles, California 90210--4321}}




% use for special paper notices
%\IEEEspecialpapernotice{(Invited Paper)}




% make the title area
\maketitle


\begin{abstract} An original dataset for semantic segmentation, ``Ciona17'', is
introduced, which to the best of the authors' knowledge, is the first dataset of
its kind with pixel-level annotations pertaining to invasive species in a marine
environment. Diverse outdoor illumination, a range of object shapes, colour, and
severe occlusion provide a significant real world challenge for the computer
vision community. An accompanying ground-truthing tool for superpixel labeling,
``Truth and Crop'', is also introduced. Finally, we provide a baseline using a
variant of Fully Convolutional Networks, and report results in terms of the
standard mean intersection over union (mIoU) metric.

\end{abstract}

\begin{IEEEkeywords}
semantic segmentation; object segmentation dataset; aquaculture management;
aquatic invasive species; Ciona intestinalis; Mytilus edulis

\end{IEEEkeywords}


% For peer review papers, you can put extra information on the cover
% page as needed:
% \ifCLASSOPTIONpeerreview
% \begin{center} \bfseries EDICS Category: 3-BBND \end{center}
% \fi
%
% For peerreview papers, this IEEEtran command inserts a page break and
% creates the second title. It will be ignored for other modes.
\IEEEpeerreviewmaketitle



\section{Introduction}
% no \IEEEPARstart
In long-line mussel farming, the fouling of mussel socks and gear by
\emph{tunicates}, a marine invertebrate species, can impede mussel growth,
decrease yield, and reduce overall farm productivity
\cite{ramsay-ciona-pei,colonial-vs-ciona,ciona-treatment,netherlands}.
\emph{Ciona intestinalis} are widely
considered to be the most problematic of the tunicates in mussel farming
due to their rapid proliferation and biomass
\cite{ramsay-ciona-pei,ciona-treatment}. Aquaculture industries in
The Netherlands, New Zealand, and particularly Prince Edward Island (PEI),
Canada, are plagued by \emph{Ciona}; having the ability to displace other
invasive tunicates, such as \emph{Styela clava}
\cite{ramsay-ciona-pei,netherlands}.\par

In response to the \emph{Ciona} problem, an above-water high-pressure treatment
system, depicted in Figure~\ref{fig:spray-treatment-example}, has been in use
for several years \cite{ciona-treatment}, but this design has several
significant limitations.
Current mitigation systems do remove substantial \emph{Ciona} biomass, but they
also introduce avoidable mussel fall-off, as the force of water jets can exceed
the tensile strength of the mussels' byssal threads. \par

The fall-off problem is further exacerbated during the summer months, when
byssal threads are weaker \cite{byssal-thread-mechanics}, spawning of
\emph{Ciona} is at peak intensity \cite{ramsay-ciona-pei}, and treatments are
most frequent. Furthermore, mussel fall-off is expected to increase
with rising ocean acidification \cite{ocean-acidification}.\par

The high-pressure treatment system can be considered open-loop with respect to
the extent of infestation on a mussel sock. Quantifying invasive
species by surface area, and location, in real-time, is central to the
feasibility of an improved closed-loop computer vision-based treatment system.
Such a system, paired with electromechanical nozzles, could
strategically target invasive species, minimizing mussel fall-off as well as
the number of requisite treatments in a growing season.
% Note the use of 'as well as' here, it's deliberate since the number of
% requisite treatments is less important than minimizing mussel fall-off

Measuring the amount of biofouling on mussel socks before and after
treatment, is also desirable for making informed management decisions. Given a
cleanliness metric, farmers can counteract inconsistent treatments caused by
the use of non-standardized equipment (e.g.~different nozzles, orientation,
water pressure), and varying treatment speeds. \par

\begin{figure}[h]
	\centering
	\includegraphics[width=0.95\linewidth]{img/IMG_6135.JPG}
	\caption{Long-line farmed mussel socks with moderate \emph{Ciona}
		infestation (orange gelatinous specimen) about to enter a high-pressure
		treatment system fixed to hull of vessel. A generic design for this system
		is available \cite{PEIAA}, but each farm's equipment is made to order.
		Best viewed in colour.}
	\label{fig:spray-treatment-example}
\end{figure}

The farming context is the motivation for formulating the problem as one of
\emph{semantic segmentation} as opposed to simply \emph{object detection},
since we seek to describe \emph{where}, \emph{what kind}, and \emph{how much}
infestation is present. Counting individual object instances is of less
importance in mitigation, but may be of scientific interest in other
capacities, such as a detailed study of recruitment patterns
\cite{colonial-vs-ciona}.\par

% I don't think this belongs in the dataset paper, at least not here.
\iffalse
This paper is primarily concerned with algorithms capable of fast inference,
robustness to occlusions and diverse luminance, and producing reasonably
granular output.
\fi

\section{Related Work}

\iffalse
% mmoussa: This comment is not important. What motivated study of one method
% or another is not important for the reader. Stay focused on the purpose of
% the paragraph

While several datasets exist for bounding-box object detection [X1-XN], there
are relatively few providing per-pixel annotations. This is largely due to the
difficulty and tedious nature of manually labeling images, with a lack of
standardized tools. As a result, the same benchmarks are used repeatedly in
computer vision and machine learning communities, which is good for measuring
progress, but limits the scope of real-world applications.
\fi

In the context of semantic segmentation, to date, several major datasets have
served as common benchmarks for assessing algorithm performance. Yet,
comparatively few arise from real-world, application specific problems. In this
section we discuss some alternative datasets that facilitate supervised semantic
segmentation with pixel-level annotations, and draw comparisons with Ciona17.

\subsection{PASCAL Visual Object Classes
\cite{pascal-voc-2012}}
\iffalse
The PASCAL Visual Object Classes (VOC) annual series of competitions first
introduced a per-pixel segmentation challenge in 2007 \cite{pascal-voc-2007}.
The most recent 2012 challenge maintains the same original 21 object classes
(e.g.~aeroplane, bicycle, bird), and provides 2,913 pixel-level
segmentations \cite{pascal-voc-2012}.
\fi

The most recent PASCAL Visual Object Classes (VOC) challenge dataset has 21
object classes (e.g.~aeroplane, bicycle, bird), and provides 2,913 pixel-level
segmentations \cite{pascal-voc-2012}.

The PASCAL dataset addressed several criticisms of earlier object recognition
datasets, which lacked clutter and diverse object orientations. Given that the
PASCAL images were drawn pseudo-randomly from the consumer photo sharing website
Flickr, they tend to be representative of challenging real-world scenes.
Although the PASCAL VOC competitions have ended, the datasets continue to
be a standard benchmark, having led to
several breakthroughs in the field of deep learning for semantic segmentation
\cite{DBLP:journals/corr/LongSD14,DBLP:journals/corr/NohHH15,CP2015Semantic}.

\subsection{The Cityscapes Dataset for Semantic Urban Scene Understanding
\cite{Cordts2016Cityscapes}}

This more modern dataset extends PASCAL VOC to high-complexity urban scenes,
and features 5,000 fine pixel-level annotations, and 20,000 self described
``coarse'' annotations, for 30 different object classes (e.g.~person,
rider, car, truck) and eight broader categories (e.g.~human, vehicle, nature).
For testing, only 18 of the sufficiently unique objects are used to assess
performance.
The dataset organizers employed two forms of quality control regarding the
annotations: 1) having 30 images labeled by two different annotators, and 2)
coarse labeling of previously fine labeled images. Both measures resulted in
greater than 96\% overlap, a testament to high consistency even among numerous
annotators.

\subsection{Comparisons}

Similarly to the Ciona17 dataset, each example in PASCAL consists of an RGB
image, and segmentation mask containing integer indices corresponding to an
object class. Unlike the PASCAL dataset, Ciona17 does not contain object
instance masks, as the practical nature of farming does not require counting of
individual species. Furthermore, a single cropped image of modest resolution
could potentially contain hundreds of individual \emph{Ciona} or mussels,
making this impractical to label accurately.

In comparing the PASCAL dataset and Ciona17, the latter contains even more
occlusion and blending of classes, as the tunicates tend to grow directly on top
of, or among, the mussels. The PASCAL dataset does contain some occlusion, such
as ``person riding a bicycle'' or ``busy street scene'', however many examples
feature one or two instances of an unobstructed object, e.g.~``aeroplane in the
sky''. In comparison, many randomly oriented object instances with heavy
occlusion, is the norm rather than the exception, in Ciona17.

The Cityscapes dataset is more similar to Ciona17, as they are both
application-specific and most examples contain several crowded objects, with
some Cityscapes scenes having over a hundred instances. Cityscapes is also
highly occluded, with vehicles and pedestrians obstructing views of other
people.

\section{The Ciona17 Dataset}

We introduce a ground-truthed dataset derived from high-resolution images taken
on two different mussel farms\footnote{http://dx.doi.org/10.5683/SP/NTUOK9}.
Images were collected with a consumer-grade Canon PowerShot SD1000 7.1MP
digital camera mounted on a tripod with ISO fixed to 100. Both farms were
located in eastern PEI and experienced significant infestations
of \emph{Ciona intestinalis}. Only one of the farms, ``Farm 2'', had visible
\emph{Styela clava}. \par

No specific instruction was given to the farmers regarding
the type of crop to show. They were simply told to treat what they had
originally intended to do that day. We have high confidence that the data
collected is representative of what a typical farm would encounter during a
normal treatment cycle. Data collection occurred on two consecutive days near
the end of the tunicate spawning season in PEI, in early November 2016.
This was the last scheduled day of treatments for Farm 1, as \emph{Ciona} cease
to reproduce below water temperatures of about $8^{\circ}C$
\cite{ramsay-ciona-pei}.\par

\subsection{High Level Overview}

The Ciona17 dataset consists of 1,472 images with corresponding pixel-level
segmentations. The images are cropped to a uniform $224\times224$px, and the
following semantic labels were assigned and stored to integer masks:

\begin{itemize}
	\item \textbf{0 - Other}
	\item \textbf{1 - Mussel}
	\item \textbf{2 - Ciona}
	\item \textbf{3 - Styela}
	\item \textbf{4 - Void}
\end{itemize}

% Probably doesn't need to be a section, but thought it made sense to have this
% as part of high level overview.
%\subsubsection{Class Balance}

The ``Other'' class represented uninteresting background classes that were not
explicitly annotated, while ``Void'' was used for regions that were difficult
to label (e.g.~foreground class overlap).
The distribution of class labels is shown in Table~\ref{class-distribution-f1}.
The dataset is split into two training, and one test set. The test
set has significantly more \emph{Ciona}, and facilitates testing how resulting
algorithms handle class-imbalance. Both training sets have a similar
volume of \emph{Ciona}, but Training 2 was unique in receiving a void label
during annotation, in similar fashion to PASCAL VOC. We are aware of additional
complexities in adapting some models to use void labels, such as Fully
Convolutional Networks (FCN) \cite{DBLP:journals/corr/LongSD14} implemented in
frameworks like TensorFlow \cite{tensorflow2015-whitepaper}, therefore we keep
Training 2 separate.

\begin{table}[h]
	\caption{Class balance for training and test data for Farm 1.}
	\label{class-distribution-f1}
	\centering
	\begin{tabular}{lccccc}
		\toprule
		Split & Images & Other \% & Mussel \% & Ciona \% & Void \%\\
		\midrule
		Training 1 & 702 & 38 & 49 & 13 & 0\\
		Training 2 & 364 & 35 & 29 & 14 & 19\\
		Test & 334 & 42 & 18 & 40 & 0\\
		\bottomrule
	\end{tabular}
\end{table}

% Angus: Leaving this out for now since I don't actually refer to histogram.
\iffalse
\begin{figure}[h]
	\centering
	\includegraphics[width=0.95\linewidth]{img/low_med_high_hist_f11_annot.pdf}
	\caption{(\emph{Top}) Samples of ``low'', ``medium'', and ``high''
	 luminance sock sections taken from the same high-resolution image. The
	 ``low'' and ``medium'' sections are from the same sock, while ``high'' is
	 from an adjacent sock. All instances of \emph{Ciona} were carefully
	 annotated in orange. (\emph{Bottom}) Corresponding histograms (\emph{left
	 y-axis}), and cumulative pixel density (\emph{right y-axis}). Original
	 image was downsampled by factor of two (2) and cropped to 200$\times$200px
	 region of interest. Best viewed in colour.}
	\label{fig:low-med-high-hist}
\end{figure}
\fi

\subsection{Truth and Crop Tool}

To facilitate quick ground truthing of full size images, a platform-independent
GUI utility ``Truth and Crop'' was implemented in
Qt\footnote{https://github.com/AngusG/truth-and-crop}. This tool let the
annotator label whole superpixels, rather than individual pixels, or by drawing
polygons. Using the tool, an annotator performs an initial ground truth, and
can then click arbitrarily to save cropped image-segmentation mask pairs.

\iffalse
The study of CIELAB colourspace in Section \ref{cielab} is partly
motivated by its use in SLIC as part of the 5D clustering procedure.
\fi

The simple linear iterative clustering (SLIC) superpixel extraction
algorithm \cite{slic} is used to automatically find boundaries around similar
parts or edges in the image. The SLIC algorithm implemented in the Python
scikit-image \cite{scikit-learn} library receives as arguments the number of
roughly equal size segments requested, width of Gaussian smoothing kernel,
$\sigma$, and a segment compactness factor. Once superpixel clusters have been
identified, they can be imposed on the original image, and the annotator
assigns a class label to all pixels in a superpixel with a single click. This
process is depicted in Figure~\ref{fig:truth-and-crop}.

\iffalse
% Angus: probably makes sense to omit
It is worth noting that
superpixels are only used for creating ground truth segmentations, and are not
required during training nor inference
\cite{DBLP:journals/corr/HariharanAGM14,Farabet2013}.
\fi

\begin{figure}[h]
	\centering
	\includegraphics[width=0.95\linewidth]{img/ground-truthing-5351-ds4-nseg300-sig3-2c.png}
	\caption{Screenshot of ``Truth and Crop'' utility with superpixel segments
	extracted via SLIC \cite{scikit-learn} for full size image downsampled by
	four (4), with $\sigma=3$, and 300 roughly equal sized segments. Two mussel
	socks, and two Styrofoam buoys are present, with lime-green bounding boxes
	where cropped regions are to be extracted. (\emph{Left}) Ground
	truthed regions of sock have a transparent colour fill, with green for
	1-Mussel, and purple for 2-Ciona. Regions that are not filled are
	considered 0-Other after cropping operation.
	(\emph{Right}) Image not ground truthed to show raw segments. Best viewed
	in colour.}
	\label{fig:truth-and-crop}
\end{figure}

\subsection{Ground Truthing Procedure}

Images were ground truthed by the primary author after meeting with
aquaculture biologists, and spending significant time manually inspecting live
mussel socks.

Through the Truth and Crop GUI, SLIC hyperparameters were set manually on a
per-image basis, while spending roughly 10-15 minutes per image for the whole
annotation procedure. An optional down-sampling factor, usually of no more than
two or three,  was first applied to the original image in both height and width
dimensions. This helped ensure that samples were drawn from a variety of
scales, and made the labeling process faster. After labeling foreground
classes, all other pixels were automatically assigned to the ``Other'' class.

Figure~\ref{fig:dataset_sample} depicts some of the challenging decisions that
had to be made during the annotation process. Some difficult images
resulted in superpixels that contained several classes, e.g.~background and a
mussel, or mussel and \emph{Ciona}. In general, for cases where the annotator
estimated that each class was roughly equally represented in a superpixel, the
void label was assigned. Since all labels had to be assigned to a whole
superpixel, the void label is more prominent than the thin border line that
appears in PASCAL, however this was still considered efficient compared to
manually drawing polygons with alternative tools such as LabelMe
\cite{Russell08labelme:a}. \par

\begin{figure}[h]
	\centering
	\includegraphics[width=0.95\linewidth]{img/raw_and_mask_with_void.pdf}
	\caption{Ciona17 sample of mussel sock top section from Farm 1. \emph{a)}
		RGB image, \emph{b)} segmentation mask imposed on \emph{a)}, and
		\emph{c)} segmentation mask with \emph{Ciona} as green area, mussels as
		blue area, and void as white area. Best viewed in colour.}
	\label{fig:dataset_sample}
\end{figure}

The majority of superpixels resulted in much sharper edges in well illuminated
areas than would have been feasible to draw in LabelMe. Superpixels that
contained only \emph{Ciona} and mussel were assigned the majority class. The
void label was also used in some cases that were too difficult to discern,
usually in areas with particularly low light.

To ensure good coverage of annotated regions, there was some overlap between
images, similarly to what can be observed in Figure~\ref{fig:truth-and-crop}.
We emphasize that for the splits outlined in Table~\ref{class-distribution-f1},
there was no overlap between the training and test sets as these images
were cropped from different original images.

\subsection{Dataset Features}

We explained previously how the Ciona17 dataset relates to open
problems in aquaculture. Now, a case is made for Ciona17 as a challenging
and interesting computer vision dataset in general.

\subsubsection{Variable Lighting}

Although the weather was clear and sunny during both days that images were
collected, sock sections cast in shadow make accurate segmentation
difficult, even for the human eye. Figure~\ref{fig:low-med-high-hist} shows
how challenging it can be to identify clusters of \emph{Ciona} on mussel socks
in broad daylight, due to a range of illumination and shadow inherent in
outdoor imagery.

\begin{figure}[h]
	\centering
	\includegraphics[width=0.95\linewidth]{img/three_samples_variable_lighting_annot.pdf}
	\caption{Samples of \emph{a)} ``low'', \emph{b)} ``medium'', and \emph{c)}
	``high'' illumination sock sections taken from the same original image.
	Sections \emph{a)} and \emph{b)} are from the same sock, while \emph{c)} is
	from an adjacent sock. All instances of \emph{Ciona} were carefully
	annotated in orange.
	% Original image was downsampled by factor of two (2) and cropped to
	% 200$\times$200px region of interest.
	% Terrance: Why is this important (I agree it's not important since
	% this is not part of the actual dataset).
	Best viewed in colour.}
	\label{fig:low-med-high-hist}
\end{figure}

\subsubsection{Colour}

In theory, colour thresholding should separate mussels from the mostly
bright orange \emph{Ciona}, but this assumes direct sunlight, and lack of
silt or debris. It has also been speculated that \emph{Ciona} tend to
darken with age \cite{aaron-email-ciona-colour-age}. The sample in
Figure~\ref{fig:ciona_diverse_colour} shows a typical spectrum of colour for
\emph{Ciona}.

\begin{figure}[h]
	\centering
	\includegraphics[width=0.95\linewidth]{img/raw_and_mask_with_void_ciona_range_colour.pdf}
	\caption{Ciona17 sample of predominantly \emph{Ciona} with diverse colour
		from Farm 2. \emph{a)} RGB image, \emph{b)} segmentation mask imposed
		on \emph{a)}, and \emph{c)} segmentation mask with \emph{Ciona} as
		green area, and void as white area. Two superpixels had to be voided
		due to mussel grouped together with debris and background. Best viewed
		in colour.}
	\label{fig:ciona_diverse_colour}
\end{figure}

Additionally, some mussel shells reflect a similar orange colour to that of
\emph{Ciona}, an effect visible in Figure~\ref{fig:low-med-high-hist} \emph{b)}
and \emph{c)}. This tends to occur naturally around the seam where the shell
halves meet, but silt also contributes to a more global effect. \par

\subsubsection{Texture}

\emph{Styela clava} are the mostly highly textured of all species in the
dataset, but their colour is less distinguishable from mussels. Older
\emph{Ciona} have a finely pitted texture as their tunic thickens, while
\emph{Styela} consistently possess a coarse sandpaper-like appearance.

\begin{figure}[h]
	\centering
	\includegraphics[width=0.55\linewidth]{img/IMG_6521_nseg900_sig2_ds3_2_x651_y401.jpg}
	\caption{Sample of individual \emph{Ciona} and \emph{Styela} from Farm 2.
	\emph{Left} \emph{Ciona} and \emph{Right} \emph{Styela}. Best viewed in
	colour.}
	\label{fig:ciona_styela_texture}
\end{figure}

\subsubsection{Occlusion}

Farm 2 employed a technique known as ``double socking''
in which mussel socks are wrapped in-place with an additional mesh to
prevent fall-off. This mesh can be considered as added occlusion in the
dataset and is a common practice on some farms. The double socking made precise
annotation extremely difficult, therefore we have held out this data for
future consideration (e.g.~for use in unsupervised or semi-supervised learning).

\section{Evaluating Performance}

To penalize naive schemes that optimize for background detection, we suggest
using the standard ``Mean Intersection over Union'' (mIoU) metric, commonly
used in semantic segmentation \cite{Everingham2015}. For a given class, and
true positives (TP), false positives (FP), and false negatives (FN), the
intersection over union (IoU) is given by:

\begin{equation}
IoU = \frac{TP}{TP+FP+FN}
\end{equation}

Averaging the IoU scores for each class results in the overall mIoU score. In
this way, a scheme that overwhelmingly predicts background will have little
intersection with masks for mussels or \emph{Ciona}, as no credit is earned for
true negatives.

\subsection{Baseline Result}

To establish a baseline mIoU score on the Ciona17 dataset, a variant of FCN
\cite{DBLP:journals/corr/LongSD14} outlined in 
Table~\ref{tab:VGG6S-Fc6-512-Deconv}, was trained end-to-end with
pixelwise softmax cross-entropy loss and ReLU activations. 

\begin{table}[h]
	\caption{Fully Convolutional Network architecture (VGG6S-Fc6-512-Deconv) used
  as our baseline.}
	\label{tab:VGG6S-Fc6-512-Deconv}
	\centering
	\begin{tabular}{llcl}
		\toprule
		Layer Type & Kernel & Stride & Output Shape \\
		\midrule
		Input & n/a & n/a & 224$\times$224$\times$3 \\
		\midrule
		Conv\_1 & 3$\times$3 & 1 & 224$\times$224$\times$16 \\
		Pool\_1 & 2$\times$2 & 2 & 112$\times$112$\times$16 \\
		\midrule
		Conv\_2 & 3$\times$3 & 1 & 112$\times$112$\times$32 \\
		Pool\_2 & 2$\times$2 & 2 & 56$\times$56$\times$32 \\
		\midrule
		Conv\_3 & 3$\times$3 & 1 & 56$\times$56$\times$64 \\
		Pool\_3 & 2$\times$2 & 2 & 28$\times$28$\times$64 \\
		\midrule
		Conv\_4 & 3$\times$3 & 1 & 28$\times$28$\times$128 \\
		Pool\_4 & 2$\times$2 & 2 & 14$\times$14$\times$128 \\
		\midrule
		Conv\_5 & 3$\times$3 & 1 & 14$\times$14$\times$256 \\
		Pool\_5 & 2$\times$2 & 2 & 7$\times$7$\times$256 \\
		\midrule
		Fc\_6 & 1$\times$1 & 1 & 7$\times$7$\times$512 \\
		Deconv\_1 & 64$\times$64 & 32 & $ 224 \times 224 \times 3$ \\
		\bottomrule
	\end{tabular}
\end{table}

Training was performed with 
the adaptive moment estimation (Adam) gradient descent scheme 
\cite{DBLP:journals/corr/KingmaB14}, initial learning rate of $1 \times 
10^{-5}$, mini-batch size of 10, and dropout with a dropout rate of 0.5 on 
layer \texttt{Fc\_6}. The RGB images were not preprocessed other than a simple 
conversion of the 8-bit pixel intensities to floating point values in the range 
[0-1]. Training was interrupted at 100k steps, re-starting the optimizer from
scratch, and allowed to continue until 300k total steps had elapsed 
resulting in $mIoU_{test} = 51.36\%$. Some samples that were drawn from the 
trained model are shown in Figure~\ref{fig:vgg6s_benchmark_samples}. 

In practice, it may be acceptable to merge the ``other'' and ``mussel''
classes, turning the problem into that of foreground/background binary
segmentation, as a closed loop treatment system only needs to distinguish
\emph{Ciona} from everything else. Despite this, farmers may wish to
contextualize the \emph{Ciona} measure with mussel biomass, hence the need to
report IoU across all classes.

\begin{figure}[h]
	\centering
	\includegraphics[width=0.95\linewidth]{img/vgg6s_benchmark_samples.png}
	\caption{Three sets of \emph{a)} RGB image sample from Ciona17 test set,
	\emph{b)} prediction of trained VGG6S model, and \emph{c)} segmentation
	mask with \emph{Ciona} as red, mussels as green, and other as blue. Best
	viewed in colour.}
	\label{fig:vgg6s_benchmark_samples}
\end{figure}


\iffalse
In terms of pixels, we define $n_{ii}$ as a correctly classified pixel,
$n_{ij}$ as pixel from class $i$ predicted to be of class $j$, and $N_c$ as the
number of classes. Then, $t_i = \sum_{j} n_{ij}$ is the number of pixels of
class $i$. Mean IU is then defined as:

\begin{equation}\label{miu}
mIoU = \frac{1}{N_c} \sum_{i} n_{ii}/\bigg(t_i + \sum_{j} n_{ji} - n_{ii}\bigg)
\end{equation}
\fi

\iffalse
In the remaining sections, there was little practical benefit to correctly
separating all three classes, therefore classes 0-Other and 1-Mussel were
combined into a single class, ``not Ciona'' at run time. Guessing the mode (not
\emph{Ciona}) results in a test pixel accuracy of $41.73+17.90=59.63\%$,
but this trivial scheme is actually worse than the existing open-loop system
since no Ciona would ever be detected or treated. To penalize schemes such as
this, we evaluate alternative means of evaluating the classifier in
Section~\ref{sec:miu}.
\fi

\iffalse

% mmoussa: You are mainly providing a new dataset to the community.
% reprocessing and  post-processing should be left to each researcher.

% I suggest removing these sections and add more details on how the labeling
% was done since that is important for the integrity of the data and also few
% details on how the data collection was performed. This should give you still
% 4 pages.

\section{Preprocessing}

Typically manual preprocessing is not performed when using deep convolutional
networks, however some alternative colourspaces, and histogram based
preprocessing were investigated given intuition about colour and lightness.

\subsection{CIELAB}
\label{cielab}

As the dataset consists of images that are taken outdoors, the luminance was a
highly variable factor. Depending on the time of day, the socks may be either
well-lit, or completely occluded by shadow. The two classes of interest, namely
the \emph{common blue mussel} and \emph{Ciona intestinalis} have fairly
consistent colour profiles provided the scene is well lit. Under certain
orientations however, an orange seam is visible on the mussels, much the same
colour as the Ciona.

\begin{figure}[h]
	\centering
	\includegraphics[width=0.95\linewidth]{img/train_image_lab_att.pdf}
	\caption{(\emph{Left}) RGB image from training set, and from left to right,
	$L^*$, $a^*$, and $b^*$ channels in OpenCV byte format. Best viewed in
	colour. }
	\label{fig:train-samp-lab}
\end{figure}

The CIELAB color representation consists of three channels, ``L*'' for
luminosity, chromaticity ``a*'', and ``b*'' representing representing colour on
red-green, and blue-yellow axis respectively. Colour can be represented in
Euclidean distance in the a* and b* channels, which naturally controls for
differences in image brightness. The channels are rescaled to the range 0-255
in a scheme adopted from OpenCV, and saved as separate images, later
reconstructed into a single data structure, due to forced colour-mapping and
scaling in Matplotlib.

\[
L^* = L^* \times \nicefrac{255}{100}; a^* = a^* + 128; b^* = b^* + 128
\]

A sample image converted from RGB into $L^*$, $a^*$, and $b^*$ channels is
shown in Figure~\ref{fig:train-samp-lab}.

\subsection{Corrections to RGB Image}

Three global corrections were evaluated on sample images, including contrast
stretching, histogram equalization, and adaptive equalization, all depicted in
Figure~\ref{fig:medium-illumination-correction}. Contrast stretching and
adaptive equalization yield more visually appealing, natural images as
evidenced by their continuous histograms. The histogram equalized sample
results in a washed out, artificial image, but it eliminates shadows
significantly. Thus, histogram equalization was compared to the raw RGB case.

\begin{figure}[h]
	\centering
	\includegraphics[width=0.95\linewidth]
	{img/IMG_5158_w100_ds2_i0_x887_y407_orig_eqhist_adapthist_rescale_annot_4x.pdf}
	\caption{(\emph{Top}) Medium luminance sample with \emph{Ciona} annotated
	in orange from Figure~\ref{fig:low-med-high-hist} . From left to right,
	original image, image with constrast stretched between 5th and 95th
	percentile, histogram equalization, and adaptive equalization
	\cite{scikit-image}. (\emph{Bottom}) Corresponding histograms (\emph{left
	y-axis}), and cumulative pixel density (\emph{right y-axis}). Original
	image was downsampled by factor of two (2) and cropped to 200$\times$200px
	region of interest. Best viewed in colour.}
	\label{fig:medium-illumination-correction}
\end{figure}
\fi


% An example of a floating figure using the graphicx package.
% Note that \label must occur AFTER (or within) \caption.
% For figures, \caption should occur after the \includegraphics.
% Note that IEEEtran v1.7 and later has special internal code that
% is designed to preserve the operation of \label within \caption
% even when the captionsoff option is in effect. However, because
% of issues like this, it may be the safest practice to put all your
% \label just after \caption rather than within \caption{}.
%
% Reminder: the "draftcls" or "draftclsnofoot", not "draft", class
% option should be used if it is desired that the figures are to be
% displayed while in draft mode.
%
%\begin{figure}[!t]
%\centering
%\includegraphics[width=2.5in]{myfigure}
% where an .eps filename suffix will be assumed under latex,
% and a .pdf suffix will be assumed for pdflatex; or what has been declared
% via \DeclareGraphicsExtensions.
%\caption{Simulation Results}
%\label{fig_sim}
%\end{figure}

% Note that IEEE typically puts floats only at the top, even when this
% results in a large percentage of a column being occupied by floats.


% An example of a double column floating figure using two subfigures.
% (The subfig.sty package must be loaded for this to work.)
% The subfigure \label commands are set within each subfloat command, the
% \label for the overall figure must come after \caption.
% \hfil must be used as a separator to get equal spacing.
% The subfigure.sty package works much the same way, except \subfigure is
% used instead of \subfloat.
%
%\begin{figure*}[!t]
%\centerline{\subfloat[Case I]\includegraphics[width=2.5in]{subfigcase1}%
%\label{fig_first_case}}
%\hfil
%\subfloat[Case II]{\includegraphics[width=2.5in]{subfigcase2}%
%\label{fig_second_case}}}
%\caption{Simulation results}
%\label{fig_sim}
%\end{figure*}
%
% Note that often IEEE papers with subfigures do not employ subfigure
% captions (using the optional argument to \subfloat), but instead will
% reference/describe all of them (a), (b), etc., within the main caption.


% An example of a floating table. Note that, for IEEE style tables, the
% \caption command should come BEFORE the table. Table text will default to
% \footnotesize as IEEE normally uses this smaller font for tables.
% The \label must come after \caption as always.
%
%\begin{table}[!t]
%% increase table row spacing, adjust to taste
%\renewcommand{\arraystretch}{1.3}
% if using array.sty, it might be a good idea to tweak the value of
% \extrarowheight as needed to properly center the text within the cells
%\caption{An Example of a Table}
%\label{table_example}
%\centering
%% Some packages, such as MDW tools, offer better commands for making tables
%% than the plain LaTeX2e tabular which is used here.
%\begin{tabular}{|c||c|}
%\hline
%One & Two\\
%\hline
%Three & Four\\
%\hline
%\end{tabular}
%\end{table}


% Note that IEEE does not put floats in the very first column - or typically
% anywhere on the first page for that matter. Also, in-text middle ("here")
% positioning is not used. Most IEEE journals/conferences use top floats
% exclusively. Note that, LaTeX2e, unlike IEEE journals/conferences, places
% footnotes above bottom floats. This can be corrected via the \fnbelowfloat
% command of the stfloats package.

\section{Conclusion}

To the best of the authors' knowledge, Ciona17 is the first dataset of its
kind with pixel-level annotations pertaining to invasive species in a marine
environment. It is the authors' intent that in making this dataset available,
the research community will propose new models that are particularly adept
at controlling for highly variable illumination and occlusions, exceeding the
initial benchmark mIoU of 51.36\%.

In making the original images and tools used for annotation available, we
expect that improved ways of annotating challenging datasets such as
this one will be proposed.
% conference papers do not normally have an appendix


% use section* for acknowledgement
\section*{Acknowledgment}

The authors thank Brian Gillis of the \emph{PEI Department of Agriculture and
Fisheries} for assisting with the data acquisition. We also thank Colin
Reynolds of \emph{Reynold's Island Mussels}, and Dana Drummond's crew at
\emph{Atlantic Aqua Farms} for offering their farms for data collection.

% trigger a \newpage just before the given reference
% number - used to balance the columns on the last page
% adjust value as needed - may need to be readjusted if
% the document is modified later
%\IEEEtriggeratref{8}
% The "triggered" command can be changed if desired:
%\IEEEtriggercmd{\enlargethispage{-5in}}

% references section

% can use a bibliography generated by BibTeX as a .bbl file
% BibTeX documentation can be easily obtained at:
% http://www.ctan.org/tex-archive/biblio/bibtex/contrib/doc/
% The IEEEtran BibTeX style support page is at:
% http://www.michaelshell.org/tex/ieeetran/bibtex/
%\bibliographystyle{IEEEtran}
% argument is your BibTeX string definitions and bibliography database(s)
%\bibliography{IEEEabrv,../bib/paper}
%
% <OR> manually copy in the resultant .bbl file
% set second argument of \begin to the number of references
% (used to reserve space for the reference number labels box)

\iffalse
\begin{thebibliography}{1}

\bibitem{IEEEhowto:kopka}
H.~Kopka and P.~W. Daly, \emph{A Guide to \LaTeX}, 3rd~ed.\hskip 1em plus
  0.5em minus 0.4em\relax Harlow, England: Addison-Wesley, 1999.

\end{thebibliography}
\fi

\bibliography{master}
\bibliographystyle{ieeetr}

% that's all folks
\end{document}

crv2017_ciona_dataset.bbl
\begin{thebibliography}{10}

\bibitem{ramsay-ciona-pei}
A.~Ramsay, J.~Davidson, D.~Bourque, and H.~Stryhn, ``Recruitment patterns and
  population development of the invasive ascidian ciona intestinalis in {Prince
  Edward Island, Canada},'' {\em Aquatic Invasions}, vol.~4, 2009.

\bibitem{colonial-vs-ciona}
C.~Paetzold, D.~Giberson, J.~Hill, and J.~Davidson, ``Effect of colonial
  tunicate presence on ciona intestinalis recruitment within a mussel farming
  environment,'' {\em Management of Biological Invasions}, vol.~3, 2012.

\bibitem{ciona-treatment}
J.~Davidson, T.~Landry, G.~Johnson, and P.~Quij{\'o}n, ``A cost-benefit
  analysis of four treatment regimes for the invasive tunicate ciona
  intestinalis on mussel farms,'' {\em Management of Biological Invasions},
  vol.~8, 2016.

\bibitem{netherlands}
A.~Gittenberger, ``Invasive tunicates on {Zeeland and Prince Edward Island}
  mussels, and management practices in {The Netherlands},'' {\em Aquatic
  Invasions}, vol.~4, 2009.

\bibitem{byssal-thread-mechanics}
G.~M. Moeser and E.~Carrington, ``Seasonal variation in mussel byssal thread
  mechanics,'' {\em The Journal of Experimental Biology}, 2006.

\bibitem{ocean-acidification}
M.~J. O'Donnell, M.~N. George, and E.~Carrington, ``Mussel byssus attachment
  weakened by ocean acidification,'' {\em Nature Climate Change}, vol.~3,
  pp.~587--590, 2013.

\bibitem{PEIAA}
{Prince Edward Island Aquaculture Alliance}, ``Final report: The development
  and early commercialization of four innovative tunicate treatment systems for
  mussel aquaculture on {PEI},'' tech. rep., Department of Fisheries and Oceans
  Aquaculture and Innovation Market Access Program, March 2010.

\bibitem{pascal-voc-2012}
M.~Everingham, L.~Van~Gool, C.~K.~I. Williams, J.~Winn, and A.~Zisserman, ``The
  {PASCAL} {V}isual {O}bject {C}lasses {C}hallenge 2012 {(VOC2012)}
  {R}esults.''
  http://www.pascal-network.org/challenges/VOC/voc2012/workshop/index.html.

\bibitem{DBLP:journals/corr/LongSD14}
J.~Long, E.~Shelhamer, and T.~Darrell, ``Fully convolutional networks for
  semantic segmentation,'' in {\em CVPR}, 2015.

\bibitem{DBLP:journals/corr/NohHH15}
H.~Noh, S.~Hong, and B.~Han, ``Learning deconvolution network for semantic
  segmentation,'' in {\em ICCV}, vol.~00, pp.~1520--1528, 2015.

\bibitem{CP2015Semantic}
L.-C. Chen, G.~Papandreou, I.~Kokkinos, K.~Murphy, and A.~L. Yuille, ``Semantic
  image segmentation with deep convolutional nets and fully connected crfs,''
  in {\em ICLR}, 2015.

\bibitem{Cordts2016Cityscapes}
M.~Cordts, M.~Omran, S.~Ramos, T.~Rehfeld, M.~Enzweiler, R.~Benenson,
  U.~Franke, S.~Roth, and B.~Schiele, ``The cityscapes dataset for semantic
  urban scene understanding,'' in {\em Proc. of the IEEE Conference on Computer
  Vision and Pattern Recognition (CVPR)}, 2016.

\bibitem{tensorflow2015-whitepaper}
M.~Abadi, A.~Agarwal, P.~Barham, E.~Brevdo, Z.~Chen, C.~Citro, G.~S. Corrado,
  A.~Davis, J.~Dean, M.~Devin, S.~Ghemawat, I.~Goodfellow, A.~Harp, G.~Irving,
  M.~Isard, Y.~Jia, R.~Jozefowicz, L.~Kaiser, M.~Kudlur, J.~Levenberg,
  D.~Man\'{e}, R.~Monga, S.~Moore, D.~Murray, C.~Olah, M.~Schuster, J.~Shlens,
  B.~Steiner, I.~Sutskever, K.~Talwar, P.~Tucker, V.~Vanhoucke, V.~Vasudevan,
  F.~Vi\'{e}gas, O.~Vinyals, P.~Warden, M.~Wattenberg, M.~Wicke, Y.~Yu, and
  X.~Zheng, ``{TensorFlow}: Large-scale machine learning on heterogeneous
  systems,'' 2015.
\newblock Software available from tensorflow.org.

\bibitem{slic}
R.~Achanta, A.~Shaji, K.~Smith, A.~Lucchi, P.~Fua, and S.~Susstrunk, ``Slic
  superpixels compared to state-of-the-art superpixel methods,'' {\em IEEE
  Trans. Pattern Anal. Mach. Intell.}, vol.~34, pp.~2274--2282, Nov. 2012.

\bibitem{scikit-learn}
F.~Pedregosa, G.~Varoquaux, A.~Gramfort, V.~Michel, B.~Thirion, O.~Grisel,
  M.~Blondel, P.~Prettenhofer, R.~Weiss, V.~Dubourg, J.~Vanderplas, A.~Passos,
  D.~Cournapeau, M.~Brucher, M.~Perrot, and E.~Duchesnay, ``Scikit-learn:
  Machine learning in {P}ython,'' {\em Journal of Machine Learning Research},
  vol.~12, pp.~2825--2830, 2011.

\bibitem{Russell08labelme:a}
B.~C. Russell, A.~Torralba, K.~P. Murphy, and W.~T. Freeman, ``{LabelMe}: A
  database and web-based tool for image annotation,'' 2008.

\bibitem{aaron-email-ciona-colour-age}
A.~Ramsay. Private Communication, 2017.
\newblock PEI Department of Agriculture and Fisheries.

\bibitem{Everingham2015}
M.~Everingham, S.~M.~A. Eslami, L.~Van~Gool, C.~K.~I. Williams, J.~Winn, and
  A.~Zisserman, ``The {Pascal} visual object classes challenge: A
  retrospective,'' {\em International Journal of Computer Vision}, vol.~111,
  no.~1, pp.~98--136, 2015.

\bibitem{DBLP:journals/corr/KingmaB14}
D.~P. Kingma and J.~Ba, ``Adam: {A} method for stochastic optimization,'' in
  {\em ICLR}, 2015.

\end{thebibliography}
