3D Pose from Detections
Cosimo Rubino %{\tt\small cosimo.rubino@iit.it
First Author
%Institution1
%Institution1 address
%{\tt\small firstauthor@i1.org
We present a novel method to infer, in closed-form, a general 3D spatial occupancy and orientation of a collection of rigid objects given 2D image detections from a sequence of images. In particular, starting from 2D ellipses fitted to bounding boxes, this novel multi-view problem can be reformulated as the estimation of a quadric (ellipsoid) in 3D. We show that an efficient solution exists in the dual-space using a minimum of three views while a solution with two views is possible through the use of regularization. However, this algebraic solution can be negatively affected in the presence of gross inaccuracies in the bounding boxes estimation. To this end, we also propose a robust ellipse fitting algorithm able to improve performance in the presence of errors in the detected objects. Results on synthetic tests and on different real datasets, involving real challenging scenarios, demonstrate the applicability and potential of our method.
2015_arxiv_sfd.tex
\documentclass[10pt,twocolumn,letterpaper]{article}

\usepackage{iccv}
\usepackage{times}
\usepackage{epsfig}
\usepackage{graphicx}
\usepackage{amsmath}
\usepackage{amsthm,amsfonts,amssymb}
\usepackage{pdfsync}
\usepackage{subfig}
\usepackage[font=footnotesize]{caption}
\usepackage{multirow}
% Include other packages here, before hyperref.

\setlength{\textfloatsep}{6pt} 
\setlength{\abovecaptionskip}{6pt}

\input{defs}
\include{macros3}
%%\newcommand\norm[1]{\left\lVert#1\right\rVert}33993
\newcommand{\minus}{\scalebox{0.5}[1.0]{$-$}}
\DeclareMathOperator*{\argmin}{arg\,min}
\DeclareMathOperator*{\argmax}{arg\,max}

% If you comment hyperref and then uncomment it, you should delete
% egpaper.aux before re-running latex.  (Or just hit 'q' on the first latex
% run, let it finish, and you should be clear).
%\usepackage[breaklinks=true,bookmarks=false]{hyperref}

\iccvfinalcopy % *** Uncomment this line for the final submission

\def\iccvPaperID{411} % *** Enter the ICCV Paper ID here
\def\httilde{\mbox{\tt\raisebox{-.5ex}{\symbol{126}}}}

% Pages are numbered in submission mode, and unnumbered in camera-ready
%\ificcvfinal\pagestyle{empty}\fi
%\setcounter{page}{1}
\begin{document}

%%%%%%%%% TITLE
\title{3D Pose from Detections}

\author{Cosimo Rubino
%{\tt\small cosimo.rubino@iit.it}
% For a paper whose authors are all at the same institution,
% omit the following lines up until the closing ``}''.
% Additional authors and addresses can be added with ``\and'',
% just like the second author.
% To save space, use either the email address or home page, not both
\and
Marco Crocco
%{\tt\small marco.crocco@iit.it}
\and
Alessandro Perina
%{\tt\small alessandro.perina@iit.it}
\and
Vittorio Murino
%{\tt\small vittorio.murino@iit.it}
\and
Alessio Del Bue \\
\\
Pattern Analysis and Computer Vision Department (PAVIS)\\
Istituto Italiano di Tecnologia\\
Via Morego 30, 16163 Genova, Italy %\\
%{\tt\small alessio.delbue@iit.it}
}
%\author{First Author\\
%Institution1\\
%Institution1 address\\
%{\tt\small firstauthor@i1.org}
%% For a paper whose authors are all at the same institution,
%% omit the following lines up until the closing ``}''.
%% Additional authors and addresses can be added with ``\and'',
%% just like the second author.
%% To save space, use either the email address or home page, not both
%\and
%Second Author\\
%Institution2\\
%First line of institution2 address\\
%{\tt\small secondauthor@i2.org}
%}

\maketitle
%\thispagestyle{empty}


%%%%%%%%% ABSTRACT
\begin{abstract}
   We present a novel method to infer, in closed-form, a general 3D spatial occupancy and orientation of a collection of rigid objects given 2D image detections from a sequence of images. In particular, starting from 2D ellipses fitted to bounding boxes, this novel multi-view problem can be reformulated as the estimation of a quadric (ellipsoid) in 3D. We show that an efficient solution exists in the dual-space using a minimum of three views while a solution with two views is possible through the use of regularization. However, this algebraic solution can be negatively affected in the presence of gross inaccuracies in the bounding boxes estimation. To this end, we also propose a robust ellipse fitting algorithm able to improve performance in the presence of errors in the detected objects. Results on synthetic tests and on different real datasets, involving real challenging scenarios, demonstrate the applicability and potential of our method.
\end{abstract}

%%%%%%%%% BODY TEXT
\section{Introduction} \label{sec:intro}

Localising and finding the pose of objects in generic scenes is a fundamental step for higher level scene understanding. This task is of such importance that major efforts have been put in Computer Vision research in order to obtain efficient detectors from single images. These methods give remarkable results in finding several objects categories in natural scenes and thus putting the basis for the understanding of the visual world. However, while these results are certainly compelling, detection algorithms have been mostly restricted to the estimation of the objects position in 2D. When imaging the same scene from different viewpoints, a crucial question is  how to deal with both 3D geometrical reasoning and higher-level object representation.  

Recent works have clearly pointed out that this lack of 3D reasoning is limiting and that bridging the gap between object detection and multi-view geometry might provide surprising improvements in classical approaches. Starting from the work of Hoeim et al. \cite{Hoiem:etal:2008}, the inclusion of 3D scene reasoning and rules has provided higher detection accuracies. % up to the estimation of coarse 3D geometry from single views \cite{gupta:etal:2010}. 
Notably, attempts of unifying geometry and object representation have been achieved by defining elaborated Maximum a Posteriori (MAP) inference \cite{bao:savarese:2011} or bundle adjustment with objects \cite{Fioraio:DiStefano:2013}. This way of pursuing high-level object reasoning in multi-view geometry has also inspired novel methods in Simultaneous Localisation and Mapping (SLAM) \cite{salas:etal:2013}. % where 3D objects are detected and localised in real time.  
Semantic information has been used, on the other hand, to infer the 3D shape of objects and the cameras viewpoints  \cite{vicente2014reconstructing}.
%
 % Remarkably, 3D object recognition modelling has been shown feasible using single images only \cite{zia:etal:2013,Zia2014}. 
Differently, several works attempted to tackle 3D pose estimation from single images only. This severely under-constrained problem has to be solved with the use of strong semantic information (geometrical and physical constraints, i.e. object lying on the ground plane), 2D appearance and shape in the form of 3D wireframe or CAD models \cite{silberman:etal:2012,zia:etal:2013,Zia2014,xiang2012estimating}. Hejrati and Ramanan \cite{hejrati2014analysis} instead use an analysis by synthesis approach that, guided by visual evidence, selects matching HOG patches that best represent a 3D object.


\begin{figure}[t]
\centering
%\fbox{\rule{0pt}{2in} \rule{0.9\linewidth}{0pt}}
\includegraphics[width=0.90\linewidth]{./figures/intro.png}
\vspace{-0.8cm}
  \caption{Example of a set of images of a given 3D rigid scene taken from a camera at different viewpoints. The problem is to recover the 3D occupancy of each object given the 2D BBs detected at each image frame.}
\label{fig:fig1}
\end{figure}

This work takes a different path from previous 3D pose estimation methods by showing that it is possible to obtain accurate 3D object localisations and orientations in closed-form using multi-view relations given only a set of image detections, i.e. 2D bounding boxes (BBs). To the best of the authors knowledge, this work is the first to provide such solution in 3D scene understanding. The solution proposed here recalls standard Structure from Motion (SfM) approaches \cite{Tomasi:Kanade:1992,sturm:triggs:1996} where multiple views of the same rigid scene are used to obtain a 3D reconstruction given a set of 2D feature points measurements. Similarly, we define a novel problem, namely \textit{Pose from Detections (PfD)}, that attempts to estimate the 3D pose (position and orientation) of an object from a set of BBs obtained by general purpose detectors in multiple views (see Fig. \ref{fig:fig1} for a graphical representation). It is worth noting that the occupancy boundary encapsulates the estimation of object center, pose, size and aspect ratio in 3D. Moreover, differently from previous works, we do not need geometric or semantic priors out of 2D bounding boxes from detections, nor advanced detectors yielding pose categorization. 
In particular, we show that there exists an efficient closed-form solution to 3D object localisation for a minimum number of three views if the problem is reformulated as the estimation of a quadric in 3D. This solution is not viable to derive using  BBs (which is a piecewise-defined curve), but is mathematically feasible given a set of ellipses fitted to the original 2D BBs as extracted by the detectors. We also show that for the ill-posed case of two views, it is possible to apply regularisation obtaining performance close to the well-posed solution. % based on priors over the object's aspect ratio that are reasonably accurate given the known class of the detector (e.g. cars, pedestrians). 
Moreover, regularisation boosts results whenever the BBs are inaccurate as it might happen in realistic scenarios. Finally, we further introduce a robust segmentation algorithm that can increase the reliability of the 2D ellipse fitting stage by finding the object profile inside the BBs. In such a way, improved results are achieved in challenging real scenarios on different freely available datasets.

 %we also propose a robust regularization method that can provide reliable results in challenging real scenarios on different freely available datasets. We further improve robustness by proposing a 
The rest of the paper is structured as follows. Section \ref{sec:problem_statement} defines the problem and the related mathematical formalisation. Section \ref{sec:dualsf} presents the PfD solution while Section \ref{sec:fittreg} describes the regularization method and robust ellipse fitting approach. Experiments on real and synthetic data are discussed in Section \ref{sec:exp} and then followed by concluding remarks in Section \ref{sec:concl}.


\section{PfD problem statement} \label{sec:problem_statement}
%In this work we focus on inferring a scene, associating to each object a bounding ellipsoid which approximates its position and volume.
%\subsection{Problem statement} \label{subsec:ellfromdet}
%An object detector recognizes objects $o$ in a frame $f$, enclosing them inside bounding boxes 
Let us consider a set of image frames $f=1 \dots F$  representing a 3D scene under different viewpoints. A set of $i=1 \dots N$ rigid objects is placed in arbitrary positions and each object can be detected in each of the $F$ images.  Each object $i$ in each image frame $f$ is identified by a 2D BB $B_{if}$ given by a generic object detector. The BB is defined by a triplet of parameters: $ B_{if} = \left\lbrace  w_{if}, h_{if},\vec c_{if}\right\rbrace$, where $w_{if}$ and $h_{if}$ are two scalars defining the BB height and width respectively and $\vec c_{if}$ is a $2$-vector defining the BB center.

Our goal is to estimate the pose (position and orientation) of each object in the 3D scene given the 2D BBs using multi-view constraints. %This problem has a strong analogy with SfM approaches where the 3D position of points is recovered using the scene rigidity assumption. 
In order to ease the mathematical formalization of the problem, we move from a BB representation of an object to an ellipsoid one. This is done by associating at each $B_{if}$ an ellipse fitting $\hat{\mat C}_{if}$ that inscribes the BB, as shown in Fig. \ref{fig:example}. 

\begin{figure}
	\centering 
	\includegraphics[width=8cm]{./figures/plotBbx.png}\hspace{.1cm} 
	\caption {Example of BBs (yellow) and corresponding fitted ellipses (blue) for a set of objects.}
	\label{fig:example}
\end{figure}


%In particular, each ellipse is centered in $\vec c_{if}$, it has the two axes aligned to the image sides, with axes length equal to $w_{if}$ and $h_{if}$. 
The aim of our problem is to find the 3D ellipsoids $\mat Q_i$ %\in \mathbb{R}^{4 \times 4}$, 
whose projections on the image planes, associated to each frame $f= 1 \dots F$, best fit the 2D ellipses $\hat{\mat C}_{if}$ in the image plane. %(see Fig. \ref{fig:example})
This will solve for both the 3D localisation and occupancy of each object starting from the image detections in the different views.

%which best approximates the  3D intersection of all the conics generated from the outlines projections $\C_{if} \in \mathbb{R}^(3 \times 3)$ in the 2D frames (as in figure \ref{fig:??}).
%We associate to each $B_{if}$ its inscribed ellipse $\mat C_{if}$, which includes intrinsicly the three information of the bounding box.
%To get the ellipse from the bounding box $ B_{if}$ we define the functional $H$ as follow:
%\begin{equation} \label{eq:bbx2ell}
%   B(w_{if}, h_{if},\vec c_{if})_{if} \stackrel{H}{\rightarrow} C( w_{if}, h_{if},\vec c_{if})
%\end{equation}
%which associates an ellipse to each bounding box. 
%In this case each ellipse has the two axes aligned to the image sides, centered in $\vec c_{if}$ with axes length equal to $w_{if}$ and $h_{if}$.\\
%Let $ B_{if} = \left\lbrace  w_{if}, h_{if},\vec c_{if}\right\rbrace$ be a set of bounding boxes associated to the detected object  $i=1 \dots N$ inside a frame $f=1 \dots F$; for each of them we associate an inscribed ellipse.\\
In the following, we represent each ellipse using the homogeneous quadratic form of a conic equation:
\begin{equation} \label{eq:con}
\vec u^{\top} \hat{\mat{C}}_{if} \,\, \vec u =0,
\end{equation}
%with $\mat C_{f} \in \mathbb{R}^{3 \times 3}$ :
%\begin{equation} \label{eq:theta2}
%\mat C_f = 
%\begin{bmatrix}
%\mat C_{22} & \vec c \\
%\vec c^{\top} & c_{33}\\
%\end{bmatrix}
%\end{equation}
where $\vec u \in \mathbb{R}^{3}$ is the homogeneous vector of a generic 2D point belonging, to the conic defined by the symmetric matrix $ \hat{\mat{C}}_{if} \in \mathbb{R}^{3 \times 3}$. %\footnote{With slight  abuse of notation we denote as $\hat{\mat{C}}_{if}$ both the ellipse and the matrix defining the related quadratic form. The same is done for the ellipsoid.}.
%and has the first minor $\mat E_{3,3}$ positive semidefinite. 
The conic has five degrees of freedom, given by the six elements of the lower triangular part of the symmetric matrix $\hat{\mat{C}}_{if}$ except one for the scale, since Eq. (\ref{eq:con}) is homogeneous in $\vec{u}$ \cite{hartley2003multiple}. %In fact, multiplying the matrix $\mat{C}_{if}$ for a scalar factor, the set of points satisfying Eq. (\ref{eq:con}) does not change \cite{hartley2003multiple}.\\
Similarly to the ellipses, we represent the ellipsoids in the 3D space with the homogeneous quadratic form of a quadric equation:
\begin{equation} \label{eq:quad}
\vec x^\top \mat Q_i \, \vec x = 0, 
\end{equation}
%with $ \mat Q \in \mathbb{R}^{4 \times 4}$:\\
%\begin{equation} \label{eq:Theta}
%\mat Q = 
%\begin{bmatrix}
%\mat Q_{{33}} & \vec q \\
%\vec q^\top   & q_{44} \\
%\end{bmatrix}
%\end{equation}
where $\vec x \in \mathbb{R}^{4}$ represents an homogeneous 3D point belonging to the quadric defined by the symmetric matrix  $\mat Q_i \in \mathbb{R}^{4 \times 4}$. The quadric has nine degrees of freedom, given by the ten elements of the symmetric matrix $\mat Q_i$ up to one for the overall scale.
%In the particular case of an ellipse, which represents the outline, we define the conic ${\mat{C}}_{if}$.
 
Each quadric $\mat{Q}_i$, when projected onto the image plane, gives a conic denoted with $\mat{C}_{if} \in \mathbb{R}^{3 \times 3}$.
The relationship between $\mat{Q}_i$ and $\mat{C}_{if}$ is defined by the projection matrices $\mat P_{f} \in \mathbb{R}^{3 \times 4}$ associated to each frame. Such matrices can be estimated from the image sequence using standard self-calibration methods \cite{hartley2003multiple,pollefeys1999self}.% $\mat P_{f} = \mat K_{f} [\mat R_{f} |\vec t_{f}] \in \mathbb{R}^{3 \times 4}$, where $\mat K_{f} \in \mathbb{R}^{3 \times 3}$, $\mat R_{f} \in \mathbb{R}^{3 \times 3}$ and $\vec t_{f} \in \mathbb{R}^{3 \times 1}$ are respectively the calibration matrix, rotation matrix and translation vector of the cameras associated to each frame. All the projection matrices $\mat P_{f}$ for $f = 1 \cdots F$ are assumed to be known (i.e. the camera is calibrated).
%Given a set of objects inside a 3D scene and considering all the frames in which the object is present, we will have for each frame $f$ an associated ellipse $\mat C_{if}$: starting from this outlines we want to recover the 3D occupancy described by the associated ellipsoid $\mat Q_i$.\\

\section{Dual space fitting} \label{sec:dualsf}
%
%Because the relationship between $\mat{Q}_i$ and $\mat{C}_{if}$ is not straightforward in the primal space, i.e. the euclidean space of 3D points in the space and 2D points on the images, it is convenient to reformulate it in the dual space, i.e. the space of the planes spanning the 3D euclidean space and the space of the lines spanning the 2D euclidean space \cite{Cross} (FIXME ADB: to rephrase later, too wordy).
Since the relationship between $\mat{Q}_i$ and $\mat{C}_{if}$ is not straightforward in the primal space, i.e. the Euclidean space of 3D points (2D points in the images) , it is convenient to reformulate it in dual space, i.e. the space of the planes (lines in the images)  \cite{Cross}. 
%Given this problem statement, it is convenient to solve this fitting problem in the dual space \cite{Cross}.\\
In particular, the conics in 2D can be represented by the envelope of all the lines tangent to the conic curve, while the quadrics in 3D can be represented by the envelope of all the planes tangent to the quadric surface. % (Figure \ref{fig:??}).
%Assuming that we know one projection matrix for each frame $\mat P_{f} = \mat K_{f} [\mat R_{f} |\vec t_{f}] \in \mathbb{R}^{3 \times 4}$, where $\mat K_{f} \in \mathbb{R}^{3 \times 3}$  and $\mat R_{f} \in \mathbb{R}^{3 \times 3}$ are respectively the camera calibration matrix and rotation matrix, 
%In particular for every plane $\pi$ tangent to the quadric $$
Hence, the dual quadric is defined by the matrix  $\mat Q^*_i = adj(\mat Q_i)$, where $adj$ is the adjoint operator, and the dual conic is defined by ${\mat{C}}^*_{if} = adj({\mat C}_{if})$ \cite{hartley2003multiple}. %as the dual of $\mat Q_i$; the $adjoint$ of a matrix is the inverse multiplied by its determinant. 
Considering that the dual conic ${\mat C}^*_{if}$, like the primal one, is defined up to an overall scale factor $\beta_{if}$, the relation between a dual quadric and its dual conic projections ${\mat C}^*_{if}$ can be written as: %\cite{hartley2003multiple}: %putting a scale factor $\beta_{if} \in \mathbb{R}^+$ doesn't affect the equation of ellipse:
%\begin{equation}
%\beta_{if} \vec u^{\top} \mat{C}_{if} \,\, \vec u =0
%\end{equation}
%the outline of a quadric could be written by the following relation:
%
\begin{equation} \label{eq:dualq2c}
\beta_{if} {\mat{C}}^*_{if} =  \mat P_{f}  \mat Q^*_i  \mat P^{\top}_{f}.
\end{equation}
%
%where $\mat C^*_{if}$ represents the dual conic tangent to the quadric in the camera pose $f$.\\
In order to recover $\mat Q^{*}_{i}$ in closed form from the set of dual conics $ \lbrace {\mat C}^*_{if} \rbrace_{f=1 \dots F} $, we have to re-arrange Eq. (\ref{eq:dualq2c}) into a linear system. Let us define  $\vec v^{*}_{i} = vech (\mat Q_i^*)$ and ${\vec c}^{*}_{if} =vech ({\mat C}^*_{if})$ as the vectorization of symmetric matrices $\mat Q^{*}_{i}$ and ${\mat C}^*_{if}$ respectively\footnote{The operator $vech$ serializes the elements of the lower triangular part of a symmetric matrix, such that, given a symmetric matrix $\mat X \in \mathbb{R}^{n \times n}$, the vector $\vec x$, defined as $\vec x = vech(\mat X)$, is $\vec x \in \mathbb{R}^{g} $ with $ g=\frac{n(n+1)}{2}$.}. % the vectorization of the dual quadric $\mat Q^{*}_{i}$,
%\begin{equation}
%\vec v^{*}_{i} =
%\arraycolsep=1.1pt
%\begin{bmatrix}
%q_{i,11}^* & q_{i,12}^* & q_{i,13}^* & q_{i,14}^* & q_{i,22}^* & q_{i,23}^* & q_{i,24}^* & q_{i,33}^* & q_{i,34}^* & q_{i,44}^*
%\end{bmatrix}
%\end{equation}
%which has once the ten symmetric elements.\\
Then, let us arrange the products of the elements of $\mat P_{f}$ and $\mat P^{\top}_{f}$ in a unique matrix $\mat G_{f} \in \mathbb R^{6 \times 10}$ as follows \cite{henderson1979vec}:  %in Eq. (\ref{eq:mxG}), which is quadratic in the elements $p_{hk}$ of $\mat P_{f}$.  
\begin{equation}
\mat G_f  = \mat D (\mat P \otimes \mat P) \mat E
\end{equation}
where $\otimes$ is the Kronecker product and matrices $\mat D \in \mathbb{R}^{6 \times 9}$ and $\mat E \in \mathbb{R}^{16 \times 10}$ are two matrices such that $vech (\mat X) =\mat D \hspace{0.1 cm} vec(\mat X)$ and $vec (\mat X) =\mat E \hspace{0.1 cm} vech(\mat X)$ respectively,  where $\mat X$ is a symmetric matrix\footnote{The operator $vec$ serializes all the elements of a generic matrix.}.
%
%\begin{table*}[t]
%\begin{equation}
%\arraycolsep=1.7pt
%\scriptsize
%\mat G_f \scalebox{0.5}[1.0]{$=$} 
% \left[ \begin {array}{cccccccccc} {p_{{11}}}^{2}&2\,p_{{12}}p_{{11}}&
%2\,p_{{13}}p_{{11}}&2\,p_{{14}}p_{{11}}&{p_{{12}}}^{2}&2\,p_{{13}}p_{{
%12}}&2\,p_{{14}}p_{{12}}&{p_{{13}}}^{2}&2\,p_{{13}}p_{{14}}&{p_{{14}}}
%^{2}\\ \noalign{\medskip}p_{{21}}p_{{11}}&p_{{21}}p_{{12}}+p_{{22}}p_{
%{11}}&p_{{23}}p_{{11}}+p_{{21}}p_{{13}}&p_{{24}}p_{{11}}+p_{{21}}p_{{
%14}}&p_{{22}}p_{{12}}&p_{{22}}p_{{13}}+p_{{23}}p_{{12}}&p_{{22}}p_{{14
%}}+p_{{24}}p_{{12}}&p_{{23}}p_{{13}}&p_{{23}}p_{{14}}+p_{{24}}p_{{13}}
%&p_{{24}}p_{{14}}\\ \noalign{\medskip}p_{{31}}p_{{11}}&p_{{31}}p_{{12}
%}+p_{{32}}p_{{11}}&p_{{33}}p_{{11}}+p_{{31}}p_{{13}}&p_{{34}}p_{{11}}+
%p_{{31}}p_{{14}}&p_{{32}}p_{{12}}&p_{{32}}p_{{13}}+p_{{33}}p_{{12}}&p_
%{{32}}p_{{14}}+p_{{34}}p_{{12}}&p_{{33}}p_{{13}}&p_{{33}}p_{{14}}+p_{{
%34}}p_{{13}}&p_{{34}}p_{{14}}\\ \noalign{\medskip}{p_{{21}}}^{2}&2\,p_
%{{22}}p_{{21}}&2\,p_{{23}}p_{{21}}&2\,p_{{24}}p_{{21}}&{p_{{22}}}^{2}&
%2\,p_{{23}}p_{{22}}&2\,p_{{24}}p_{{22}}&{p_{{23}}}^{2}&2\,p_{{23}}p_{{
%24}}&{p_{{24}}}^{2}\\ \noalign{\medskip}p_{{31}}p_{{21}}&p_{{31}}p_{{
%22}}+p_{{32}}p_{{21}}&p_{{33}}p_{{21}}+p_{{31}}p_{{23}}&p_{{34}}p_{{21
%}}+p_{{31}}p_{{24}}&p_{{32}}p_{{22}}&p_{{32}}p_{{23}}+p_{{33}}p_{{22}}
%&p_{{32}}p_{{24}}+p_{{34}}p_{{22}}&p_{{33}}p_{{23}}&p_{{33}}p_{{24}}+p
%_{{34}}p_{{23}}&p_{{34}}p_{{24}}\\ \noalign{\medskip}{p_{{31}}}^{2}&2
%\,p_{{32}}p_{{31}}&2\,p_{{33}}p_{{31}}&2\,p_{{34}}p_{{31}}&{p_{{32}}}^
%{2}&2\,p_{{33}}p_{{32}}&2\,p_{{34}}p_{{32}}&{p_{{33}}}^{2}&2\,p_{{33}}
%p_{{34}}&{p_{{34}}}^{2}\end {array} \right]
%\label{eq:mxG}
%\end{equation}
%\end{table*}
%%Defining in the same way  $\hat{\vec c}^{*}_{if} =vech\left(\hat{\mat C}^*_{if}\right)$, 
Given $\mat G_{f}$, we can rewrite Eq. (\ref{eq:dualq2c}) as:
%Each element of $\vec v^{*}_{i}$ is mapped in $\hat{\vec c}^{*}_{if}$, the vectorized form of $\hat{\mat C}^*_{if}$:\\
%If we divide all the elements of $c^{*}_{if}$ by its last element, assigning this value to $\beta_f$, the equation \ref{eq:dualq2c} becomes:
\begin{equation} \label{eq:eqOut}
\beta_{if} {\vec c}^*_{if} = \mat G_{f} \vec v^{*}_{i}.
\end{equation}
%then $\vec c_i$ i the reprojection of the ellipsoid up to a scale.
In order to get a unique solution for $\vec v^{*}_i$ at least three image frames are needed. Therefore, stacking column-wise 
%Starting from outlines, three views are needed at least to get a unique solution. 
Eqs. (\ref{eq:eqOut}) for $f = 1 \dots F$, with $F \geq 3$, we have that:
%For a number $f$ of views, the equation system can be written in matrix form as:
\begin{equation}\label{eq:eqsys}
\mat M_{i} \vec w_{i} = \vec 0_{6F},
\end{equation}
where the matrix $\mat M_i \in \mathbb{R}^{6F \times (10+F)}$ and the vector $\vec w \in \mathbb{R}^{10+F}$ are defined as follows:
\begin{equation}\label{eq:eqsysmx}
%\renewcommand{\arraystretch}{1.4}
\mat M_{i} = 
\arraycolsep=1.4pt
\begin{bmatrix}
\mat G_1 		& \minus {\vec c}_{i1}^* 	& \vec 0_6 		& \vec 0_6 		& \ldots & \vec 0_6 \\
\mat G_2 	& \vec 0_6   	& \minus {\vec c}_{i2}^{*}	& \vec 0_6  		& \ldots & \vec 0_6 \\
\mat G_3 	& \vec 0_6  	& \vec 0_6 		& \minus {\vec c}_{i3}^{*}  & \ldots & \vec 0_6 \\
\vdots 		& \vec 0_6  	& \vec 0_6 		& \vec 0_6   		& \ddots & \vec 0_6 \\
\mat G_F	& \vec 0_6  	& \vec 0_6 		& \vec 0_6   		& \ldots & \minus {\vec c}_{iF}^{*} \\
\end{bmatrix} \;\;\;
\vec w_{i} = 
\begin{bmatrix}
\vec v_i^*\\
\vec{\beta}_i
\end{bmatrix}
%\vec w_{i} = 
%\begin{bmatrix}
%\vec v_i^* \\
%\beta_{i1} \\
%\beta_{i2} \\
%\beta_{i3} \\
%\vdots \\
%\beta_{iF} \\
%\end{bmatrix}
\end{equation}
%
with 
\begin{equation}
\vec{\beta}_i = \left[ \beta_{i1},\beta_{i2},\cdots,\beta_{iF} \right]^{\top}
\end{equation}
and $\vec 0_{d} $ being a zero column vector of length $d$.
%Given the equation \ref{eq:eqsys}, if all the $\vec c_{if}^*$ are the vectorized dual of the exact reprojection of the ellipsoid, the solution $\vec w_{i}$ is unique for less than a scale factor and corresponds to the %null space
%The solution of Eq. (\ref{eq:eqsys}) is unique for less than a scale factor and corresponds to the %null space
%right singular vector of the null singular value of the matrix $\mat M_{i}$.

Note that in real cases the ellipses $\hat{\mat C}_{if}$ computed by a general purpose object detector might be inaccurate regarding the location of the BB and the window size. Likewise, this will have an effect on the ellipse fitting, inducing an error on the $\mat C_{if}$.
%
% from the bounding boxes generally differ from the exact projections $\hat{\mat C}_{if}$ of the ellipsoid. Therefore Eq. (\ref{eq:eqsysmx}) has to be rewritten in the following way: 
%\begin{equation}\label{eq:eqsys1}
%\tilde{\mat M}_i \vec w_{i} \approx \vec 0_{6F}
%\end{equation}
%where the matrix $\tilde{\mat M}_i \in \mathbb{R}^{6F \times (10+F)}$ is defined as follows:
%\begin{equation}\label{eq:eqsysmx1}
%%\renewcommand{\arraystretch}{1.4}
%\tilde {\mat M}_{i} = 
%\arraycolsep=1.4pt
%\begin{bmatrix}
%\mat G_1 	& \minus {\vec c}_{i1}^* 		& \vec 0_6 							& \vec 0_6 					& \ldots & \vec 0_6 				\\
%\mat G_2 	& \vec 0_6   					& \minus {\vec c}_{i2}^{*}			& \vec 0_6  				& \ldots & \vec 0_6 				\\
%\mat G_3 	& \vec 0_6  					& \vec 0_6 							& \minus {\vec c}_{i3}^{*}  & \ldots & \vec 0_6 				\\
%\vdots 		& \vec 0_6  					& \vec 0_6 							& \vec 0_6   				& \ddots & \vec 0_6 				\\
%\mat G_F	& \vec 0_6  					& \vec 0_6 							& \vec 0_6   				& \ldots & \minus {\vec c}_{iF}^{*} \\
%\end{bmatrix}
%\end{equation}
%with $\vec c_{if}^*=vech\left( adj\left( \mat C_{if} \right)\right)$.\\
%
For this reason, if $\tilde{\mat M}_i$ is the matrix given by object detections %, we have that instead of looking for the right singular vector corresponding to the null singular value
%In our case the final ellipsoid should be the volume which best fits all the ellipses, which are generated from the bounding boxes (\ref{eq:bbx2ell}).
%A way to reach this goal is to
we can find the solution by minimizing: %we use an algebraic minimization. In particular the problem to be solved is:
\begin{equation} \label{eq:argmin}
 \tilde{\vec{w}}_{i}= \arg \min_{\vec w_i} \| \tilde{\mat M}_i \vec w_i \|_2^2 \hspace{0.3cm} s.t. \hspace{0.2cm} \|\vec w_i\|^2_2=1,
\end{equation}
where the equality constraint $\|\vec w_i\|^2_2=1$ avoids the trivial zero solution. 
The solution to the minimization problem in Eq. (\ref{eq:argmin}) can be obtained via $\mat{SVD}$ on the $\tilde{\mat M}_i$ matrix, taking the right singular vector associated to the minimum singular value.
%
The first $10$ rows of $\tilde{\vec{w}}_i$  are the vectorized elements of the estimated dual quadric denoted by $\tilde{\vec v}^*_i$. To get back the estimated matrix of the quadric in the primal space, we obtain first the dual estimated quadric by $\tilde{\mat{Q}}^*_i = vech^{-1}( {\tilde{\vec v}_i^* })$, and subsequently apply the following relation:
\begin{equation} \label{eq:invAdj}
 \tilde{\mat{Q}}_{i} = |\det{\tilde{\mat{Q}}^*_i}|^{\frac{1}{3}} (\tilde{\mat{Q}}^*_i)^{\minus 1}.
%\label{primal}
\end{equation}
%
%The overall computational load of the method is very low, since the most demanding operation is the computation of an SVD on the matrix $\tilde{\mat M}_i$ whose size is proportional to the number of views.
%
\section{Regularisation and robust ellipse fitting} \label{sec:fittreg}
%\section{Robust ellipsoid fitting} \label{sec:fittreg}
%
%
Image detectors can provide inaccurate results given occlusions, illumination variations and complex object poses. %
To this end, we propose a regularisation approach for Eq. (\ref{eq:argmin}) and a method to re-adjust the object  orientation and size from the BBs extracted in the image sequence.

\subsection{Regularized cost function} \label{subsec:reg}
%
As seen in the previous section, moving to the dual space allows an efficient linearisation of the problem. However this implies a drawback: the algebraic minimization is carried on the dual quadrics in Eq. (\ref{eq:argmin}) and the primal one is obtained by a matrix inversion as in Eq. (\ref{eq:invAdj}). In the presence of noise the matrix $\tilde{\mat{Q}}^*_i$ may become ill-conditioned and therefore small errors may cause relatively large errors in the estimated primal quadric. This problem is particularly evident with the ill-posed case of two views, where the solution to Eq. \ref{eq:eqsys} is a 1-dimensional family of quadrics (i.e. is not unique), or whenever few camera views are spanning a limited range of angles. In such a case, the estimated quadric may result in a nearly degenerate ellipsoid, with dramatically unbalanced axes lengths, or even in wrong quadrics such as an hyperboloid. 

To tackle this problem, we propose to add a regularization term that penalizes the departure of the estimated quadric from a sphere of a given size and center. In other words, we include a regularization on the aspect ratios of the objects modeled by the ellipsoids. %The underlying assumption is that the aspect ratios of objects modeled by ellipsoids are FIXME...FORSE SI PUO' RIFORMULARE IN TERMINI DI PRIOR SUGLI OGGETTI. 
A sphere in the dual space, centered at the origin, can be represented as a $4 \times 4$ diagonal matrix with the first three diagonal elements positive and equal each other and the fourth diagonal element negative. To account for arbitrary translations, it is sufficient  to pre- and post-multiply the diagonal matrix by a translation matrix. Hence a generic sphere in the dual space can be written as a function of a vector of five parameters  $\vec s=\left[ t_1,t_2,t_3,a,b \right]^{\top}$ as: 
\begin{equation}
\mat S^*(\vec s) = \mat T \mat D \mat T^{\top}
\end{equation}
with 
\begin{equation}
\mat T = 
\begin{bmatrix}
1 & 0 & 0 & t_1\\
0 & 1 & 0 & t_2\\
0 & 0 & 1 & t_3\\
0 & 0 & 0 & 1\\
\end{bmatrix},
\hspace{.6cm}
%\label{T}
%\end{equation}
%%
%\begin{equation}
\mat D=
\begin{bmatrix}
a & 0 & 0 & 0\\
0 & a & 0 & 0\\
0 & 0 & a & 0\\
0 & 0 & 0 & \minus b\\
\end{bmatrix},
\label{D}
\end{equation}
with $a>0$ and  $b>0$.
Thus, the regularization term can be defined as: 
\begin{equation}
R\left(\hat{\vec{v}}^*_i,\vec s \right) = \|  \hat{\vec{v}}^*_i -vech \left(\mat S^*\left(\vec s \right)\right)\|^2_2
\label{reg}
\end{equation}
where $\hat{\vec{v}}^*_i$ is the vectorized $i$-th dual quadric normalized by minus its last (tenth) element giving: 
\begin{equation}
\hat{\vec{v}}^*_i= \vec{v}^*_i/\left(-\vec v_i^*(10)\right).
\label{normal}
\end{equation}
Such normalization avoids the trivial solution when minimizing for Eq. (\ref{reg}). The minus sign has been set in order to preserve the coherence with the constraint $b>0$.\\
Finally, the solution to the regularized problem can be found solving for:  
\begin{equation}
\tilde{\vec f}= \arg \min_{\vec f}\|\tilde{\mat M}_i \hat{\vec w}_i \|^2_2+\lambda R\left(\hat{\vec{v}}^*_i,\vec s \right) \hspace{0.3 cm} s.t. \hspace{0.2 cm} a >0, \hspace{0.1 cm} b > 0
\label{reg_cost}
\end{equation}
where 
\begin{equation}
\vec f = \left[ \hat{\vec w}_i^{\top} \hspace{0.2 cm}  \vec s^{\top} \right]^{\top},  \hspace{1cm} \hat{\vec w}_i = \left[\hat{\vec v}_i^{*\top} \hspace{0.2 cm} \vec{\beta}_i^{\top} \right]^{\top}.
\end{equation}
%and 
%\begin{equation}
%\hat{\vec w}_i = \left[\hat{\vec v}_i^{*\top} \hspace{0.2 cm} \vec{\beta}_i^{\top} \right]^{\top}
%\end{equation}
%
%\begin{equation}
%\hat{\vec w}_i = 
%\begin{bmatrix}
%\hat{\vec v}_i^* \\
%\beta_{i1} \\
%\beta_{i2} \\
%\beta_{i3} \\
%\vdots \\
%\beta_{iF} \\
%\end{bmatrix}
%\end{equation}
%
%
%The normalization avoids the trivial solution when minimizing \ref{reg_cost} and at the same time substitutes the
Note that the normalization in Eq. (\ref{normal}) avoids the need for the quadratic equality constraint $\|\vec w_i\|^2_2=1$ in Eq. (\ref{reg_cost}). 
%The alternative approach of normalizing the vector $\vec w_i$ by $\|\vec w_i\|_2$ has been discarded since the symbolic derivatives would become too onerous to be evaluated at each iteration. 
Finally, the cost function in Eq. (\ref{reg_cost}) is minimized with a nonlinear least squares procedure. %\cite{} %with bound constraints $\alpha>0$, $\beta>0$.
\paragraph{Initialisation.} Since the convexity of the cost function cannot be guaranteed, a good initialization is mandatory, in particular for the elements of the dual quadric $\hat{\vec{v}}^*_i$. 

%The simplest solution of initializing $\hat{\vec{v}}^*_i$%$\hat{\vec{w}}_i$,
%with the normalized result of the SVD solving for Eq. (\ref{eq:argmin}), i.e. $\tilde{\vec{v}}^*_i$, delivers poor solutions. In fact, if $\tilde{\vec{v}}^*_i$ is a degenerate (or close to be) ellipsoid, the convergence to an acceptable solution may be difficult.
%Instead, the adopted strategy consists in evaluating

First of all we evaluated the sphere with the same center and the same volume of the quadric represented by $\tilde{\vec{v}}^*_i$. Next, we initialized both $\hat{\vec{v}}^*_i$ and the parameters $\vec s$ according to such a sphere.
In detail, let us denote the starting guess values as $a^{(0)}$ and $b^{(0)}$ given by: 
\begin{equation}
%a^{(0)}=\frac{1}{\sqrt[3]{|e_{i1}e_{i2}e_{i3}|}}, \hspace{1cm}  b^{(0)}=1,
a^{(0)}= {|e_{i1}e_{i2}e_{i3}|}^{-1/3}, \hspace{1cm}  b^{(0)}=1,
\label{alpha0}
\end{equation}
%\begin{equation}
%b^{(0)}=1
%\end{equation}  
where $e_{i1}$, $e_{i2}$ and $e_{i3}$ are the three elements of a vector $\vec e_i$ defined as:
\begin{equation}
\vec e_i=\frac{-det(\tilde{\mat Q}_i)}{det(\tilde{\mat Q}_{i,3\times 3})}eig(\tilde{\mat Q}_{i,3\times 3}) 
\end{equation}
where $eig()$ is the operator that computes the eigenvalues of a square matrix and $\tilde{\mat Q}_{i,3\times 3}$ is the $3 \times 3$ principal submatrix of the primal quadric $\tilde{\mat Q}_i$ found by Eq. (\ref{eq:invAdj})\footnote{
We recall that the volume of an ellipsoid is proportional to $\sqrt{\frac{1}{|e_{i1}e_{i2}e_{i3}|}}$ and the volume of a sphere is proportional to $\sqrt{a^3}$.}. If the initial $\tilde{\mat Q}_i$ is not an ellipsoid, obviously the concept of volume preservation does not hold any more. However the initialization strategy, thanks to the modulus in Eq. (\ref{alpha0}) guarantees to start from a feasible solution corresponding to a sphere.
The initialized translation terms  $t_{1}^{(0)}$ $t_{2}^{(0)}$ and $t_{3}^{(0)}$ are set equal to the three translation parameters extracted by $\tilde{\mat Q}_i$. 

The initial vector $\vec{v}_{i}^{*^{(0)}}$ 
is set equal to $vech(\mat T^{(0)} \mat D^{(0)} \mat T^{(0)^{\top}})$, where $\mat T^{(0)}$ and $\mat D^{(0)}$ are defined by substituting $a^{(0)}$, $b^{(0)}$, $t_{1}^{(0)}$, $t_{2}^{(0)}$ and $t_{3}^{(0)}$ to the corresponding variables in Eq. (\ref{D}). Finally  $\vec{\beta}_{i}^{(0)}=\tilde{\vec{\beta}}_i/\left(-\tilde{\vec v}_i^*(10)\right)$, where $\tilde{\vec{\beta}}_i$ is the vector of scale factors corresponding to the elements of $\tilde{\vec w}_i$ from the $11$-th to the last. 
% 
%
%
%To this end we propose a method to re-adjust the object position and orientation from the bounding boxes extracted in the image sequence.
%
%\subsection{Regularisation of Eq. (\ref{eq:argmin})}
%
%Moving to the dual space allows an efficient linearisation of the PfD problem. However this implies a drawback: the algebraic minimization is carried on the dual quadrics in Eq. (\ref{eq:argmin}) and the primal one is obtained by a matrix inversion as in Eq. (\ref{eq:invAdj}). In the presence of noise the matrix $\tilde{\mat{Q}}^*_i$ may become ill-conditioned and therefore small errors may cause relatively large errors in the estimated primal quadric. %The problem of finding the reconstruction of an ellipsoid given a set $ \left\lbrace \mat  C_{i} \right\rbrace_{f=1 \dots F} $ of ellipses includes a series of matrix inversions. To cope with this we apply a regularization technique \cite{tikhonov1977solutions} to the problem of passing from the dual space to the primal space. The regularization we used is \textit{diagonal loading}; in our case it consist in putting a penality on the off-diagonal therms of the non-homogeneous part of the matrix $\mat M_i$, in order to regularize the matrix inversion when the matrix becomes ill-conditioned. Consequently, the minimization problem can be written as:
%
%To cope with this eventuality, we impose a regularisation that enforces the well-conditioning of the estimated dual quadric. A standard procedure to do this, known as diagonal loading, consists in adding a diagonal matrix of constants. %The same effect can be obtained by multiplying all the off-diagonal matrix elements for a constant lower than $1$. %However an \emph{a posteriori} regularization on the matrix $\tilde{\mat{Q}}^*_i$ would be risky, since the regularized matrix could depart too much from the exact one, thus yielding to a high algebraic error. For this reason, we prefer to 
%Following this criterion, we add a penalty to the cost function in Eq. (\ref{eq:argmin}), that concerns only the elements of the vector $\vec w_i$ corresponding to off-diagonal elements of the dual quadric. In this way well-conditioning of the dual quadric is enforced, granting at the same time a low algebraic error in the dual space. 
%
%In detail, the problem (\ref{eq:argmin}) is modified as follows: 
%\begin{equation} \label{eq:argminReg}
% \hat{\vec{w}}_{i}= \arg \min_{\vec w_i} (\norm{ \tilde{\mat M}_i \vec w_i }_2^2+ \gamma \norm{\hat{\mat I}\vec w_i}_2^2) \hspace{0.3cm} s.t. \hspace{0.2cm} \|\vec w_i\|^2_2=1,
%\end{equation}
%where $\gamma$ is the penalty weight and the matrix $\hat{\mat{I}} \in \mathbb{R}^{(10+F) \times (10+F)}$ is defined as: 
%
%\begin{equation}
% \hat{\mat{I}} = diag \left( 
% \begin{bmatrix}
% vech(\mat H)^{\top} & \vec 0_{F}^{\top} 
% \end{bmatrix} 
% \right),
%\end{equation}
%
%where the operator $diag$ defines the mapping from a vector to a diagonal matrix and the matrix $\mat H$ is given by $\mat H = \vec 1_{4}  {\vec 1_{4}}^\top - \mat I��?$ where $\vec 1_{4}$ and $\vec 0_{F}$ are vectors of $4$ ones and $F$ zeros respectively.  
%\begin{equation}
%\mat H = 
%\begin{bmatrix}
%0 & 1 & 1 & 1\\
%1 & 0 & 1 & 1\\
%1 & 1 & 0 & 1\\
%1 & 1 & 1 & 0\\
%\end{bmatrix}.
%\end{equation}
%%
%The solution $\hat{\vec{w}}_{i}$ to the problem (\ref{eq:argminReg}) is given by the eigenvector corresponding to the smallest eigenvalue of the matrix 
%$\tilde{\mat M}_i^{\top}\tilde{\mat M}_i+\gamma\hat{\mat I}$. 
%
%\begin{equation} \label{eq:argminReg}
%\hat{\vec{w}}_i = \arg \min_{\vec w_i}(\vec w^{\top}_i (\mat M_i^\top \mat M_i + \tilde{\gamma}   \hat{\mat I})\vec w_i)
%\end{equation}
%where $\hat{\mat{I}} \in \mathbb{R}^{(10+F) \times (10+F)}$ is:
%\begin{equation} \label{eq:regmx}
%\hat{\mat I}  = diag(\begin{array}{ccccccccc}
%0 & 1 & 1 & 0 & 0 & 1 & 0 & \dots & 0 
%\end{array})
%\end{equation}
%and $\tilde{\gamma}$ has been found iteratively using the method described in section \ref{sec:??}.
%In this case $\hat{\vec{w}}_i$ has been obtained applying the total least-square to the argument of \ref{eq:argminReg}, taking the eigenvector correspondent to the minimum eigenvalue of the matrix $(\mat M_i^\top \mat M_i + \tilde{\gamma}  \hat{\mat{I}})$. 
%The estimated dual quadric $\hat{\mat {Q}}_i$ can be obtained taking the first 10 rows of the vector $\hat{\vec{w}}_i$. Finally the primal quadric is derived from the dual one applying the same relation defined in Eq. (\ref{eq:invAdj}).

%A crucial issue with regularization is the tuning of the penalty parameter $\gamma$. Remarkably,  best performance are achieved with $\gamma = 10^{-7}$ after a normalisation of the matrix $\tilde{\mat M}_i^{\top}\tilde{\mat M}_i$ by its Frobenius norm. This parameter has been kept fixed for all the tested datasets, despite different projection matrices $\mat P_f$, ellipsoid types and number of frames. Moreover experimental results remains rather similar given a large range of values, i.e. for $\gamma \in [10^{-8}, 10^{-6}]$. 

\subsection{Ellipse fitting from image segmentation}
\label{subsec:fitt}
%
%The procedure to associate an ellipse to each bounding box, described in section \ref{sec:bbrec} has several drawbacks. 
%In general, the bounding boxes from generic object detectors are not precisely aligned with the true object center and often they include a relevant portion of background. 
In general, the BBs from generic object detectors are not precisely aligned with the true object center and often they include a relevant portion of background.
More importantly, BB axes are aligned by construction to the image axes. Thus, the related ellipses are aligned to these axes as well: this results in a relevant rotation mismatch between the real ellipse enclosing the object in the image and the one given by the detector. % ellipses from bounding boxes and ground truth ellipses whenever the latter have their axes strongly rotated in respect the image axes. 
%
This situation is even more complex when the Ground Truth (GT) ellipses have both strong axes rotation and eccentricity: the GT ellipses are badly approximated by the ellipse obtained from the BB, both in terms of rotation, shape and area (as in Fig. \ref{fig:seg}(a)).

\begin{figure}[h]
	\centering 
	\includegraphics[width=3.9cm]{./figures/segmentation/imm1.png}\hspace{.2cm}
    \includegraphics[width=3.9cm]{./figures/segmentation/imm2.png}\\
  	 (a)  \hspace{3.6cm} (b) 
	\caption{(a) Example of mismatch between GT ellipse (blue) and ellipse from BB (red); (b) segmented object using SCA algorithm with ellipse fit to the segmentation (red) and GT ellipse (blue). }
	\label{fig:seg}
\end{figure}

 To cope with these limitations, we design an approach to obtain a tighter and more reliable ellipse fitting on the object inside the BB. % a further approach could exploit the information contained in the image patch enclosed by the bounding box. 
%
%In detail, the pixel color and intensity distribution of the object in a set of frames are often sufficiently different from the background to allow an accurate object segmentation. 
In particular, we adopt a variation of the method proposed in \cite{jojic2009stel}, based on spatial correlations among different image patches, to produce a binary segmentation mask. Here we exploit the fact that the pixel color and intensity distribution of the object in a set of frames are often sufficiently different from the background to allow an accurate object segmentation. Moreover as images come from a video sequence, the pose of the objects smoothly changes from one frame to another and these are the optimal conditions for stel component analysis - SCA algorithm \cite{jojic2009stel}.
SCA is a co-segmentation algorithm that segments images in $S$ segments by learning $K$  components of the model. Such components represent peculiar poses of an object and they are blended together to create a segmentation prior that well adapts to the different poses of the object in the image. The final segmentation is obtained by applying this flexible prior to a GMM-based segmentation.

%Given the binary mask localising the object in each bounding box, we extract the object alignment, orientation and size from the mean $\vec{\mu}_{if}$ and covariance matrix $\mat{Cov}_{if}$ of the 2D spatial distribution of the pixels belonging to the object $i$ in frame $f$.
Given the binary mask localising the object in each BB, we extract the object orientation and size from the covariance matrix $\mat{Cov}_{if}$ of the 2D spatial distribution of the pixels belonging to the object $i$ in frame $f$.
 %set of points segmented as belonging to the object $i$ in frame $f$. %of the set of pixel included in the object segmentation. From such information an ellipse can be derived that better fit the true ellipsoid projection. %In particular given indicating with $$ and $$ the mean and covariance matrix of segmented pixels in the bounding box, the related ellipse
%Instead of using ellipses aligned with respect to the image axes, we can add further information about the orientation including, as new parameter, the angle $\alpha$, which is computed from the object as described in the section \ref{sec:??}. The oriented ellipse $\mat{C}_{if}$ should be obtained from the inverse of the aligned one as follow:
%\begin{equation}
%\mat{C}_{if} = (\tilde{\mat{R}}(\alpha) \hat{\mat{C}}_{if}^{\minus 1} \tilde{\mat{R}}(\alpha)^\top)^{\minus 1}
%\end{equation}
%The idea is to calculate the mean $\vec{\mu}_{if}$ and covariance matrix $\mat{\Cov}_{if}$ of the 2D spatial distribution of the set of points inside the bounding box segmented as belonging to the object. 
%In more details, we can approximate the 2D object image with an ellipse centered on $\vec{\mu}_{if}$, with axes aligned along the two eigenvectors of $\mat{Cov}_{if}$ and with their magnitude proportional to the square root of its eigenvalues, denoted as $\lambda^j_{if}$ with $j = 1,2$. 
%
%This information can be used 
Now we can fit a 2D ellipse with axes aligned along the two eigenvectors of $\mat{Cov}_{if}$ and with their magnitude proportional to the square root of its eigenvalues (i.e. $\lambda^j_{if}$ with $j = 1,2$). The magnitude of the ellipse's axes is set making the second central moments of the region enclosed by the ellipse equal to the second 
central moments of the segmented region. An example of this segmentation process is displayed in Fig. \ref{fig:seg}(b) where the matching between GT and fitted ellipses is by far improved with respect to the ellipse from BB in Fig. \ref{fig:seg}(a).


%Now we can fit a 2D ellipse %, we can approximate the 2D object image with an ellipse 
%centered on $\vec{\mu}_{if}$, with axes aligned along the two eigenvectors of $\mat{Cov}_{if}$ and with their magnitude %proportional to the square root of its eigenvalues (i.e.$\lambda^j_{if}$ with $j = 1,2$). This proportional value %$a^j_{if}$ for $j =1,2$ has to assure the ellipses to be tightly inscribed in the bounding box. This happens if the axes %are aligned to the bounding box sides and equal to $w_{if}$ and $h_{if}$. Now, the covariance matrix for a rectangular 2D %uniform distribution is diagonal, with its eigenvalues equal to $(w_{if})^2/12$ and  $(h_{if})^2/12$. Hence, this gives %the proportional factor to be fixed as:
%\begin{equation}
%a^j_{if}=\sqrt{12\lambda^j_{if}}.
%\end{equation}


%FIXME : VERSIONE ALTERNATIVA  Given the binary mask localising the object in each bounding box, we extract the object alignment from the covariance matrix $\mat{Cov}_{if}$ of the 2D spatial distribution of the pixels belonging to the object $i$ in frame $f$.
%Now we can fit a 2D ellipsoid with axes aligned along the two eigenvectors of $\mat{Cov}_{if}$. 

%


 
%To fix a proportionality constant between axes and square root of the eigenvalues we carried out the following reasoning. In an ideal case, when all the points inside the bounding box are segmented as belonging to the object, the ellipse should be tightly inscribed in the bounding box. 

 %Hence, the axes length $a^j_{if}, \hspace{0.2 cm} j =1,2$ is fixed as:

%FIXME : HO SCRITTO ANCHE LA VERSIONE CON ELLISSE CIRCOSCRITTA ALLA BB SE SERVISSE, NEL TESTO COMMENTATO QUI SOTTO
% ADB: OK
%In an ideal case when all the points inside the bounding box are segmented as belonging to the object, the ellipse should be circumscribed to the bounding box. To do this the axes should be equal to $\sqrt{2}w_{if}$ and $\sqrt{2}h_{if}$. Now the covariance matrix for a rectangular 2D uniform distribution is diagonal with the eigenvalues equal to $(w)^2/12$,  $h^2/12$. Hence, the axes length $a^j_{ij}, \hspace{0.2 cm} j =1,2$ is fixed as:
%\begin{equation}
%a^j_{ij}=\sqrt{24\lambda^j_{if}}
%\end{equation}
%
%where $\tilde{\mat{R}}(\alpha) \in \mathbb{R}^{3 \times 3}$ is the homogeneous rotation matrix.
%
\section{Experiments} \label{sec:exp}
%%FIXME : RESULTS ON SVD ARE A BIT IRREGULAR -> WE ARE MAKING MORE SIMULATIONS IN ORDER TO REDUCE STATISTICAL OSCILLATIONS
%%The proposed method has been tested on a synthetic scenario and on three real datasets. % how the method has been applied in a synthetic scenario firstly and then on real %datasets. In both cases we used an as metric to assess the goodness of the method the AP criteria FIXME DA CAMBIARE IN CASO DI VARIAZIONE DI METRICA, comparing bounding boxes %recovered by 2D reprojections and the ground-truth bounding boxes in each dataset.
%%In all the tests, accuracy of the result was evaluated by three different metrics, all based on intersection over union of ground truth and estimated regions, the latter being %defined by ellipsoids, ellipses or bounding boxes respectively. 
%%The general formula for all the metrics is given by:
%%\begin{equation}
%%\label{metric}
%%Acc_{l} = \frac{s_{l}^{GT} \cap s_l^{Est}}{s_l^{GT} \cup s_l^{Est}}
%%\end{equation}
%%where $s_l^{GT}$ and  $s_l^{Est}$ are respectively the ground truth and estimated set of points defining an object position and occupancy in a given representation, indicated %by the variable $l$. In particular, $s_l^{GT}$ and $s_l^{Est}$ denote: 1) for $l=3D$, the sets of 3D points enclosed by an ellipsoid; 2) for $l=2D$, the sets 2D points enclosed %in ellipses given by the projection of the corresponding ellipsoid in a given frame; for $l=BB$, the sets of 2D points enclosed in the bounding box obtained from the %corresponding ellipse. In the third case the bounding box was the one circumscribed to the ellipse, enlarged of a factor of FIXME COSIMO $\%$ in order to comply with the %typical padding yielded by generic object detectors. It is to note that the bounding boxes provided by an objector detector, i.e. the input to the proposed algorithm, are in %general different from both $s_{BB}^{GT}$ and $s_{BB}^Est$ since they are not derived from the projection of an ellipsoid, be it ground truth or estimated. For a given frame %sequence, $Acc_{l}$ is evaluated in terms of mean value and standard deviation over the number of objects in the sequence, for $Acc_{3D}$, and over the number of objects and %number of frames, for $Acc_{2D}$ and $Acc_{BB}$.
%%Because it is not guaranteed that the estimated quadrics correspond in all cases to feasible ellipsoids, especially for high geometric errors, we restrict the computation of %\ref{metric} to the feasible ellipsoids and we evaluate the percentage of quadrics not corresponding to ellipsoids as a further metric. 
%%
%%FIXME: COMMENTI SUL PERCHE' SI SONO SCELTI QUESTI TIPI DI METRICHE ? PARALLELI CON LA LETTERATURA ?  $Acc_{2D}$ more closely related to the algebric error minimized by the %cost function since it minimizes the difference between the ellipsoid projections and the ellipses form boundin boxes. $Acc_{3D}$ is more intuitive as a metric but it can %depart more from the algebric error whenever the views are restricted to a small set of angles (near degenerate configurations).  
%% 
%The proposed method has been tested on a synthetic scenario and on three real datasets. % how the method has been applied in a synthetic scenario firstly and then on real %datasets. In both cases we used an as metric to assess the goodness of the method the AP criteria FIXME DA CAMBIARE IN CASO DI VARIAZIONE DI METRICA, comparing bounding boxes %recovered by 2D reprojections and the ground-truth bounding boxes in each dataset.
%In all the tests, accuracy of the results was evaluated by two metrics, based either on the volume overlap between ground truth and estimated ellipsoids ($O_{3D}$), or on the %image area overlap ($O_{2D}$) between their projections in each frame. 
%%In the first one, ground truth $\mat Q_i$ and estimated $\tilde{\mat Q}_i$ ellipsoids in 3D were taken into account and the metric was defined as the intersection over union %of the point sets enclosed by the ellipsoid surfaces, averaged over all the objects in a dataset. 
%In detail $O_{2D}$ and $O_{3D}$ were defined as:
%%
%%\begin{equation}
%%\label{metric2D}
%%O_{3D} = \sum_{i=1}^N \frac{\mathcal{Q}_i \cap \tilde{\mathcal{Q}}_i}{N\left(\mathcal{Q}_i \cup \tilde{\mathcal{Q}}_i\right)},  \hspace{0.5 %cm}  O_{2D} = \sum_{i=1}^N %\sum_{f=1}^F\frac{\mathcal{C}_{if} \cap \tilde{\mathcal{C}}_{if}}{NF\left(\mathcal{C}_{if} \cup %\tilde{\mathcal{C}}_{if}\right)},
%%\end{equation}
%\begin{equation}
%\label{metric3D}
%O_{3D} = \frac{1}{N}\sum_{i=1}^N \frac{\mathcal{Q}_i \cap \tilde{\mathcal{Q}}_i}{\mathcal{Q}_i \cup \tilde{\mathcal{Q}}_i},  %\hspace{0.2cm}  O_{2D} = \frac{1}{NF}\sum_{i=1}^N %\sum_{f=1}^F\frac{\mathcal{C}_{if} \cap \tilde{\mathcal{C}}_{if}}{\mathcal{C}_{if} \cup \tilde{\mathcal{C}}_{if}},
%\end{equation}
%%
%%\begin{equation}
%%\label{metric2D}
%%O_{2D} = \frac{1}{NF}\sum_{i=1}^N \sum_{f=1}^F\frac{\mathcal{C}_{if} \cap \tilde{\mathcal{C}}_{if}}{\mathcal{C}_{if} \cup \tilde{\mathcal{C}}_{if}},
%%\end{equation}
%%
%\begin{equation}
%\label{metric2D}
%O_{2D} = \frac{1}{J}\sum_{f=1}^F\sum_{i=1}^N I(i,f)\frac{\mathcal{C}_{if} \cap \tilde{\mathcal{C}}_{if}}{\mathcal{C}_{if} \cup \tilde{\mathcal{C}}_{if}}
%\end{equation}
%%
%with
%%
%\begin{equation}
%J=\sum_{f=1}^F\sum_{i=1}^N I(i,f)
%\end{equation}
%where $\mathcal{Q}_i$ and $\tilde{\mathcal{Q}}_i$ denote the volume %the sets of 3D points enclosed by the surfaces 
%of the ellipsoids given by the matrices $\mat Q_i$ and $\tilde{\mat Q}_i$ respectively, while $\mathcal{C}_{if}$ and $\tilde{\mathcal{C}}_{if}$ denote the area of the ellipses %given by the matrices $\mat C_{if}$ and $\tilde{\mat C}_{if}$, corresponding to the projections on the frame $f$ of the ellipsoids $\mat Q_i$ and $\tilde {\mat Q}_i$ %respectively. $I(i,f)$ is an indicator function equal to $1$ if the object $i$ is visible in frame $f$ and $0$ otherwise.
%%
%%The second metric was defined in an analogous way, but considering the ellipses given by the projections of ground truth and estimated ellipsoids in each frame, and averaging %over all the frames and all the objects in a dataset. In detail:
%%\begin{equation}
%%\label{metric3D}
%%Acc_{2D} = \sum_{i=1}^N \sum_{f=1}^F\frac{\mathcal{C}_{if} \cap \tilde{\mathcal{C}}_{if}}{NF\left(\mathcal{C}_{if} \cup %\tilde{\mathcal{C}}_{if}\right)},
%%\end{equation}
%%where $\mathcal{C}_{if}$ and $\tilde{\mathcal{C}}_{if}$ denote the sets of 2D points enclosed by the curves of the ellipses represented by the matrices $\mat C_{if}$ and %$\tilde{\mat C}_{if}$, corresponding to the projections on the frame $f$ of the ellipsoids $\mat Q_i$ and $\tilde {\mat Q}_i$ respectively.
%%
%%where $s_l^{GT}$ and  $s_l^{Est}$ are respectively the ground truth and estimated set of points defining an object position and occupancy in a given representation, indicated %by the variable $l$. In particular, $s_l^{GT}$ and $s_l^{Est}$ denote the sets of 3D points enclosed by an ellipsoid, for $l=3D$ and the sets of 2D points enclosed in ellipses %given by the projection of the corresponding ellipsoid in a given frame for $l=2D$. For a given frame sequence, $Acc_{l}$ is averaged over the number of objects in the %sequence, for $Acc_{3D}$, and over the number of objects and number of frames, for $Acc_{2D}$.
%When the $i$-th estimated quadric is not an ellipsoid both $\tilde{\mathcal{Q}}_i$ and $\tilde{\mathcal{C}}_{if}$ for $f = 1,\cdots,F$ are set to zero. 
%%When simulating experiments with remarkable detection inaccuracies, it is not guaranteed that the estimated quadrics correspond to feasible ellipsoids. Hence, we restrict the %computation of Eq. (\ref{metric2D}) and Eq. (\ref{metric3D}) to the feasible ellipsoids but we also show the number of quadrics not corresponding to ellipsoids as a further %information. 
%
%The metric given by $O_{3D}$ measures in a more direct way the success of the algorithm in recovering the 3D position and occupancy of an object. However, when camera views are %restricted to a small range of angles or they are related to quasi-planar trajectories, the metric could give poor results even with a small algebraic error in Eq. %(\ref{eq:argmin}). This might happen because of the ill-conditioning of the problem, yielding near degenerate solutions corresponding to ellipsoids with strong eccentricity. On %the contrary, the metric given by $O_{2D}$ is related to reprojection of the estimated ellipsoid on the set of available views, and thus is more closely related to the %algebraic error minimized by the cost function. 
%
%In the following we will denote the non regularized solution obtained with SVD (section \ref{sec:dualsf}) as PfD, the regularized solution  (Section \ref{subsec:reg}) as %PfD+REG, the solution obtained from ellipses fitted to segmented images (Section \ref{subsec:fitt}) as PfD+ES (Ellipse Segmentation) and finally the solution implying both %regularization and image segmentation as PfD+ES+REG.
%   
   
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% VERSION   WITH JUST O3 METRIC%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\vspace{-1ex}
%\textbf{Please check the supplementary material available in CMT for
 % further experimental evaluation and visualisation of the results.} 
The proposed method has been tested on a synthetic scenario and on three real datasets. 
In all the tests, the accuracy of the estimated pose was measured by the volume overlap between GT and estimated ellipsoids ($O_{3D}$) defined as:
\begin{equation}
\label{metric3D}
O_{3D} = \frac{1}{N}\sum_{i=1}^N \frac{\mathcal{Q}_i \cap \tilde{\mathcal{Q}}_i}{\mathcal{Q}_i \cup \tilde{\mathcal{Q}}_i},  %\hspace{0.2cm}  O_{2D} = \frac{1}{NF}\sum_{i=1}^N \sum_{f=1}^F\frac{\mathcal{C}_{if} \cap \tilde{\mathcal{C}}_{if}}{\mathcal{C}_{if} \cup \tilde{\mathcal{C}}_{if}},
\end{equation}
where $\mathcal{Q}_i$ and $\tilde{\mathcal{Q}}_i$ denote the volume %the sets of 3D points enclosed by the surfaces 
of the ellipsoids given by the matrices $\mat Q_i$ and $\tilde{\mat Q}_i$ respectively. 
When the $i$-th estimated quadric is not an ellipsoid $\tilde{\mathcal{Q}}_i$ is set to zero. 
Notice that, when camera views are restricted to a small range of angles or they are related to quasi-planar trajectories, $O_{3D}$ could give poor results even with a small algebraic error in Eq. (\ref{eq:argmin}). However, we chose such metric since it measures in a direct way the success of the algorithm in recovering the 3D position and occupancy of an object.
Moreover, we also evaluated the orientation error by using the measure $ \theta_{err}$, which is the angle in radians between the main axes of estimated and GT ellipsoids. % of ... %(FIXME COSIMO: put the measure you used).}
% 
In the following we will denote the non regularized solution obtained with SVD (Sec. \ref{sec:dualsf}) as PfD, the regularized solution  (Sec. \ref{subsec:reg}) as PfD+REG, the solution obtained from ellipses fitted to segmented images (Sec. \ref{subsec:fitt}) as PfD+ES and finally the solution implying both regularization and image segmentation as PfD+ES+REG. Regarding orientation estimation, we have reported the SVD solution only since the regularization provides improvement mainly on the 3D localization rather than the object orientation.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%   
   
   
   
   
\subsection{Synthetic setup}
%%The first step has been to model a 3D synthetic enviro
%%nment. 
%%The structure matrix of a set of points is composed by $i=1 \dots N$ objects containing $p = 1 \dots P_N$ 3D points; in the 3D synthetic scene all the objects are still and %rigid with respect to the world frame. Inside each object, points are uniformly distributed around a randomly chosen centre $\vec c_N \in \mathbb{R}^{4}$: 
%%\begin{equation}
%%\mat S_{i} = \mathcal{N} (\vec s_p \: | \: \vec c_{N}, \sigma^2_S \mat I)
%%\end{equation}
%%where $\mat S_{i} \in \mathbb{R}^{4 \times P_N}$ is a homogeneous structure matrix with a variance of $\sigma^2_N$. We associate to each frame a projection matrix $P_f \in %\mathbb{R}^{3 \times 4}$, starting from a certain location and then generating random linear trajectories. From this 3D features we obtain the projections using the relation %$\mat W_{if} = \mat P_f \mat S_i$, where $\mat W_{if} \in \mathbb{R}^{3 \times P_N}$ is the homogeneous matrix of all the 2D measurements.\\
%%Starting from this cloud of points, we used as axis length on each direction the covariance of each set $cov(S_{i})$ associated to each object $i$. From this we computed the %final ellipsoid $Q_(\mat S_i)$
%%To emulate the object detection, we enclose each object in a set of bounding boxes $B_{if}$, taking each extreme reprojection of the measurement $ \mat W_{if}$ in the two %coordinates and then adding a padding of a fixed size.\\ 
%%To this synthetic data we applied the method as illustrat
%%ed in the section \ref{sec:bbrec}.
%%
%% 
%The synthetic environment contains $50$ objects and each one is modeled by an ellipsoid. The ellipsoid centers lay inside a cube of side $20$ units, %centered at world %coordinates $(0,0,0)$, 
%with uniform probability distribution. The length of the largest axis was set according to a uniform distribution ranging from $3$ to $12$ units. The lengths of the other two %axes were set proportional to the largest axis with a proportionality factor  ranging from $0.3$ to $1$ units (according again to a uniform distribution). Finally, axes %orientation was randomly generated. %the orientation of the largest and second largest axes were chosen according to uniform distribution across the solid angle and plane angle %respectively.
%
%A set of $20$ views were generated with a camera distance from the cube center of $200$ units and a simulated trajectory was computed so that azimuth and elevation angles span %the range $[0^{\circ},60^{\circ}]$ and $[0^{\circ},70^{\circ}]$ respectively. Notice that the variability of object appearance due to such range of angle views can be handled %by state-of-art object detectors \cite{divvala2012important}. %if the range of angles does not exceed the above one . In figure Fig. \ref{} (FIXME: DA AGGIUNGERE) an example of %the synthetic setup is depicted %(for ease of visualization just $5$ ellipsoids have been displayed). 
%Given the projection matrix $\mat P_f$ of each camera frame, the ellipses given by the exact projections of the ellipsoids were calculated, thus generating our ground truth %data.
%
%Synthetic tests were mainly aimed at validating the robustness of the proposed method against common inaccuracies affecting object detectors such as coarse estimation of the %object center, tightness of the bounding box in respect to the object size and variations over the object pose.
%%
%%from object detectors and their fitted ellipses. 
%Thus, each ellipse %from the ground truth ellipsoids 
%was corrupted by three kinds of geometric errors, namely translation error (TE), rotation error (RE) and size error (SE), and fed to the proposed algorithm. 
%%Three types of errors %given a simulated detector output 
%%were taken into account: 
%To impose such errors, the ellipses center coordinates $c_1$, $c_2$, the axes length $l_1$, $l_2$ and the orientation $\alpha$ of the first axis %in respect to the image $x$ %axis, 
%were perturbed %adding to them a random variable 
%in the following way\footnote{We omit for simplicity the object and frame indexes.}: 
%\begin{equation}
%\label{trasl}
%\hat{c}_j = c_j +\bar{l}{\nu}_j^{\vec c}, %\hspace{0.6cm} \hat{\alpha} =  \alpha+\nu^{\alpha}, \hspace{0.6cm} \hat{l}_j = l_j\left( 1+\nu^{l}\right),
%\end{equation}
%\begin{equation}
%\label{rot}
%\hat{\alpha} =  \alpha+\nu^{\alpha},
%\end{equation}
%\begin{equation}
%\label{size}
%\hat{l}_j = l_j\left( 1+\nu^{l}\right),
%\end{equation}
%where $\nu_j^{\vec{c}}$, $\nu^{\alpha}$ and $\nu^{l}$ are random variables with uniform probability density functions (PDF) and mean value equal to zero, and %$\bar{l}=(l_1+l_2)/2$. %and tunable symmetric boundary values and. TE (\ref{trasl}) has been set proportional to the mean of axes length, allowing higher errors on bigger %ellipses, RE (\ref{rot}) is an absolute error in degrees and SE (\ref{size}) is expressed as a percentage of the axes length. 
%In order to highlight the specific impact of each error, %on the reconstruction performance, the above errors 
%they were applied separately. For each kind of error, a set of magnitudes was tested, setting accordingly the boundary values of the uniform PDFs for $\nu_j^{\vec c}$, %$\nu^{\alpha}$ and $\nu^l$. 
%
%
%\begin{figure*}[thbp!]  
%\centering
%%\includegraphics[width=5cm]{figures/crocco_rot.png}   
%%\includegraphics[width=5cm]{figures/crocco_size.png} 
%%\includegraphics[width=5cm]{figures/crocco_trasl.png} \\
%%(a) \hspace{4.5cm} (b) \hspace{4.6cm} (c) \\
%\includegraphics[width=5cm]{./figures/synth/angErr2Dcorr}   
%\includegraphics[width=5cm]{./figures/synth/axErr2Dcorr} 
%\includegraphics[width=5cm]{./figures/synth/cntErr2Dcorr} \\
%(a) \hspace{4.6cm} (b) \hspace{4.6cm} (c) \\
%\includegraphics[width=5cm]{./figures/synth/angErr3Dcorr}   
%\includegraphics[width=5cm]{./figures/synth/axErr3Dcorr} 
%\includegraphics[width=5cm]{./figures/synth/cntErr3Dcorr} \\
%(d) \hspace{4.6cm} (e) \hspace{4.6cm} (f) \\
%%\end{tabular}
%   \caption{Reconstruction performance for the synthetic tests versus different types of errors. Average accuracy given by PfD or PfD+REG for rotation error (a,d), size error %(b,e), translation error (c,f), measured by metrics $O_{2D}$ (a,b,c) and $O_{3D}$ (d,e,f) }
%\label{fig:synth}
%\end{figure*}
%%
%In Fig. \ref{fig:synth} the average accuracy of the proposed algorithm, for both PfD and PfD+REG, is displayed versus the error magnitude (i.e. the boundary value of the %uniform distribution), for RE (Fig. \ref{fig:synth}(a), (d)), SE (Fig. \ref{fig:synth}(b), (e)) and TE (Fig. \ref{fig:synth}(c), (f)), according to the two metrics $O_{2D}$ %(Fig. \ref{fig:synth}(a), (b), (c)) and $O_{3D}$ (Fig. \ref{fig:synth}(d), (e), (f)). Concerning PfD+REG, the  weighting parameter $\lambda$ has been kept equal for all the %tests. % to the value of $5\cdot10^8$.   %Moreover the percentage of failures, i.e. estimated quadrics that are not ellipsoids, is displayed in Fig. \ref{fig:synth}(d) for RE, %in Fig. \ref{fig:synth}(e) for SE in Fig. \ref{fig:synth}(f) for TE. %Moreover the effect of the regularization procedure ( subsection \ref{subsec:fittreg}) is also checked, %displaying the corresponding average accuracy. 
%%(FIXME PER ORA LE ACCURACIES SONO ESPRESSE SOLO IN TERMINI DI INTERSEZIONE ED UNIONE DEGLI ELLISSI; SONO DA AGGIUNGERE LE VALUTAZIONI IN TERMINI DI OVERLAP FRA GLI ELLISSOIDI %IN 3D.) 
%%
%The results for PfD are perfect in absence of errors but undergo a significant drop as the error increases. This confirms an expected sensitivity of the closed form solution to %mismatches between fitted and ground truth ellipses. Nevertheless, when errors are moderate, the accuracy achieved is still remarkable. %For example RE of $10$ degrees, SE of %$17\%$ and TE of $7\%$ yield respectively  $O_{2D}=0.42$, $O_{2D}=0.35$ and $O_{2D}=0.20$. FIXME TROPPO NEGATIVO COME COMMENTO FIN QUI?. 
%
%On the contrary, PfD+REG exhibits a very smooth decrease in accuracy versus the errors magnitudes, and a definitely higher performance in comparison to PfD for almost all the %error magnitudes, except for the case of zero error (RE, Se and TE) and the $5^{\circ}$ RE. In detail, the accuracy for PfD+REG, measured in terms of $O_{2D}$, drops from %$0.67$ (no errors) to $0.56$, $0.44$ and $0.21$ for maximum RE, SE and TE respectively, while the accuracy of PfD drops from $1$ to $0.21$ (RE), $0.09$ (SE) and $0.04$ (TE) %respectively.  
%%In general, the accuracy smoothly decreases as the error increase thus denoting a stable behavior of our implementation. 
%As expected, the performance measured with $O_{3D}$ is worse than with $O_{2D}$; nevertheless their general trend is very similar.%and the difference tend to increase when %errors become larger. 
%
%Notice that both the methods are generally more robust toward RE and SE (Figs. \ref{fig:synth}(a), (b), (d) and (e) ), than toward TE. %In detail, accuracy decreases to $O_{2D} %= 0.86$ ( $O_{3D}= 0.66$ ) for a RE of $45^{\circ}$ but the number of failures remains still negligible. 
%The higher robustness toward RE and SE is quite important since such kind of errors are likely to happen very frequently whenever ellipses are fitted to bounding boxes. Even if %the detector is accurate, the bounding box quantizes the object alignment at steps of $90^{\circ}$, yielding a maximum RE of $45^{\circ}$. This tends to overestimate the object %area, thus affecting SE, whenever the object is not aligned to the bounding box axes (see Fig. \ref{fig:seg}(a)). %Also the robustness to SE is remarkable: even allowing to %halve or double the ellipses size, the average accuracy does not decrease below $O_{2D}=0.90$ ($ O_{2D} = 0.69$), with a percentage of failures of about $8\%$. 
%
%%Differently, the TE seems to be the most problematic both in terms of accuracy measured in 3D ( $O_{3D}=0.48$ ) and percentage of failures (about $30\%$) even if the 2D %reprojection still provide a reasonable overlap of the ellipses ($O_{2D}=0.85$,). 
%Concerning TE, we noticed that the robustness to such kind of errors decreases when ellipses are far from the image center and small in comparison to the image size. This is %mainly due to the structure of the $3 \times 3$ matrix of the conic, whose entries become strongly unbalanced whenever the translation terms in the third row and column %increase in respect to the $2 \times 2$ principal sub-matrix, leading likely to numerical problems. However, the maximum degree of failure for the regularized solution is %reasonable considering that an object detector placing the bounding box with a TE of $0.3$ is considered to fail the detection\footnote{The TE is applied independently to both %the horizontal and vertical components of the ellipse center, i.e. see Eq. (\ref{trasl}), resulting in a maximum overall translation of $0.3\sqrt{2}$.}. %the TE is applied %independently to both the horizontal and vertical components of the ellipse center (see \ref{trasl}) resulting in a maximum overall translation of $0.3\sqrt{2}$. An object %detector placing the bounding box with such TE is considered to fail the detection: 
%%evertheless in about $70\%$ of cases an ellipsoid is reconstructed with an accuracy above $0.5$ as can be seen in Figs. \ref{fig:synth}(c) and \ref{fig:synth}(f).
%%Finally, for the synthetic test the regularization approach described in subsection \ref{subsec:fittreg} did not yield significant improvements and is therefore not shown in %the figures. 
%
%%\onecolumn 
% 
%%\subsection{Real scenario}
%%The proposed method was tested on three different datasets, namely TUW dataset , ACCV dataset %Hinterstoisser
%% and KITTI dataset \cite{Geiger2013}. %In the first one: In the real scenario, we got all the bounding boxes by using a generic object %detector   FIXME : COME ABBIAMO %DEFINITO LE BOUNDING BOXES ?
%%
%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%SYNTHETIC SETUP NEW VERSION %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
The synthetic environment contains $50$ ellipsoids randomly placed inside a cube of side $20$ units.
The length of the largest axis $L$ ranges from $3$ to $12$ units, according to a uniform PDF. The lengths of the other two axes are equal to $\gamma L$ with $\gamma \in [0.3,1]$. Finally, axes orientation was randomly generated. 
%
A set of $20$ views were generated with a camera distance from the cube center of $200$ units and a trajectory was computed so that azimuth and elevation angles span the range $[0^{\circ},60^{\circ}]$ and $[0^{\circ},70^{\circ}]$ respectively\footnote{Notice that the variation of object appearance due to such range of angle views can be handled by state-of-art object detectors \cite{divvala2012important}}. 
Given the projection matrix $\mat P_f$ of each camera frame, GT ellipses given by the exact projections of the ellipsoids were calculated.

Synthetic tests were aimed at validating the robustness of the proposed method against common inaccuracies affecting object detectors, such as coarse estimation of the object center, tightness of the BB with respect to the object size and variations over the object pose.
Thus, each ellipse 
was corrupted by three errors, namely translation error (TE), rotation error (RE) and size error (SE), and fed to the proposed algorithm. 
To impose such errors, the ellipses centers coordinates $c_1$, $c_2$, the axes length $l_1$, $l_2$ and the orientation $\alpha$ of the first axis 
were perturbed 
as follows \footnote{We omit for simplicity the object and frame indexes.}: 
\begin{equation}
\label{trasl}
\hat{c}_j = c_j +\bar{l}{\nu}_j^{\vec c}, \hspace{0.6cm} \hat{\alpha} =  \alpha+\nu^{\alpha}, \hspace{0.6cm} \hat{l}_j = l_j\left( 1+\nu^{l}\right),
\end{equation}
%\begin{equation}
%\label{rot}
%\hat{\alpha} =  \alpha+\nu^{\alpha},
%\end{equation}
%\begin{equation}
%\label{size}
%\hat{l}_j = l_j\left( 1+\nu^{l}\right),
%\end{equation}
where $\nu_j^{\vec{c}}$, $\nu^{\alpha}$ and $\nu^{l}$ are random variables with uniform PDF and mean value equal to zero, and $\bar{l}=(l_1+l_2)/2$.  
In order to highlight the specific impact of each error,
they were applied separately. Error magnitudes were set tuning the boundary values of the uniform PDFs of $\nu_j^{\vec c}$, $\nu^{\alpha}$ and $\nu^l$.
%For each kind of error, a set of magnitudes was tested, setting accordingly the boundary values of the uniform PDFs for $\nu_j^{\vec c}$, $\nu^{\alpha}$ and $\nu^l$. 
%
%
\begin{figure*}[thbp!]  
\centering
%\includegraphics[width=5cm]{figures/crocco_rot.png}   
%\includegraphics[width=5cm]{figures/crocco_size.png} 
%\includegraphics[width=5cm]{figures/crocco_trasl.png} \\
%(a) \hspace{4.5cm} (b) \hspace{4.6cm} (c) \\
%\includegraphics[width=5cm]{./figures/synth/angErr2Dcorr}   
%\includegraphics[width=5cm]{./figures/synth/axErr2Dcorr} 
%\includegraphics[width=5cm]{./figures/synth/cntErr2Dcorr} \\
%(a) \hspace{4.6cm} (b) \hspace{4.6cm} (c) \\
\includegraphics[width=5cm]{./figures/synth/angErr3Dcorr}   
\includegraphics[width=5cm]{./figures/synth/axErr3Dcorr} 
\includegraphics[width=5cm]{./figures/synth/cntErr3Dcorr} \\
(a) \hspace{4.6cm} (b) \hspace{4.6cm} (c) \\
%(d) \hspace{4.6cm} (e) \hspace{4.6cm} (f) \\
%\end{tabular}
  % \caption{Reconstruction performance for the synthetic tests versus different types of errors. Average accuracy given by PfD or PfD+REG for rotation error (a,d), size error (b,e), translation error (c,f), measured by metrics $O_{2D}$ (a,b,c) and $O_{3D}$ (d,e,f) }
    \caption{Results for the synthetic tests versus different types of errors. Average accuracy given by PfD or PfD+REG for rotation error (a), size error (b), translation error (c), measured by $O_{3D}$ metric.} 
\label{fig:synth}
\end{figure*}
%
In Fig. \ref{fig:synth} the average accuracy $O_{3D}$ of the proposed algorithm, for both PfD and PfD+REG, is displayed versus the error magnitude (i.e. the boundary value of the uniform PDF), for RE (Fig. \ref{fig:synth}(a)), SE (Fig. \ref{fig:synth}(b)) and TE (Fig. \ref{fig:synth}(c)). Concerning PfD+REG, the  weighting parameter $\lambda$ has been kept equal for all the tests. 
The results for PfD are perfect in absence of errors but undergo a significant drop as the error increases. This confirms an expected sensitivity of the closed form solution to mismatches between fitted and GT ellipses. Nevertheless, when errors are moderate, the accuracy achieved is remarkable. 

On the contrary, PfD+REG exhibits a very smooth decrease in accuracy versus the errors magnitudes, and a definitely higher performance in comparison to PfD for almost all the error magnitudes, except for the case of zero error (RE, Se and TE) and the $5^{\circ}$ RE. In detail, $O_{3D}$ for PfD+REG, drops from $0.52$ (no errors) to $0.39$, $0.33$ and $0.16$ for maximum RE, SE and TE respectively, while the accuracy of PfD drops from $1$ to $0.15$ (RE), $0.05$ (SE) and $0.02$ (TE) respectively.  

%As expected, the performance measured with $O_{3D}$ is worse than with $O_{2D}$; nevertheless their general trend is very similar.

Notice that both the methods are generally more robust toward RE and SE (Figs. \ref{fig:synth}(a), (b) ), than toward TE.
The higher robustness toward RE and SE is quite important since such kind of errors are likely to happen very frequently whenever ellipses are fitted to BBs. Even if the detector is accurate, the BB quantizes the object alignment at steps of $90^{\circ}$, yielding a maximum RE of $45^{\circ}$. This tends to overestimate the object area, thus affecting SE, whenever the object is not aligned to the BB axes (see Fig. \ref{fig:seg}(a)). 
Concerning TE, we noticed that the robustness to such kind of errors decreases when ellipses are far from the image center and small in comparison to the image size. This is mainly due to the structure of the $3 \times 3$ matrix of the conic, whose entries become strongly unbalanced whenever the translation terms in the third row and column increase in respect to the $2 \times 2$ principal sub-matrix, leading likely to numerical problems. However, the maximum degree of failure for the regularized solution is reasonable considering that an object detector placing the BB with a TE of $0.3$ is considered to fail the detection\footnote{The TE is applied independently to both the horizontal and vertical components of the ellipse center, i.e. see Eq. (\ref{trasl}), resulting in a maximum overall translation of $0.3\sqrt{2}$.}. 

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

%
%
\subsection{ACCV dataset evaluation}
\label{Hint}
%The extensive ACCV database is public (over 18000 real images with 15 different objects and ground truth pose)! Here you can find the databases for the ape, benchvise, bowl, can, cat, cup, driller, duck, glue, holepuncher, iron, lamp, phone, cam and eggbox. Each dataset contains the 3D model saved as a point cloud (format: #_of_voxels size_of_voxel_in_cm x1_in_cm y1_in_cm z1_in_cm normal_x1 normal_y1 normal_z1 color_x1_normalized_to_1 color_y1_normalized_to_1 color_z1_normalized_to_1 ...) and a file that contains called distance.txt with the maximum diameter of the object (in cm). For some datasets we also provide a nice mesh model in the ply format (in mm - with better normals). The original mesh is contained in OLDmesh.ply. For most datasets we registered this OLDmesh.ply to the point cloud with the transformation stored in transform.dat (first number is not important, then each first number of a line is obsolete - for the rest: the transformation matrix [R|T] is stored rowwise (in m)). The registered mesh is stored in mesh.ply. In the folder data you can find the color images, the aligned depth images and the ground truth rotation and translation (in cm). In order to read the depth images you can use this function. The internal camera matrix parameters for the kinect are: fx=572.41140, px=325.26110, fy=573.57043; py=242.04899; Color image and depth image are already aligned by the internal alignement procedure of Kinect.\\
%
The ACCV dataset \cite{hinterstoisser2012accv} contains $15$ sequences, each related to a single object laying on a table at different camera viewpoints (from $100$ to $1000$ per sequence). %For each sequence the number of views ranges from $100$ to $1000$ %and is typically limited to a range of azimuth and elevation angles $[0^{\circ},45^{\circ}]$ (FIXME VERIFICARE QUESTO DATO). 
We selected the subset of $8$ sequences for which the 3D point cloud of the object is provided, and limit the number of views to $100$ for each sequence.
%A 3D point cloud for each object is also provided.
For each object we evaluated the GT ellipsoid as the envelope of the 3D point cloud. Moreover, for each frame and each object, we generated a 2D BB by simulating the output of a multi-scale object detector providing BBs with variable aspect ratio, like the well known Deformable Part Model (DPM) \cite{Felzenszwalb2010}.

%we extracted the 2D bounding boxes in the following way: given the set of 2D projections of the 3D point cloud, we searched for the minimum size bounding box enclosing all the projected points and we increase such bounding box by a factor of $0.06$ in order to include a minimum of background. Such a procedure can be considered as a reasonable simulation of the output of a multi-scale object detector providing bounding boxes with variable aspect ratio, like the well known Deformable Part Model (DPM) \cite{Felzenszwalb2010}. %The resulting enriched dataset will be made available online at FIXME.\\
%In the following we denote the reconstruction with the non regularized approach as SVD and the regularized one as REG. Moreover, if ellipses are obtained directly from bounding boxes we denote it with EBB while, if ellipses are fitted from the object segmentation , we denote it with ES.

%A 3D point cloud was provided for each object. 
%As for TUW dataset, we got the ground truth ellipsoids for each object and the 2D bounding boxes. %as the output of an object detector. 
%We selected the subset of $9$ sequences for which all the camera parameters and 3D point cloud are provided, and limit the number of views to $100$ for each sequence. %corresponding to the following objects (FIXME ELENCO OGGETTI)  The chosen object are the ones for which all the camera parameters and 3D point cloud are provided. 
In Fig. \ref{fig*:real_exp}, first row, an example of the localisation performance for the PfD method is displayed for the "duck" sequence. The estimated ellipsoid almost perfectly fits the GT one in respect to location, size, eccentricity and alignment, as it can be seen in three frames from the sequence and in the overall 3D localisation.
In Table \ref{tab:accv} the accuracy for each sequence, for PfD, is reported in terms of $O_{3D}$ and $\theta_{err}$.   %according to the two metrics $O_{3D}$ and $O_{2D}$. 
In general, the accuracy is lower for objects with a strongly not convex shape, like the driller, and higher for ellipsoid-like objects like  the duck. %Robust ellipse fitting increases both $O_{2D}$ and $O_{3D}$ of about FIXME percentual points, thus providing a significant improvement over the non robust solution FIXME  DA VEDERE.

Due to the large number of views and to the centering of each object in each frame (see discussion on synthetic results), the accuracy  is on average quite good even without regularization and ellipses from BBs, with an $O_{3D}$ of $0.60$. %overlap in 3D of $0.60$. 
For this reason, either adding regularization or fitting ellipses to segmented images did not improve significantly the results on this dataset. % In this sequence we used a large number of views, then we didn't apply regularization and robust ellipse fitting because they don't increase 
%significantly the performances over the non robust solution.
\begin{figure*}[btph]
\centering
\includegraphics[width=4 cm]{./figures/ACCV/1_ACCV.png} \hspace{.2cm}
\includegraphics[width=4 cm]{./figures/ACCV/2_ACCV.png} \hspace{.2cm}
\includegraphics[width=4 cm]{./figures/ACCV/3_ACCV.png} \hspace{.2cm}
\includegraphics[width=4 cm]{./figures/ACCV/ACCV3D.png} \\ \vspace{.3cm}
\includegraphics[width=4 cm]{./figures/TUW/1_TUW.png} \hspace{.2cm}
\includegraphics[width=4 cm]{./figures/TUW/2_TUW.png} \hspace{.2cm}
\includegraphics[width=4 cm]{./figures/TUW/3_TUW.png} \hspace{.2cm}
\includegraphics[width=4 cm]{./figures/TUW/TUW3D_.png} \\
%\caption{}
%\end{subfigure}
\caption{Localisation results for the ACCV (first row), TUW (second row) datasets. The first three columns show a close up of the views with the output of a generic object detector (yellow BB) and projections of GT and reconstructed ellipsoids (blue and green ellipses respectively). the last column show the localisation of the object (red), GT ellipsoid (blue) and estimated ellipsoid (green).}
\label{fig*:real_exp}
\end{figure*}

\begin{figure}[btph]
\centering
\includegraphics[width=8cm]{./seq9/KITTI2A.png} \\ \vspace{.3cm}
\includegraphics[width=8cm]{./seq9/KITTI3A.png} \\ \vspace{.3cm}
\includegraphics[width=8cm]{./seq9/KITTI3D_side.png}
%\caption{}
%\end{subfigure}
\caption{Localisation results for KITTI (third row) datasets. The first two images show a close up of the views with the output of a generic object detector (yellow BB) and projections of GT and reconstructed ellipsoids (blue and green ellipses respectively). the last column show the localisation of the object (red), GT ellipsoid (blue) and estimated ellipsoid (green).}
\label{fig*:real_exp2}
\end{figure}

%\begin{table}[h]
%\caption{Average accuracies for the eight sequences from ACCV dataset.}
%\centering
%\footnotesize
%\begin{tabular}{|c|c|c|}
%\hline  
 %& $O_{2D}$ & $O_{3D}$   \\
 %\hline
 %Iron	& 0,80 &	0,71 \\
 %\hline
 %Duck	& 0,89 &	 0,83	 \\
 %\hline  
%Ape   &	0,46  &	0,47  \\
% \hline
%Can	  &  0,82	 & 0,74   \\
% \hline
%Driller	& 0,70 & 	0,54  \\
 %\hline
%Benchvise &	0,78  &0,67  \\
% \hline
%Glue &	0,54 & 0,33 \\
%\hline
%Cat &	0,70 & 	0,56 	 \\
 %\hline
%\bf{Average}  & &    \\
% \hline
%%\hline
%\end{tabular}
%\label{tab:accv}
%\end{table}

%\begin{table}[h]
%\caption{Average accuracies for the eight sequences from ACCV dataset for SVD-EBB method.}
%\centering
%\footnotesize
%\begin{tabular}{|c|c|c|}
%\hline  
% 			& $O_{2D}$ & $O_{3D}$  \\
% \hline
% Iron		& 0.80 	& 0.71 \\
% \hline
% Duck		& 0.89 	& 0.83 \\
% \hline  
%Ape   		& 0.46 	& 0.47 \\
% \hline
%Can	  		& 0.82 	& 0.74 \\
% \hline
%Driller		& 0.70 	& 0.54 \\
% \hline
%Benchvise 	& 0.78 	& 0.67 \\
% \hline
%Glue 		& 0.54 	& 0.33 \\
%\hline
%Cat 		& 0.70 	& 0.56 \\
% \hline
%\bf{Average} & \bf{0.71} & \bf{0.60} \\
% \hline
%%\hline
%\end{tabular}
%\label{tab:accv}
%\end{table}

%\begin{table}[h]
%\caption{Average accuracies for the eight sequences from ACCV dataset for PfD method.}
%\centering
%\footnotesize
%\resizebox{\columnwidth}{!}{
%\begin{tabular}{|c|c|c|c|c|c|c|c|c|c|}
%\hline 
%           &  Iron & Duck & Ape & Can & Driller & Vise & Glue & Cat & Avg \\
%   \hline
%  $O_{2D}$ &  0.80 & 0.89 & 0.46 & 0.82 & 0.70 & 0.78 & 0.54 & 0.70 & \bf{0.71}\\
%   \hline  
%  $O_{3D}$ &  0.71 & 0.83 & 0.47 & 0.74 & 0.54 & 0.67 & 0.33 & 0.56 & \bf{0.60}\\ 
% \hline
%%\hline
%
%\end{tabular}
%}
%\label{tab:accv}
%\end{table}

\begin{table}[h]
\caption{$O_{3D}$ and $\theta_{err}$ for sequences from ACCV dataset for PfD.}
\centering
\footnotesize
\resizebox{\columnwidth}{!}{
\begin{tabular}{|c|c|c|c|c|c|c|c|c|c|}
\hline 
& Iron & Duck & Ape & Can & Driller & Vise & Glue & Cat & Avg \\
\hline  
$O_{3D}$ & 0.71 & 0.83 & 0.47 & 0.74 & 0.54 & 0.67 & 0.33 & 0.56 & \bf{0.60}\\ \hline
$\theta_{err}$ & 0.17 &  0.13 &  0.23 & 0.13 & 1.00 & 0.43 & 0.01 &  0.57 & $\bf{ 0.33}$\\
 \hline
%\hline
\end{tabular}
}
\label{tab:accv}
\end{table}

%\begin{table}[h]
%\caption{$O_{3D}$ for sequences from ACCV dataset for PfD.}
%\centering
%\footnotesize
%\resizebox{\columnwidth}{!}{
%\begin{tabular}{|c|c|c|c|c|c|c|c|c|}
%\hline 
%             Iron & Duck & Ape & Can & Driller & Vise & Glue & Cat & Avg \\
%   \hline  
%    0.71 & 0.83 & 0.47 & 0.74 & 0.54 & 0.67 & 0.33 & 0.56 & \bf{0.60}\\ 
% \hline
%%\hline
%
%\end{tabular}
%}
%\label{tab:accv}
%\end{table}

%\begin{table}[h]
%\caption{$O_{3D}$ for sequences from ACCV dataset for PfD.}
%\centering
%\footnotesize
%\resizebox{\columnwidth}{!}{
%\begin{tabular}{|c|c|c|c|c|c|c|c|c|}
%\hline 
%             Iron & Duck & Ape & Can & Driller & Vise & Glue & Cat & Avg \\
%   \hline  
%        &    &   &    &   &   &   &   & \\ 
% \hline
%%\hline
%
%\end{tabular}
%}
%\label{tab:accv}
%\end{table}

\subsection{TUW dataset evaluation}
\label{TUW}
The TUW dataset \cite{aldoma2014automation} has $15$ annotated sequences showing a table with different sets of objects deployed over it. The number of frames per sequence ranges from $6$ to $20$. A 3D point cloud for each object is also provided. As for the ACCV dataset, we obtained the GT ellipsoids for each object and the 2D BBs.
%For each object we evaluated the ground truth ellipsoid as the envelope of the 3D point cloud. Moreover, for each frame and each object, we extracted the 2D bounding boxes in the following way: given the set of 2D projections of the 3D point cloud, we searched for the minimum size bounding box enclosing all the projected points and we increase such bounding box by a factor of $0.06$ in order to include a minimum of background. Such a procedure can be considered as a reasonable simulation of  the output of a multi-scale object detector providing bounding boxes with variable aspect ratio, like the well known Deformable Part Model (DPM) \cite{Felzenszwalb2010}. The resulting enriched dataset will be made available online at FIXME.
%
We discarded sequences with strong occlusions that cannot be handled by current object detectors, and sequences where objects appear for a number of frames lower than $3$, thus retaining $5$ sequences. 
In Fig. \ref{fig*:real_exp}, second row, an example of the localisation performance with the PfD+ES+REG method for a selected sequence is displayed. The accuracy in the estimation of the objects' pose is remarkable and this trend is confirmed for all the other objects in the ACCV dataset in term of size, eccentricity and alignment of GT ellipsoids. %Notice that for some objects the reconstructed ellipsoids (green wire mesh) is even tighter to the real object volume than the ground truth ellipsoid given by the object envelope (blue solid mesh). FIXME: FORSE QUESTO COMMENTO SUSCITA TROPPE DOMANDE. FIXME: si, se non si riesce a giustificare il perchè. %The difference probably depends on the mismatch between the ellipses inscribed in the bounding boxes and the ground truth ellipses for the four green objects, as can be inferred looking at the image frames (a), (b) and (c). 
%

All the selected sequences have been tested with the four methods PfD, PfD+REG, PfD+ES and PfD+ES+REG. The accuracy for each sequence is reported in Table \ref{tab:iros}, %for non regularized (NR) and regularized (R) methods,
according to $O_{3D}$ and $\theta_{err}$.  %$the two metrics $O_{2D}$ and $O_{3D}$. %In brackets is reported the percentage of failures for each sequence.  
It can be noticed that the regularization yields a sharp improvement in the accuracy, in terms of $O_{3D}$, with respect to non regularized methods on all the tested sequences. In particular, the accuracy is %increased of more than $50\%$ ($O_{2D}$) or
even doubled %($O_{3D}$) 
moving from PfD to PfD+REG. The ellipse fitting from segmentation yields a further improvement, leading a $5\%$ increment in accuracy moving from PfD+REG to PfD+ES+REG %($O_{3D}$) 
and a $3\%$ increment moving from PfD to PfD+ES. %($O_{3D}$). 
Remarkably, the improvement from PfD+REG to PfD+ES+REG is achieved in every sequence. % when measured in terms of $O_{3D}$. 
%All objects are reconstructed for all the sequences FIXME is it true ?
%\begin{table}[h]
%\footnotesize
%\caption{Average accuracies for the two sequences from TUW dataset.}
%\centering
%\begin{tabular}{|c|c|c|}
%\hline  
%    & $O_{2D}$ & $O_{3D}$      \\
%\hline
%Seq. 1 & 0,62 [0.0] &	 0,44 [0.0]   \\
%\hline
%Seq. 2   &  x  &  x  \\
%\hline
%\end{tabular}
%\label{tab:iros}
%\end{table}
%
%
%\begin{table}[h]
%\caption{Average accuracies for the sequences from TUW dataset. FIXME}
%\centering
%%\footnotesize
%\tabcolsep=0.08cm
%\begin{tabular}{|c|c|c|c|c|c|c|c|c|}
%\hline
%                 & \multicolumn{4}{c|}{\textbf{SVD}}                            & \multicolumn{4}{c|}{\textbf{REG}}       %                     \\ \hline
%                 & \multicolumn{2}{c|}{\textbf{N-RE}} & \multicolumn{2}{c|}{\textbf{RE}} & %\multicolumn{2}{c|}{\textbf{N-RE}} & \multicolumn{2}{c|}{\textbf{RE}} \\ \hline
%                 & $O_{2D}$    & $O_{3D}$    & $O_{2D}$   & $O_{3D}$   & $O_{2D}$    & $O_{3D}$    & $O_{2D}$   & %$O_{3D}$   \\ \hline
%seq1             & 0.16 & 0.09 & 0.17 & 0.12 & 0.71 & 0.48 & 0.69 & 0.50 \\ \hline
%seq7             & 0.63 & 0.43 & 0.59 & 0.38 & 0.64 & 0.49 & 0.68 & 0.56 \\ \hline
%seq8             & 0.00 & 0.00 & 0.17 & 0.16 & 0.58 & 0.48 & 0.61 & 0.52 \\ \hline
%seq10            & 0.75 & 0.60 & 0.79 & 0.65 & 0.81 & 0.71 & 0.81 & 0.74 \\ \hline
%seq11            & 0.34 & 0.16 & 0.21 & 0.09 & 0.64 & 0.40 & 0.59 & 0.43 \\ \hline
%\textbf{Average} & \bf{0.38} & \bf{0.25} & \bf{0.39} & \bf{0.28} & \bf{0.67} & \bf{0.51} & \bf{0.67} & \bf{0.55} \\ \hline
%\end{tabular}
%\end{table}
%
%
%\begin{table}[h]
%\caption{Average accuracies for the sequences from TUW dataset.}
%\centering
%%\footnotesize
%\tabcolsep=0.08cm
%\begin{tabular}{|c|c|c|c|c||c|c|c|c|}
%\hline
%                 & \multicolumn{4}{c||}{\textbf{$O_{2D}$}}                          &\multicolumn{4}{c|}{\textbf{$O_{3D}$}}                            \\ \hline
%                 & \multicolumn{2}{c|}{PfD} & \multicolumn{2}{c||}{PfD+REG} & \multicolumn{2}{c|}{PfD} & \multicolumn{2}{c|}{PfD+REG} \\ \hline
%                 &     & +ES    &    & +ES   &     & +ES   &    & +ES   \\ \hline
%seq1             & 0.16 & 0.17 & \bf{0.71} & 0.69 & 0.09 & 0.12 & 0.48 & \bf{0.50} \\ \hline
%seq7             & 0.63 & 0.59 & 0.64 & \bf{0.68} & 0.43 & 0.38 & 0.49 & \bf{0.56} \\ \hline
%seq8             & 0.00 & 0.17 & 0.58 & \bf{0.61} & 0.00 & 0.16 & 0.48 & \bf{0.52} \\ \hline
%seq10            & 0.75 & 0.79 & \bf{0.81} & \bf{0.81} & 0.60 & 0.65 & 0.71 & \bf{0.74} \\ \hline
%seq11            & 0.34 & 0.21 & \bf{0.64} & 0.59 & 0.16 & 0.09 & 0.40 & \bf{0.43} \\ \hline \hline
%\textbf{Average} & 0.38 & 0.39 & \bf{0.67} & \bf{0.67} & 0.25 & 0.28 & 0.51 & \bf{0.55} \\ \hline
%\end{tabular}
%\label{tab:iros}
%\end{table}
%
%
%\begin{table}[h]
%\caption{$O_{3D}$ for the sequences from TUW dataset.}
%\centering
%%\footnotesize
%\tabcolsep=0.08cm
%\begin{tabular}{|c|c|c|c|c|}
%\hline                                         
%                 &   \multicolumn{2}{c|}{PfD} & \multicolumn{2}{c|}{PfD+REG} \\ \hline
%                 &      &         +ES      &      & +ES   \\ \hline
%seq1             & 0.09 & 0.12 & 0.48 & \bf{0.50} \\ \hline
%seq7             &  0.43 & 0.38 & 0.49 & \bf{0.56} \\ \hline
%seq8             &  0.00 & 0.16 & 0.48 & \bf{0.52} \\ \hline
%seq10            &  0.60 & 0.65 & 0.71 & \bf{0.74} \\ \hline
%seq11            &  0.16 & 0.09 & 0.40 & \bf{0.43} \\ \hline \hline
%\textbf{Average} & 0.25 & 0.28 & 0.51 & \bf{0.55} \\ \hline
%\end{tabular}
%\label{tab:iros}
%\end{table}
%\vspace{-0.3 cm}

\begin{table}[h]
\footnotesize
\caption{$O_{3D}$ and $\theta_{err}$ for the sequences from TUW dataset.}
\centering
%\footnotesize
\tabcolsep=0.08cm
\begin{tabular}{|c|c|c|c|c|c|c|c|}
\hline                                         
              &   & Seq.1 & Seq.7 & Seq.8 & Seq.10 & Seq11 & \textbf{Avg}\\ \hline
\multirow{2}{*}{PfD}& $O_{3D}$  & 0.09 & 0.43  & 0.00 & 0.60 & 0.16 & 0.25 \\ 
              & $\theta_{err}$  & $ 0.69$ & $ 0.73$  & $ NN $ & $ 0.97$ & $ 1.34$  & $ 0.93$ \\ \hline
\multirow{2}{*}{PfD+ES}& $O_{3D}$ & 0.12 & 0.38 & 0.16 & 0.65 & 0.09 & 0.28 \\ 
         & $\theta_{err}$  & $ 0.69$ & $ 0.86$ & $ 0.77$ & $ 0.68$   & $ 1.1$  & $ 0.82$ \\ \hline
PfD+REG   &   $O_{3D}$   & 0.48  & 0.49   & 0.48 & 0.71   & 0.40  & 0.51 \\ \hline
PfD+REG+ES &  $O_{3D}$    & \bf{0.50} & \bf{0.56} & \bf{0.52} & \bf{0.74} & \bf{0.43} & \bf{0.55} \\ \hline

\end{tabular}
\label{tab:iros}
\end{table}

%\begin{table}[h]
%\footnotesize
%\caption{$O_{3D}$ for the sequences from TUW dataset.}
%\centering
%%\footnotesize
%\tabcolsep=0.08cm
%\begin{tabular}{|c|c|c|c|c|c|c|}
%\hline                                         
%                 & Seq.1 & Seq.7 & Seq.8 & Seq.10 & Seq11 & \textbf{Avg}\\ \hline
%PfD              & 0.09  & 0.43  & 0.00  & 0.60   & 0.16  & 0.25 \\ \hline
%PfD+ES           & 0.12  & 0.38   & 0.16 & 0.65   & 0.09  & 0.28 \\ \hline
%PfD+REG          & 0.48  & 0.49   & 0.48 & 0.71   & 0.40  & 0.51 \\ \hline
%PfD+REG+ES       & \bf{0.50} & \bf{0.56} & \bf{0.52} & \bf{0.74} & \bf{0.43} & \bf{0.55} \\ \hline
%
%\end{tabular}
%\label{tab:iros}
%\end{table}


%\vspace{-0.3 cm}
%\begin{table}[h]
%\caption{$O_{3D}$ for the sequences from TUW dataset [FIXME: Projection matrices from Samantha].}
%\centering
%%\footnotesize
%\tabcolsep=0.08cm
%\begin{tabular}{|c|c|c|c|c|c|c|}
%\hline                                         
%                 & Seq.1 & Seq.7 & Seq.8 & Seq.10 & Seq11 & \textbf{Avg}\\ \hline
%PfD              & 0.10  & 0.19  & 0.06  &  0.30  &  0.11 & 0.15 \\ \hline
%PfD+ES           & 0.07  & 0.17  & 0.08  &  0.34  &  0.07 & 0.14 \\ \hline
%PfD+REG          & 0.42  & 0.34  & 0.33  &  0.41  &  0.27 & 0.35 \\ \hline
%PfD+REG+ES       & 0.41  & 0.34  & 0.30  &  0.42  &  0.27 & 0.35 \\ \hline
%
%\end{tabular}
%\label{tab:iros}
%\end{table}

%\begin{table}[h]
%\footnotesize
%\caption{Average accuracies for the two sequences from TUW dataset.}
%\centering
%\begin{tabular}{|c|c|c|c|c|}
%\hline  
% & NR-$O_{2D}$  &  R-$O_{2D}$  & NR-$O_{3D}$  & R-$O_{3D}$ \\
%\hline
%Seq. 1   &  0,63 [0.62] &	x & 0,33 [0.62] & x \\
%\hline
%Seq. 3   &  0.72 [0.0]  & x & 0.59 [0.0]  & x \\
%\hline
%Seq. 4   &  0.68 [0.52] & x & 0.48 [0.52] & x \\
%\hline
%Seq. 5   &  0.72 [0.60] & x & 0.50 [0.60] & x \\
%\hline
%Seq. 6   &  0.65 [0.60] & x & 0.50 [0.60] & x \\
%\hline
%Seq. 7   &  0.63 [0.0]  & x & 0.42 [0.0]  & x \\
%%Seq. 8   &  0.75 [1] &  x  & 0.60 [1] & x \\
%\hline
%Seq. 9   &  0.62 [0.0]  & x & 0.51 [0.0]  & x \\
%\hline
%Seq. 10  &  0.75 [0.0]  & x & 0.60 [0.0]  & x \\
%\hline
%Seq. 11  &  0.67 [0.50] & x & 0.32 [0.50] & x \\
%\hline
%Seq. 12  &  0.65 [0.44] & x & 0.40 [0.44] & x \\
%\hline
%Seq. 14  &  0.68 [0.53] & x & 0.51 [0.53] & x \\
%\hline
%Seq. 15  &  0.63 [0.36] & x & 0.37 [0.36] & x \\
%\hline
%Average  &       []     & x &      []     & x \\
%\end{tabular}
%\label{tab:iros}
%\end{table}
%(FIXME: MANCANO I RISULTATI QUANTITATIVI SULL'INTERO DATASET)
%(FIXME: MANCA UN ACCENNO SUL FATTO CHE LE PROJECTIVE CAMERA MATRIX SI POSSONO STIMARE DALLE IMMAGINI)
%
%
%
%\vspace{-0.1 cm}
\subsection{KITTI dataset evaluation and comparisons}
\label{KITTI}
%
%
%
The KITTI dataset \cite{Geiger2013} is composed by a set of sequences taken from a camera mounted on a moving car in an urban environment. The dataset provides full annotations for cars appearing in each frame from which GT ellipsoids can be computed. We sampled $6$ sub-sequences displaying parked cars\footnote{The selected sequences (Seq.) and the corresponding frames (Fr.) defining the sub-sequences are the following: Seq 5 (Fr. 142 - 153); Seq. 9 (Fr. 90 - 106); Seq. 22 (Fr. 49 - 86); Seq. 23 (Fr. 1 - 17); Seq. 35 (Fr. 1 - 5); Seq. 36 (Fr. 44 - 57).}.   % We selected FIXME sequences with rigid objects only where a set of cars were detected on the sides of a street. 
%including 2D bounding boxes and 3D ground truth ellipsoids. We selected two sequences with rigid objects only where a set of cars were detected on the sides of a street. %and appear in more than three frames, since otherwise the 3D reconstruction would not be feasible, 
%and we pruned out the other sequences with moving objects, since their relative motion violates the rigidity assumption. 
%The number of frames for the selected sequences is respectively $12$ in \textit{sequence 5} and $40$ in \textit{sequence 9}. FIXME
%Differently from the previous datasets, we took advantage from the available pre-trained DPM classifier and generated 2D BB boxes applying the classifier to every frame of every sequence. FIXME SCRIVIAMO CHE IL MATCHING DELLE BB ACROSS FRAMES E' FATTO A MANO ?
We generated 2D BBs simulating a multi-scale object detector for each car and each frame. We pruned out cars with strong occlusions for which a reliable detection is unlikely. The number of remaining cars was of $34$ and the average number of views in which a car is visible is $10$. %From all the other objects we run the proposed method, taking as input the provided 2D bounding boxes.
Ellipsoids estimation is particularly challenging on this dataset, since the camera motion is almost planar. % (i.e. the one parallel to the street plane with distance from the street plane equal to the car height). 
Moreover, cars are usually placed at the street borders and the camera moves straight in most of the sequences. Hence, the range of angles between car and camera spanned by the sequence of camera views is very narrow and almost limited to the azimuth plane. Finally, each car appears in a limited subset of frames. We did not apply segmentation on this dataset due to the extreme difficulty in segmenting some of the cars that are partially overlapping each other.

%
In Table \ref{tab:kitti} quantitative results are displayed for the six selected sequences. Despite the difficulty of the dataset, PfD achieves a reasonable result of $O_{3D}=0.17$  and $\theta_{err}= 0.36$ and the use of regularization almost doubles the accuracy, yielding $O_{3D}=0.36$.  %demonstrated by the poor results achieved with PfD, with an $O_{2D}$ of just $0.05$. However, a striking improvement is gained with PfD+REG, leading the average accuracy to reasonable values ($O_{2D}=0.41$, $O_{3D}=0.21$), and achieving a successful reconstruction of most of the objects. 
The result for PfD+REG is visually confirmed by looking at Fig. \ref{fig*:real_exp2}: % regarding sequence 9: 
all the seven cars are correctly located in the 3D space, with a reasonable precision concerning the object size. The estimated aspect ratio and orientation of the cars is less accurate but acceptable considering the  challenging conditions characterizing this dataset.
Besides the occupancy values $O_{3D}$, our algorithm achieves excellent 3D localization performance. In Table \ref{tab:distances} the percentage of estimated ellipsoid centers within $1$ m and $2$ m from the GT centroids of the objects is reported. With PfD+REG almost all the cars ($93\%$) are within $2$ m and $76\%$ are within $1$ m. Moreover, PfD alone achieves very good results ($83\%< 2$ m and $81\%<1$ m on average). The lower average value of PfD+REG w.r.t. PfD ($<1$ m) can be explained by the fact that regularization enforces the sphere-likeness of the quadric, and due to the non-linearity of the problem, this may sometime decrease the accuracy in the 3D center estimation. Notice that, even for those estimated quadrics not corresponding to feasible ellipsoids, a quadric center can be still computed and also in this case its position is most of the times within $2$ m of the GT centroids.\\
The localization accuracy has been also compared with the one obtained by Choi et al. \cite{choi2010multiple} on the same sequences. Despite the use of priors given by ground plane, results of \cite{choi2010multiple}, displayed in Table \ref{tab:kitti}, are significantly worse than the ones obtained with our method.
%\footnote{In sequence $36$ results of \cite{choi2010multiple} are not available due to the lack of detected BBs.}
%\textbf{The comparison with other works can be performed considering only algorithms which do not use priors about the shape of the car (3D CAD shapes) or other inputs provided by the dataset (in eg. Velodyne point clouds) FIXME DA RIVEDERE}.


%The qualitative result with regularization for sequence FIXME can be appreciated in Fig. \ref{fig:kitti}: occupancy of parked cars on the left side is tightly estimated, while on the right side the occupancy of the nearest car is less precise, probably due to the strong perspective distortion causing a mismatch between bounding box and ground truth projection centers.
%
%
%\begin{table}[h]
%\footnotesize
%\caption{Average accuracies for the two sequences from KITTI dataset.}
%\centering
%%\footnotesize
%\tabcolsep=0.08cm
%\begin{tabular}{|c|c|c|c|c|c|c|c|c|}
%\hline
%                 & \multicolumn{2}{c|}{\textbf{SVD}}	& \multicolumn{2}{c|}{\textbf{REG}}  	\\ \hline
%                 	& $O_{2D}$ & $O_{3D}$		& $O_{2D}$ & $O_{3D}$ 		\\ \hline
%seq5             & 0.10          	& 0.07          	& 0.40          	& 0.14          	\\ \hline
%seq11            & 0.01          	& 0.01          	& 0.42          	& 0.27          	\\ \hline
%\textbf{Average} & \textbf{0.05} 	& \textbf{0.04} 	& \textbf{0.41} 	& \textbf{0.21} 	\\ \hline
%\end{tabular}
%\label{tab:kitti}
%\end{table}
%

%\begin{table}[h]
%\footnotesize
%\caption{Average accuracies for the two sequences from KITTI dataset.}
%\centering
%%\footnotesize
%%\tabcolsep=0.08cm
%\begin{tabular}{|c|c|c||c|c|}
%\hline
%                 & \multicolumn{2}{c||}{$O_{2D}$}	& \multicolumn{2}{c|}{$O_{3D}$}  	\\ \hline
%                 & PfD & PfD+REG		& PfD & PfD+REG 		\\ \hline
%Seq. 5             & 0.55          	& 0.72         	& 0.31          	& 0.32          	\\ \hline
%Seq. 9            & 0.34          	& 0.64          	& 0.13          	& 0.50          	\\ \hline \hline
%\textbf{Average} & 0.45 	& \textbf{0.68} 	& 0.22 	& \textbf{0.41} 	\\ \hline
%\end{tabular}
%\label{tab:kitti}
%\end{table}


%\begin{table}[h]
%\footnotesize
%\caption{$O_{3D}$ for the two sequences from KITTI dataset.}
%\centering
%%\footnotesize
%%\tabcolsep=0.08cm
%\begin{tabular}{|c|c|c|}
%\hline            
%                 			& PfD 				& PfD+REG 			\\ \hline
%Seq. 5                    	& 0.31          	& 0.32         		\\ \hline
%Seq. 9                    	& 0.13          	& 0.50          	\\ \hline \hline
%\textbf{Average}			& 0.22 				& \textbf{0.41} 	\\ \hline
%\end{tabular}
%\label{tab:kitti}
%\end{table}


%\begin{table}[h]
%\footnotesize
%\caption{$O_{3D}$ for the two sequences from KITTI dataset. [FIXME: BB from detections, Camera pose from GT]}
%\centering
%%\footnotesize
%%\tabcolsep=0.08cm
%\begin{tabular}{|c|c|c|}
%\hline            
%                 			& PfD 	& PfD+REG 		\\ \hline
%Seq. 5                    	& 0.20  & 0.30        	\\ \hline
%Seq. 9                    	& 0.07 	& 0.31    		\\ \hline \hline
%\textbf{Average}			& 0.14	& 0.31			\\ \hline
%\end{tabular}
%\label{tab:kitti}
%\end{table}


%\begin{table}[h]
%\footnotesize
%\caption{$O_{3D}$ for the two sequences from KITTI dataset. [FIXME: BB from detections, Camera pose from Samantha]}
%\centering
%%\footnotesize
%%\tabcolsep=0.08cm
%\begin{tabular}{|c|c|c|}
%\hline            
%                 			& PfD 		& PfD+REG 		\\ \hline
%Seq. 5                    	& 0.27      & 0.37         	\\ \hline
%Seq. 9                    	& 0.08     	& 0.25    		\\ \hline \hline
%\textbf{Average}			& 0.18		& 0.31			\\ \hline
%\end{tabular}
%\label{tab:kitti}
%\end{table}

%\begin{table}[h]
%\footnotesize
%\caption{$O_{3D}$ for the FIXME sequences from KITTI dataset. Bounding Boxes in 2D are yielded by the DPM classifier \cite{Felzenszwalb2010}.}
%\centering
%%\footnotesize
%%\tabcolsep=0.08cm
%\begin{tabular}{|c|c|c|c|c|c|}
%\hline            
%                 			& Seq. 5  & Seq. 9 	& Seq. x & Seq. y	& \textbf{Avg.}\\ \hline
%PfD                  	&  0.20   & 0.07  &   &  & \\ \hline
%PfD+REG               &  0.30   & 0.31  &  	&	 & \\ \hline \hline
%\textbf{Average}			& 		& 	&		&  & \\ \hline
%\end{tabular}
%\label{tab:kitti}
%\end{table}

\begin{table}[h]
\footnotesize
\caption{$O_{3D}$  and $\theta_{err}$ for the sequences from the KITTI dataset.}
\centering
%\footnotesize
%\tabcolsep=0.08cm
\begin{tabular}{|p{0.5cm}|p{0.5cm}|c|c|c|c|c|c|c|}
\hline            
         &        	  &  S.5  	& S.9 	& S.22 & S.23	& S.35 & S.36 & \bf{Avg.} \\ \hline
\multirow{2}{*}{PfD}  & $O_{3D}$             &  0.13  	& 0.05 & 0.13  & 0.48 & 0.17  & 0.08  & 0.17 \\  
& $\theta_{err}$ & $0.24$ & $0.22$ & $ 0.16$ & $ 0.21$ & $ 0.55$ & $ 0.07$ & $ 0.24$ \\ \hline
PfD \newline +REG    & $O_{3D}$           &  0.21 	& 0.49  & 0.45 & 0.37   & 0.30 & 0.35 &	0.36 \\ \hline 
\end{tabular}
\label{tab:kitti}
\end{table}

%\begin{table}[h]
%\footnotesize
%\caption{$O_{3D}$ for the sequences from the KITTI dataset.}
%\centering
%%\footnotesize
%%\tabcolsep=0.08cm
%\begin{tabular}{|c|c|c|c|c|c|c|c|}
%\hline            
%                 	  &  S.5  	& S.9 	& S.22 & S.23	& S.35 & S.36 & \bf{Avg.} \\ \hline
%PfD                   &  0.13 	& 0.05  & 0.13 & 0.48  	& 0.17 & 0.08 & 0.17 \\ \hline
%PfD+REG               &  0.21 	& 0.49  & 0.45 & 0.37   & 0.30 & 0.35 &	0.36 \\ \hline 
%\end{tabular}
%\label{tab:kitti}
%\end{table}

\begin{table}[h]
\caption{Percentages of estimated centroids within 1 m or 2 m w.r.t. GT centroids for the 6 sequences of the KITTI dataset.}
\centering
\footnotesize
%\footnotesize
%\tabcolsep=0.08cm
\begin{tabular}{|c|c|c|c|c|c|c|c|}
\hline            
                 		& S.5  		& S.9		& S.22	  & S.23	 &  S.35    & S.36   & \bf{Avg}	  \\ \hline
PfD  $<$1m              & $\bf{80}$        & 71        & 86      & $\bf{100}$      & $\bf{50}$       & $\bf{100}$    &  $\bf{81}$    \\ \hline                    	
PfD+REG $<$1m          	& 60        & $\bf{86}$       	& $\bf{100}$	  & 75		 & $\bf{50}$   & 86     &  76     \\ \hline
\cite{choi2010multiple} $<$1m & 20        & 0       	& 10	  & 27		 & 0       & 0     &  10     \\ \hline \hline
PfD $<$2m           	& $\bf{80}$        & 71        & $\bf{100}$      & $\bf{100}$      & 50       & $\bf{100}$   & 83      \\ \hline
PfD+REG $<$2m        	& $\bf{80}$        & $\bf{100}$       & $\bf{100}$     & $\bf{100}$      & $\bf{75}$      & $\bf{100}$   & $\bf{93}$ 	\\ \hline
\cite{choi2010multiple} $<$2m & 40        & 33        & 30      & 45     & 0       & 0   & 25      \\ \hline
                
                  
\end{tabular}
\label{tab:distances}
\end{table}

\subsection{Minimal configurations test}

In order to stress the method capabilities, we also tested minimal configurations of three and two views, the latter being solvable with regularization only due to its ill-posed nature. We have chosen a set of views with a wide baseline (about $45$ deg in rotation) so making difficult to compute any disparity from the detected objects. Table \ref{tab:Min} shows that in general accuracy is lower (ACCV) or slightly lower (TUW, KITTI) than in case of more views but still remaining reasonable for all the datasets. The results obtained by adding regularisation provides increments and performance in line with the previous experiments for the ACCV and KITTI datasets, while in TUW the third view allows to increase notably the accuracy.%may be comparable with respect to the case of three views for ACCV: this happens more in case of objects which have an isotropical distribution of the volume in the space (Duck, Ape and Vise in the dataset ACCV and some specific objects in the TUW dataset). In the KITTI dataset, the regularization improves the performances in both cases (two or three views).}. 


%\begin{table}[h]
%\caption{$O_{3D}$ for sequences of 3 or 2 views from ACCV dataset for PfD.}
%\centering
%\footnotesize
%\resizebox{\columnwidth}{!}{
%\begin{tabular}{|c|c|c|c|c|c|c|c|c|c|}
%\hline 
% & Iron & Duck & Ape & Can & Driller & Vise & Glue & Cat & Avg \\
%\hline  
%PfD+REG 2V  & 0.45 & 0.68 & 0.63 & 0.50 & 0.37 & 0.43 & 0.18 & 0.45 & \bf{0.46}\\ \hline
%PfD 3V   & 0.47 & 0.63 & 0.62 & 0.51 & 0.33 & 0.43 & 0.19 & 0.47 & \bf{0.46}\\
% \hline
%%\hline
%\end{tabular}
%}
%\label{tab:accvMin}
%\end{table}

%\begin{table}[h]
%\footnotesize
%\caption{$O_{3D}$ considering 2 or 3 views for the sequences from TUW dataset.}
%\centering
%%\footnotesize
%\tabcolsep=0.08cm
%\begin{tabular}{|c|c|c|c|c|c|c|}
%\hline                                         
%                 & Seq.1 & Seq.7 & Seq.8 & Seq.10 & Seq11 & \textbf{Avg}\\ \hline
%PfD 3V            & 0.10 & 0.21  & 0.07 & 0.54 & 0.10 & 0.20 \\ \hline
%PfD+ES 3V        & 0.06 & 0.36  & 0.09 & 0.65 & 0.06 & 0.25 \\ \hline
%PfD+REG 2V        & 0.34  & 0.23   & 0.32 & 0.01   & 0.36  & 0.25 \\ \hline
%PfD+REG 3V        & 0.47  & 0.46   & 0.53 & 0.72   & 0.38  & 0.51 \\ \hline
%PfD+REG+ES 2V      & 0.24 & 0.14 & 0.42 & 0.06 & 0.37 & 0.25 \\ \hline
%PfD+REG+ES 3V      & 0.47 & 0.53 & 0.51 & 0.74 & 0.44 & 0.54 \\ \hline
%\end{tabular}
%\label{tab:irosMin}
%\end{table}



\begin{table}[h]
\footnotesize
\caption{Average $O_{3D}$ for 2 and 3 views for the sequences from ACCV, TUW and KITTI dataset.}
\centering
%\footnotesize
\tabcolsep=0.08cm
\begin{tabular}{|c|c|c|c|}
\hline                                         
                 & ACCV & TUW & KITTI   \\ \hline
PfD 3views           & 0.46 & 0.20  & 0.12  \\ \hline
%PfD+ES 3        &      & 0.25  &       \\ \hline
PfD+REG 3views       & 0.47     & 0.51  & 0.34  \\ \hline
PfD+REG 2views       & 0.46 & 0.25  & 0.34  \\ \hline
%PfD+REG+ES 2V    &      & 0.25 &    \\ \hline
%PfD+REG+ES 3V    &      & 0.54 &     \\ \hline
\end{tabular}
\label{tab:Min}
\end{table}

% RESULTS WITH CENTERING/// MAYBE BETTER WITHOUT CENTERING, DATA ARE SIMILAR WITH PREVIOUS!!
%\begin{table}[h]
%\footnotesize
%\caption{$O_{3D}$ considering 2 or 3 views for the sequences from the KITTI dataset.}
%\centering
%%\footnotesize
%%\tabcolsep=0.08cm
%\begin{tabular}{|c|c|c|c|c|c|c|c|}
%\hline            
%                 	  &  S.5  	& S.9 	& S.22 & S.23	& S.35 & S.36 & \bf{Avg.} \\ \hline
%                 	  
%PfD 3V                &  0.08  	& 0.00 & 0.20  & 0.31 & 0.06  & 0.07  & 0.12 \\ \hline 
%%   CASE WITH CENTERING             	  
%%PfD 3V                &  0.29  	& 0.49 & 0.48  & 0.00 & 0.10  & 0.44  & 0.30 \\ \hline
%PfD+REG 2V            &  0.25 	& 0.50  & 0.21 & 0.17   & 0.49 & 0.41 &	0.34 \\ \hline
%PfD+REG 3V            &  0.28 	& 0.46  & 0.37 & 0.21   & 0.39 & 0.36 &	0.34 \\ \hline 
%\end{tabular}
%\label{tab:kitti23views}
%\end{table}


%\begin{figure*}
%\centering
%\includegraphics[width=4cm]{./figures/ACCV1.png}
%\includegraphics[width=4 cm]{./figures/ACCV2.png}
%\includegraphics[width=4 cm]{./figures/ACCV3.png}
%\includegraphics[width=4 cm]{./figures/3dShape.png}\\
%\hspace{-0cm} (a)  \hspace{3.5cm} (b)  \hspace{3.5cm}  (c) \hspace{3.5cm} (d)
%\label{fig:kitti}
%\caption{Cars reconstruction from KITTI \cite{hinterstoisser2012accv} dataset. Subfigures (a),(b),(c) represent three views taken from the first tested sequence. The red bounding box reproduces the output of a generic object detector; the blue ellipse and the green ellipse are the projections of ground truth and reconstructed ellipsoids respectively. Subfigure (d) contains the 3D shape of the object (grey), the ground truth ellipsoid which fits the object (blue solid mesh) and the reconstructed ellipsoid (green wire mesh).}
%\end{figure*}
%

%Finally, an example of the segmentation process for accurate ellipse fitting (Section. ) is displayed in Fig. \ref{}. (a) (b) (c) (d) 
%(FIXME : RELATIVAMENTE AI TRE DATASET VERRANNO AGGIUNTI ESEMPI RELATIVI ALLE PERFORMANCE OTTENUTE TRAMITE SEGMENTAZIONE DEL CONTENUTO DELLE BOUNDING BOXES)
%\vspace{-0.2cm}
\section{Discussions and Future Work} \label{sec:concl}

This paper presented a closed-form solution to recover the 3D occupancy of objects from 2D detections in multi-view. This algebraic solution is achieved through the estimation of a 3D quadric given 2D ellipsoids fitted at the objects detectors BBs. Moreover a regularization approach was devised to cope with possible ill-conditioning of the problem.  The approach was tested against the common inaccuracies affecting object detectors such as coarse estimation of the object center, tightness of the BB in respect to the object size and variations over the object pose. To strengthen further the approach, a  robust ellipse fitting method was introducing using a segmentation algorithm over the detected BBs in multi-view.   Experiments show that even with relevant errors, the estimated quadrics are able to localise the object in 3D and to define a reasonable occupancy. Moreover, the proposed %regularization and the
estimation of object orientation, by means of a segmentation algorithm, can be used in order to increase accuracy and the percentage of the reconstructed quadrics.

The solutions of this problem has strong practical breakthroughs given the recent evolution of recognition algorithms. In particular, object detection is certainly going towards increased generality, so providing detectors for several object classes \cite{dean:etal:2013}. Thus, the proposed method can provide a quick and very efficient solution to leverage the 2D information for 3D scene understanding where objects can be inter-related given their position in the metric space. %Also, the model for obtaining quadrics from 2D ellipsoid can be used as a quick mean to prune out wrong detections that do not comply with the rigidity of the object scene (NOTE, FIXME: this is an icip or even bmvc paper, not sure we should give hint, lomettiamo nel paper submitted, non nell'arxiv). 
This will inject important 3D reasoning in classic frameworks for object detection that are mostly restricted to 2D reasoning.

Regarding future work, mis-detections (i.e. outliers) might affect negatively the estimation of the quadrics. Thus, including further robustness in the optimization, through ad hoc regularization terms in the cost function, might improve the performance of the system. Moreover, the ellipsoid orientations given the BBs can be further improved, especially when objects occlusions or similar appearance textures are present. %(FIXME: more things to do?).  

{\small
\bibliographystyle{ieee}
%\bibliography{egbib,2015_bib_sfom}
\bibliography{2015_bib_sfom}
}


\end{document}

defs.tex
\renewcommand{\floatpagefraction}{0.9}

\def\v#1{\protect\vec #1}
\newcommand{\tabeq}[2]{\begin{minipage}{#1}\vspace{-.75\baselineskip}\begin{equation}#2\end{equation}\vspace{-.8\baselineskip}\end{minipage}}
\newcommand{\area}[3]{{A}_{\v {#1} \v {#2} \v {#3}}}
\newcommand{\Det}[3]{|\v #1 \v #2 \v #3|}
%\newtheorem{theorem}{Theorem}
\def\mat#1{\mathchoice{\mbox{\boldmath$\displaystyle\tt#1$}}
{\mbox{\boldmath$\textstyle\tt#1$}}
{\mbox{\boldmath$\scriptstyle\tt#1$}}
{\mbox{\boldmath$\scriptscriptstyle\tt#1$}}}
\def\m#1{\protect\mat #1}
\newcommand{\matx}[1]{{\tt #1}}
\def\vec#1{\mathchoice{\mbox{\boldmath  $\displaystyle\bf#1$}}
{\mbox{\boldmath  $\textstyle\bf#1$}}
{\mbox{\boldmath  $\scriptstyle\bf#1$}}
{\mbox{\boldmath  $\scriptscriptstyle\bf#1$}}}
\newcommand{\tr}{\mbox{$^{\top}$}}
\newenvironment{myenum}{\newcounter{myenumi}\begin{list}{(\roman{myenumi})}{\usecounter{myenumi}}}{\end{list}}

\def\dac{\mat Q_\infty^\ast}
\def\diac{\mat \omega_\infty^\ast}

% User-defined lengths
% -------------------
\newlength{\colwidth}
\setlength{\colwidth}{0.2\textwidth}

macros3.tex
%\input{local-definitions}
\newcommand{\trinv}{\mbox{$^{\top\,-1}$}}
\newcommand{\mtr}{\mbox{$^{-\top}$}}
\newcommand{\rank}{\mbox{\rm rank}}
\newcommand{\sign}{\mbox{\rm sign}}
\newcommand{\diag}{\mbox{\rm diag}}
\newcommand{\med}{\mbox{\rm med}} % maybe change/drop this
\newcommand{\adj}{\mbox{$^*$}}
\def\ijk{_i^{jk}}
\def\Tri{{\cal T}}
\def\Tijk{\Tri\ijk}

% Real and complex numbers
\newcommand{\R}{{\cal R}}
\newcommand{\C}{{\cal C}}

% Standardized references
\newcommand{\app}[1]{appendix~\ref{app:#1}}
\newcommand{\App}[1]{Appendix~\ref{app:#1}}
\newcommand{\eq}[1]{(\ref{eq:#1})}
\newcommand{\eqn}[1]{(\ref{eq:#1})}
\newcommand{\fig}[1]{Fig~\ref{fig:#1}}
\newcommand{\tab}[1]{table~\ref{tab:#1}}
\newcommand{\Tab}[1]{Table~\ref{tab:#1}}
\newcommand{\sect}[1]{section~\ref{sec:#1}}
\newcommand{\ch}[1]{chapter~\ref{ch:#1}}
\newcommand{\Ch}[1]{Chapter~\ref{ch:#1}}
\newcommand{\thm}[1]{Theorem~\ref{thm:#1}}

\newcommand{\num}[1]{(~\ref{num:#1})}
\newcommand{\Thm}[1]{Theorem~\ref{thm:#1}}
\newcommand{\pro}[1]{Proposition~\ref{pro:#1}}
\newcommand{\Pro}[1]{Proposition~\ref{pro:#1}}
\newcommand{\defn}[1]{definition~\ref{def:#1}}
\newcommand{\algo}[1]{table~\ref{alg:#1}}
\newcommand{\lem}[1]{lemma~\ref{lem:#1}}

\newcommand{\SKIP}[1]{} % Used to skip stuff we do not want type-set

%
\newcommand{\vv}[2]
   {\mbox{$
      \left(
         \begin{array}{c}
         #1 \\ #2
         \end{array}
      \right )
   $}}
%
\newcommand{\vvv}[3]
   {\mbox{$
      \left(
         \begin{array}{c}
         #1 \\ #2 \\ #3
         \end{array}
      \right )
   $}}
%
\newcommand{\vvvv}[4]
   {\mbox{$
      \left(
         \begin{array}{c}
         #1 \\ #2 \\ #3 \\ #4
         \end{array}
      \right )
   $}}
%
\newcommand{\vm}[2]
   {\mbox{$
      \left[
         \begin{array}{c}
         #1 \\ #2
         \end{array}
      \right ]
   $}}
%
\newcommand{\vvm}[3]
   {\mbox{$
      \left[
         \begin{array}{c}
         #1 \\ #2 \\ #3
         \end{array}
      \right ]
   $}}
%
\newcommand{\vvvm}[4]
   {\mbox{$
      \left[
         \begin{array}{c}
         #1 \\ #2 \\ #3 \\ #4
         \end{array}
      \right ]
   $}}
%
\newcommand{\mm}[4]
   {\mbox{$
      \left[
         \begin{array}{ccc}
         #1 & #2    \\
         #3 & #4 
         \end{array}
      \right]
   $}}
%
\newcommand{\mmm}[9]
   {\mbox{$
      \left[
         \begin{array}{ccc}
         #1 & #2 & #3   \\
         #4 & #5 & #6   \\
         #7 & #8 & #9
         \end{array}
      \right]
   $}}

%
% Quick way to do matrices
\newcommand{\mbegin} {\left [ \begin{array}}
\newcommand{\mstart} {\mbegin}
\newcommand{\mc}     {\mbegin{c}}
\newcommand{\mcc}    {\mbegin{cc}}
\newcommand{\mccc}   {\mbegin{ccc}}
\newcommand{\mcccc}  {\mbegin{cccc}}
\newcommand{\mccccc} {\mbegin{ccccc}}
\newcommand{\mend}   {\end{array} \right ]}

% Same for vectors
\newcommand{\vbegin} {\left ( \begin{array}{c}}
\newcommand{\vstart} {\vbegin}
\newcommand{\vend} {\end{array}\right )}

%
%
%
\newcommand{\lra}{\leftrightarrow}
\newcommand{\Po}{\mbox{\({\cal P}^1\)}}
\newcommand{\PP}{\mbox{\({\cal P}^2\)}}
\newcommand{\PPP}{\mbox{\({\cal P}^3\)}}
\newcommand{\PPPPP}{\mbox{\({\cal P}^5\)}}
\newcommand{\Pn}{\mbox{\({\cal P}^n\)}}
\newcommand{\sk}[1]{[#1]_\times}
\newcommand{\bm}[1]{\mbox{\boldmath $#1$}}
%

% Stuff for doing the qed sign.
\def\squareforqed{\hbox{\rlap{$\sqcap$}$\sqcup$}}

\def\qed{\ifmmode\squareforqed\else{\unskip\nobreak\hfil
	\penalty50\hskip1em\null\nobreak\hfil\squareforqed
	\parfillskip=0pt\finalhyphendemerits=0\endgraf}\fi}

% Stuff for doing vector. NB need to not use small for lower case.
\def\vecs#1{\mathchoice%
	{\mbox{\boldmath \small $\displaystyle\bf#1$}}
	{\mbox{\boldmath \small $\textstyle\bf#1$}}
	{\mbox{\boldmath \small $\scriptstyle\bf#1$}}
	{\mbox{\boldmath \small $\scriptscriptstyle\bf#1$}}}
\def\vs#1{\protect\vecs #1}

\def\vec#1{\mathchoice%
	{\mbox{\boldmath $\displaystyle\bf#1$}}
	{\mbox{\boldmath $\textstyle\bf#1$}}
	{\mbox{\boldmath $\scriptstyle\bf#1$}}
	{\mbox{\boldmath $\scriptscriptstyle\bf#1$}}}
\def\v#1{\protect\vec #1}

% Stuff for making labels visible
\newcommand{\showlabel}[1]
	{\hskip 0.0em\marginpar{\quad\fbox{\scriptsize\rm#1}}}

\newcommand{\showeqnlabel}{
	\hbox to 0pt{\quad\quad\relax\fbox{\scriptsize\rm\eqnlblx}%
	\gdef\eqnlblx{xxxx}}} \newcommand{\eqnlblx}{}

\def\@eqnnum{\rm (\theequation)\showeqnlabel}

\newcommand{\eqnlabel}[1]{\protect\label{#1}\gdef\eqnlblx{#1}}

\newcommand{\mylabel}[1]{\label{#1}\showlabel{#1}}

\newcommand{\nofig}[1]{\centerline{\bf Figure here}}

% Stuff for doing matrix.
\def\mat#1{\mathchoice{\mbox{\boldmath$\displaystyle\tt#1$}}
	{\mbox{\boldmath$\textstyle\tt#1$}}
	{\mbox{\boldmath$\scriptstyle\tt#1$}}
	{\mbox{\boldmath$\scriptscriptstyle\tt#1$}}}
\def\m#1{\protect\mat #1}

%\newcommand{\matx}[1]{{\tt #1}}


2015_arxiv_sfd.bbl
\begin{thebibliography}{10}\itemsep=-1pt

\bibitem{aldoma2014automation}
A.~Aldoma, T.~Faulhammer, and M.~Vincze.
\newblock Automation of �ground truth� annotation for multi-view rgb-d
  object instance recognition datasets.
\newblock In {\em 2014 IEEE/RSJ International Conference on Intelligent Robots
  and Systems (IROS)}, pages 5016--5023. IEEE, 2014.

\bibitem{bao:savarese:2011}
S.~Y. Bao and S.~Savarese.
\newblock Semantic structure from motion.
\newblock In {\em 2011 IEEE Conference on Computer Vision and Pattern
  Recognition (CVPR)}, pages 2025--2032. IEEE, 2011.

\bibitem{choi2010multiple}
W.~Choi and S.~Savarese.
\newblock Multiple target tracking in world coordinate with single, minimally
  calibrated camera.
\newblock In {\em Computer Vision--ECCV 2010}, pages 553--567. Springer, 2010.

\bibitem{Cross}
G.~Cross and A.~Zisserman.
\newblock Quadric reconstruction from dual-space geometry.
\newblock In {\em Sixth International Conference on Computer Vision (ICCV)},
  pages 25--31. IEEE, 1998.

\bibitem{dean:etal:2013}
T.~Dean, M.~A. Ruzon, M.~Segal, J.~Shlens, S.~Vijayanarasimhan, and J.~Yagnik.
\newblock Fast, accurate detection of 100,000 object classes on a single
  machine.
\newblock In {\em 2013 IEEE Conference on Computer Vision and Pattern
  Recognition (CVPR)}, pages 1814--1821. IEEE, 2013.

\bibitem{divvala2012important}
S.~K. Divvala, A.~A. Efros, and M.~Hebert.
\newblock How important are "deformable parts" in the deformable parts model?
\newblock In {\em 2012 European Conference on Computer Vision (ECCV) .
  Workshops and Demonstrations}, pages 31--40. Springer, 2012.

\bibitem{Felzenszwalb2010}
P.~F. Felzenszwalb, R.~B. Girshick, D.~McAllester, and D.~Ramanan.
\newblock Object detection with discriminatively trained part-based models.
\newblock {\em IEEE Transactions on Pattern Analysis and Machine Intelligence},
  32(9):1627--1645, 2010.

\bibitem{Fioraio:DiStefano:2013}
N.~Fioraio and L.~Di~Stefano.
\newblock Joint detection, tracking and mapping by semantic bundle adjustment.
\newblock In {\em 2013 IEEE Conference on Computer Vision and Pattern
  Recognition (CVPR)}, pages 1538--1545, 2013.

\bibitem{Geiger2013}
A.~Geiger, P.~Lenz, C.~Stiller, and R.~Urtasun.
\newblock Vision meets robotics: The kitti dataset.
\newblock {\em International Journal of Robotics Research (IJRR)}, June 2013.

\bibitem{hartley2003multiple}
R.~Hartley and A.~Zisserman.
\newblock {\em Multiple view geometry in computer vision}.
\newblock Cambridge University Press, 2003.

\bibitem{hejrati2014analysis}
M.~Hejrati and D.~Ramanan.
\newblock Analysis by synthesis: 3d object recognition by object
  reconstruction.
\newblock In {\em 2014 IEEE Conference on Computer Vision and Pattern
  Recognition (CVPR)}, pages 2449--2456. IEEE, 2014.

\bibitem{henderson1979vec}
H.~V. Henderson and S.~Searle.
\newblock Vec and vech operators for matrices, with some uses in jacobians and
  multivariate statistics.
\newblock {\em Canadian Journal of Statistics}, 7(1):65--81, 1979.

\bibitem{hinterstoisser2012accv}
S.~Hinterstoisser, V.~Lepetit, S.~Ilic, S.~Holzer, G.~Bradski, K.~Konolige, and
  N.~Navab.
\newblock Model based training, detection and pose estimation of texture-less
  3d objects in heavily cluttered scenes.
\newblock In {\em 2012 Asian Conference on Computer Vision (ACCV)}, pages
  548--562. Springer, 2012.

\bibitem{Hoiem:etal:2008}
D.~Hoiem, A.~A. Efros, and M.~Hebert.
\newblock Putting objects in perspective.
\newblock {\em International Journal of Computer Vision}, 80(1):3--15, 2008.

\bibitem{jojic2009stel}
N.~Jojic, A.~Perina, M.~Cristani, V.~Murino, and B.~Frey.
\newblock Stel component analysis: Modeling spatial correlations in image class
  structure.
\newblock In {\em 2009 IEEE Conference on Computer Vision and Pattern
  Recognition ( CVPR)}, pages 2044--2051. IEEE, 2009.

\bibitem{pollefeys1999self}
M.~Pollefeys, R.~Koch, and L.~Van~Gool.
\newblock Self-calibration and metric reconstruction inspite of varying and
  unknown intrinsic camera parameters.
\newblock {\em International Journal of Computer Vision}, 32(1):7--25, 1999.

\bibitem{salas:etal:2013}
R.~F. Salas-Moreno, R.~A. Newcombe, H.~Strasdat, P.~H. Kelly, and A.~J.
  Davison.
\newblock Slam++: Simultaneous localisation and mapping at the level of
  objects.
\newblock In {\em 2013 IEEE Conference on Computer Vision and Pattern
  Recognition (CVPR)}, pages 1352--1359. IEEE, 2013.

\bibitem{silberman:etal:2012}
N.~Silberman, D.~Hoiem, P.~Kohli, and R.~Fergus.
\newblock Indoor segmentation and support inference from rgbd images.
\newblock In {\em 2012 European Conference on Computer Vision ( ECCV)}, pages
  746--760. Springer, 2012.

\bibitem{sturm:triggs:1996}
P.~Sturm and B.~Triggs.
\newblock A factorization based algorithm for multi-image projective structure
  and motion.
\newblock In {\em 1996 European Conference on Computer Vision (ECCV)}, pages
  709--720. Springer, 1996.

\bibitem{Tomasi:Kanade:1992}
C.~Tomasi and T.~Kanade.
\newblock Shape and motion from image streams under orthography: a
  factorization method.
\newblock {\em International Journal of Computer Vision}, 9(2):137--154, 1992.

\bibitem{vicente2014reconstructing}
S.~Vicente, J.~Carreira, L.~Agapito, and J.~Batista.
\newblock Reconstructing pascal voc.
\newblock In {\em 2014 IEEE Conference on Computer Vision and Pattern
  Recognition (CVPR)}, pages 41--48. IEEE, 2014.

\bibitem{xiang2012estimating}
Y.~Xiang and S.~Savarese.
\newblock Estimating the aspect layout of object categories.
\newblock In {\em 2012 IEEE Conference on Computer Vision and Pattern
  Recognition (CVPR)}, pages 3410--3417. IEEE, 2012.

\bibitem{zia:etal:2013}
M.~Z. Zia, M.~Stark, B.~Schiele, and K.~Schindler.
\newblock Detailed 3d representations for object recognition and modeling.
\newblock {\em IEEE Transactions on Pattern Analysis and Machine Intelligence},
  35(11):2608--2623, 2013.

\bibitem{Zia2014}
M.~Z. Zia, M.~Stark, and K.~Schindler.
\newblock Are cars just 3d boxes? jointly estimating the 3d shape of multiple
  objects.
\newblock In {\em 2014 {IEEE} Conference on Computer Vision and Pattern
  Recognition (CVPR)}, pages 3678--3685, 2014.

\end{thebibliography}

2015_bib_sfom.bbl
% This file was created with JabRef 2.9.2.
% Encoding: Cp1252

@INPROCEEDINGS{Zia2014,
  author = {Muhammad Zeeshan Zia and Michael Stark and Konrad Schindler},
  title = {Are Cars Just 3D Boxes? Jointly Estimating the 3D Shape of Multiple
	Objects},
  booktitle = {2014 {IEEE} Conference on Computer Vision and Pattern Recognition (CVPR)},
  year = {2014},
  pages = {3678--3685},
}

@INPROCEEDINGS{aldoma2014automation,
  author = {Aldoma, Aitor and Faulhammer, Thomas and Vincze, Markus},
  title = {Automation of �ground truth� annotation for multi-view RGB-D object
	instance recognition datasets},
  booktitle = { 2014 IEEE/RSJ International
	Conference on Intelligent Robots and Systems (IROS)},
  year = {2014},
  pages = {5016--5023},
  organization = {IEEE}
}

@ARTICLE{Andrews2013,
  author = {Andrews, James and S\'{e}quin, Carlo H},
  title = {{Type-Constrained Direct Fitting of Quadric Surfaces}},
  year = {2013},
  volume = {10},
  pages = {1--15},
  doi = {10.3722/cadaps.2013.xxx-yyy},
  keywords = {10,2013,3722,cadaps,cones,cylinders,doi,ellipsoids,hyperboloids,paraboloids,quadrics,reverse
	engineering,rotational symmetry,surface fitting,xxx-yyy}
}

@ARTICLE{Ax,
  author = {Ax, C and Hua, Dal},
  title = {{On the Symmetric Solutions of Linear Matrix Equations}},
  volume = {7},
  pages = {1--7},
  number = {2},
  file = {:C$\backslash$:/Users/CRubino/Desktop/1-s2.0-002437959090370R-main.pdf:pdf}
}

@INPROCEEDINGS{bao:etal:2012,
  author = {Bao, Sid Yingze and Bagra, Mohit and Chao, Yu-Wei and Savarese, Silvio},
  title = {Semantic structure from motion with points, regions, and objects},
  booktitle = { 2012 IEEE Conference
	on Computer Vision and Pattern Recognition (CVPR)},
  year = {2012},
  pages = {2703--2710},
  organization = {IEEE}
}

@INPROCEEDINGS{bao:savarese:2011,
  author = {Bao, Sid Yingze and Savarese, Silvio},
  title = {Semantic structure from motion},
  booktitle = {2011 IEEE Conference
	on Computer Vision and Pattern Recognition (CVPR) },
  year = {2011},
  pages = {2025--2032},
  organization = {IEEE}
}

@ARTICLE{Bian2013,
  author = {Bian, Yuchen and Dong, Feng and Zhang, Weida and Wang, Hongyi and
	Tan, Chao and Zhang, Zhiqiang},
  title = {{3D reconstruction of single rising bubble in water using digital
	image processing and characteristic matrix}},
  journal = {Particuology},
  year = {2013},
  volume = {11},
  pages = {170--183},
  number = {2},
  month = apr,
  doi = {10.1016/j.partic.2012.07.005},
  issn = {16742001},
  publisher = {Chinese Society of Particuology},
  url = {http://linkinghub.elsevier.com/retrieve/pii/S1674200112001915}
}

@ARTICLE{Chu1989,
  author = {Chu, King-wah Eric},
  title = {{Symmetric solutions of linear matrix equations by matrix decompositions}},
  journal = {Linear Algebra and its Applications},
  year = {1989},
  volume = {119},
  pages = {35--50},
  month = jul,
  doi = {10.1016/0024-3795(89)90067-0},
  issn = {00243795},
  url = {http://linkinghub.elsevier.com/retrieve/pii/0024379589900670}
}

@INPROCEEDINGS{Cross,
  author = {Cross, Geoffrey and Zisserman, Andrew},
  title = {Quadric reconstruction from dual-space geometry},
  booktitle = {Sixth International Conference on Computer Vision (ICCV)},
  year = {1998},
  pages = {25--31},
  organization = {IEEE}
}

@INPROCEEDINGS{dean:etal:2013,
  author = {Dean, Thomas and Ruzon, Mark A and Segal, Mark and Shlens, Jonathon
	and Vijayanarasimhan, Sudheendra and Yagnik, Jay},
  title = {Fast, accurate detection of 100,000 object classes on a single machine},
  booktitle = {2013 IEEE Conference
	on Computer Vision and Pattern Recognition (CVPR)},
  year = {2013},
  pages = {1814--1821},
  organization = {IEEE}
}

@INPROCEEDINGS{divvala2012important,
  author = {Divvala, Santosh K and Efros, Alexei A and Hebert, Martial},
  title = {How important are "Deformable Parts"  in the Deformable Parts
	Model?},
  booktitle = {2012 European Conference on Computer Vision (ECCV) . Workshops and Demonstrations},
  year = {2012},
  pages = {31--40},
  organization = {Springer}
}

@ARTICLE{Eberly2013,
  author = {Eberly, David},
  title = {{Reconstructing an Ellipsoid from its Perspective Projection onto
	a Plane Determining the Elliptical Cone Bounding the Ellipsoid}},
  year = {2013},
  pages = {5--9}
}

@ARTICLE{Felzenszwalb2010,
  author = {Felzenszwalb, Pedro F and Girshick, Ross B and McAllester, David
	and Ramanan, Deva},
  title = {Object detection with discriminatively trained part-based models},
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence },
  year = {2010},
  volume = {32},
  pages = {1627--1645},
  number = {9},
  publisher = {IEEE}
}

@INPROCEEDINGS{Fioraio:DiStefano:2013,
  author = {Fioraio, N. and Di Stefano, L.},
  title = {Joint Detection, Tracking and Mapping by Semantic Bundle Adjustment},
  booktitle = {2013 IEEE Conference
	on Computer Vision and Pattern Recognition (CVPR) },
  year = {2013},
  pages = {1538-1545}
}

@ARTICLE{Geiger2013,
  author = {Andreas Geiger and Philip Lenz and Christoph Stiller and Raquel Urtasun},
  title = {Vision meets Robotics: The KITTI Dataset},
  journal = {International Journal of Robotics Research (IJRR)},
month = {June},
  year = {2013}
}

@INCOLLECTION{gupta:etal:2010,
  author = {Gupta, Abhinav and Efros, Alexei A and Hebert, Martial},
  title = {Blocks world revisited: Image understanding using qualitative geometry
	and mechanics},
  booktitle = {2010 European Conference on Computer Vision (ECCV) },
  publisher = {Springer},
  year = {2010},
  pages = {482--496}
}

@ARTICLE{Gurdjos,
  author = {Gurdjos, Pierre and Charvillat, Vincent},
  title = {{Multiple View Reconstruction of a Quadric of Revolution from its
	Occluding Contours}}
}

@ARTICLE{Gurdjos2009,
  author = {Gurdjos, Pierre and Charvillat, Vincent and Morin, G},
  title = {{Multiple View Reconstruction of a Quadric of Revolution from its
	Occluding Contours}},
  year = {2009},
  pages = {2009},
  number = {september}
}

@BOOK{hartley2003multiple,
  title = {Multiple view geometry in computer vision},
  publisher = {Cambridge University Press},
  year = {2003},
  author = {Hartley, Richard and Zisserman, Andrew}
}

@INPROCEEDINGS{hejrati2014analysis,
  author = {Hejrati, Mohsen and Ramanan, Deva},
  title = {Analysis by synthesis: 3d object recognition by object reconstruction},
  booktitle = {2014 IEEE Conference
	on Computer Vision and Pattern Recognition (CVPR) },
  year = {2014},
  pages = {2449--2456},
  organization = {IEEE}
}

@ARTICLE{henderson1979vec,
  author = {Henderson, Harold V and Searle, SR},
  title = {Vec and vech operators for matrices, with some uses in jacobians
	and multivariate statistics},
  journal = {Canadian Journal of Statistics},
  year = {1979},
  volume = {7},
  pages = {65--81},
  number = {1},
  publisher = {Wiley Online Library}
}

@INCOLLECTION{hinterstoisser2012accv,
  author = {Hinterstoisser, Stefan and Lepetit, Vincent and Ilic, Slobodan and
	Holzer, Stefan and Bradski, Gary and Konolige, Kurt and Navab, Nassir},
  title = {Model based training, detection and pose estimation of texture-less
	3D objects in heavily cluttered scenes},
  booktitle = {2012 Asian Conference on Computer Vision (ACCV)},
  publisher = {Springer},
  year = {2012},
  pages = {548--562}
}

@ARTICLE{Hoiem:etal:2008,
  author = {Hoiem, Derek and Efros, Alexei A. and Hebert, Martial},
  title = {Putting Objects in Perspective},
  journal = {International Journal of Computer Vision},
  year = {2008},
  volume = {80},
  pages = {3--15},
  number = {1},
  month = October,
  issn = {0920-5691},
  issue_date = {October 2008}
}

@ARTICLE{Jacq2007,
  author = {Jacq, Jean-jos\'{e} and Burdin, Val\'{e}rie and Roux, Christian and
	Bretagne, Get-enst and Couture, Christine},
  title = {{Type-Constrained Robust Fitting of Quadrics with Application to
	the 3D Morphological Characterization of Saddle-Shaped Articular
	Surfaces St\'{e}phane Allaire}},
  year = {2007},
  isbn = {9781424416318}
}

@INPROCEEDINGS{jojic2009stel,
  author = {Jojic, Nebojsa and Perina, Alessandro and Cristani, Marco and Murino,
	Vittorio and Frey, Brendan},
  title = {Stel component analysis: Modeling spatial correlations in image class
	structure},
  booktitle = { 2009 IEEE Conference
	on Computer Vision and Pattern Recognition ( CVPR)},
  year = {2009},
  pages = {2044--2051},
  organization = {IEEE}
}

@ARTICLE{Kang,
  author = {Kang, Kongbin and Kangbrownedu, Kongbin and Tarel, Jean-philippe
	and Cedex, Le Chesnay and Cooper, David B},
  title = {{A Unified Linear Fitting Approach for Singular and Non-Singular
	3D Quadrics from Occluding Contours}}
}

@ARTICLE{Kang2001,
  author = {Kang, K. and Tarel, J.-P. and Fishman, R. and Cooper, D.},
  title = {{A linear dual-space approach to 3D surface reconstruction from occluding
	contours using algebraic surfaces}},
  journal = {Proceedings Eighth IEEE International Conference on Computer Vision.
	ICCV 2001},
  year = {2001},
  volume = {1},
  pages = {198--204},
  doi = {10.1109/ICCV.2001.937518},
  isbn = {0-7695-1143-0},
  publisher = {IEEE Comput. Soc},
  url = {http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=937518}
}

@ARTICLE{Karl1993,
  author = {Karl, William C and Verghese, George C and Willsky, Alan S},
  title = {{Reconstructing Ellipsoids from Projections *}},
  year = {1993},
  number = {617}
}

@ARTICLE{lorenz2005robust,
  author = {Lorenz, Robert G and Boyd, Stephen P},
  title = {Robust minimum variance beamforming},
  journal = {Signal Processing, IEEE Transactions on},
  year = {2005},
  volume = {53},
  pages = {1684--1696},
  number = {5},
  publisher = {IEEE}
}

@ARTICLE{Ma1993,
  author = {Ma, Song},
  title = {{Conics-based stereo, motion estimation, and pose determination}},
  journal = {International Journal of Computer Vision},
  year = {1993},
  volume = {10},
  pages = {7--25},
  number = {1},
  month = feb,
  doi = {10.1007/BF01440844},
  issn = {0920-5691},
  url = {http://link.springer.com/10.1007/BF01440844}
}

@ARTICLE{Pattern1996,
  author = {Pattern, National},
  title = {{Ellipsoid Reconstruction from Three Perspective Views*}},
  year = {1996},
  pages = {344--348}
}

@ARTICLE{pollefeys1999self,
  author = {Pollefeys, Marc and Koch, Reinhard and Van Gool, Luc},
  title = {Self-calibration and metric reconstruction inspite of varying and
	unknown intrinsic camera parameters},
  journal = {International Journal of Computer Vision},
  year = {1999},
  volume = {32},
  pages = {7--25},
  number = {1},
  publisher = {Springer}
}

@ARTICLE{Pope2008,
  author = {Pope, Stephen B},
  title = {{Algorithms for Ellipsoids}},
  year = {2008},
  number = {February}
}

@INPROCEEDINGS{salas:etal:2013,
  author = {Salas-Moreno, Renato F and Newcombe, Richard A and Strasdat, Hauke
	and Kelly, Paul HJ and Davison, Andrew J},
  title = {Slam++: Simultaneous localisation and mapping at the level of objects},
  booktitle = {2013 IEEE Conference
	on Computer Vision and Pattern Recognition (CVPR) },
  year = {2013},
  pages = {1352--1359},
  organization = {IEEE}
}

@INCOLLECTION{sturm:triggs:1996,
  author = {Sturm, Peter and Triggs, Bill},
  title = {A factorization based algorithm for multi-image projective structure
	and motion},
  booktitle = {1996 European Conference on Computer Vision (ECCV)},
  publisher = {Springer},
  year = {1996},
  pages = {709--720}
}

@ARTICLE{Surface1994,
  author = {Surface, Quadric and Occluding, From},
  title = {{Quadric Surface From Occluding Contour}},
  year = {1994},
  pages = {27--31}
}

@ARTICLE{tikhonov1977solutions,
  author = {Tikhonov, Andrej Nikolaevich and Arsenin, Vasiliy Yakovlevich},
  title = {Solutions of ill-posed problems},
  year = {1977},
  publisher = {Winston}
}

@ARTICLE{Tomasi:Kanade:1992,
  author = {Tomasi, Carlo and Kanade, Takeo},
  title = {Shape and motion from image streams under orthography: a factorization
	method},
  journal = {International Journal of Computer Vision},
  year = {1992},
  volume = {9},
  pages = {137--154},
  number = {2},
  publisher = {Springer}
}

@ARTICLE{Vetter1975,
  author = {Vetter, William J},
  title = {{Vector Structures and Solutions of Linear Matrix Equations}},
  year = {1975},
  volume = {188},
  pages = {181--188},
  file = {:C$\backslash$:/Users/CRubino/AppData/Local/Mendeley Ltd./Mendeley Desktop/Downloaded/Vetter - 1975 - I ’.pdf:pdf}
}

@INPROCEEDINGS{vicente2014reconstructing,
  author = {Vicente, Sara and Carreira, Joao and Agapito, Lourdes and Batista,
	Jorge},
  title = {Reconstructing pascal voc},
  booktitle = {2014 IEEE Conference
	on Computer Vision and Pattern Recognition (CVPR)},
  year = {2014},
  pages = {41--48},
  organization = {IEEE}
}

@ARTICLE{Wokes2010,
  author = {Wokes, David S. and Palmer, P. L.},
  title = {{Perspective Reconstruction of a Spheroid from an Image Plane Ellipse}},
  journal = {International Journal of Computer Vision},
  year = {2010},
  volume = {90},
  pages = {369--379},
  number = {3},
  month = jul,
  doi = {10.1007/s11263-010-0368-0},
  issn = {0920-5691},
  keywords = {image plane,perspective,perspective reconstruction,projection,spheroid},
  url = {http://link.springer.com/10.1007/s11263-010-0368-0}
}

@INPROCEEDINGS{xiang2012estimating,
  author = {Xiang, Yu and Savarese, Silvio},
  title = {Estimating the aspect layout of object categories},
  booktitle = {2012 IEEE Conference
	on Computer Vision and Pattern Recognition (CVPR) },
  year = {2012},
  pages = {3410--3417},
  organization = {IEEE}
}

@ARTICLE{zia:etal:2013,
  author = {Zia, M Zeeshan and Stark, Michael and Schiele, Bernt and Schindler,
	Konrad},
  title = {Detailed 3d representations for object recognition and modeling},
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence },
  year = {2013},
  volume = {35},
  pages = {2608--2623},
  number = {11},
  publisher = {IEEE}
}

@INPROCEEDINGS{,
  owner = {mcrocco},
  timestamp = {2014.11.09}
}

@MISC{,
  title = {{Objects from projections.pdf}}
}

@ARTICLE{,
  title = {{The Central Conic Sections Revisited}},
  year = {2014},
  volume = {66},
  pages = {322--325},
  number = {5}
}

@incollection{silberman:etal:2012,
  title={Indoor segmentation and support inference from RGBD images},
  author={Silberman, Nathan and Hoiem, Derek and Kohli, Pushmeet and Fergus, Rob},
  booktitle={2012 European Conference on Computer Vision ( ECCV) },
  pages={746--760},
  year={2012},
  publisher={Springer}
}

@incollection{choi2010multiple,
  title={Multiple target tracking in world coordinate with single, minimally calibrated camera},
  author={Choi, Wongun and Savarese, Silvio},
  booktitle={Computer Vision--ECCV 2010},
  pages={553--567},
  year={2010},
  publisher={Springer}
}

