Automatic Dialect Detection in Arabic Broadcast Speech
\input{abstract
abstract.tex
In this paper, we investigate different approaches for dialect identification in Arabic broadcast speech. These methods are based on phonetic and lexical features obtained from a speech recognition system, and bottleneck features using the i-vector framework. We studied both generative and discriminative classifiers, and we combined these features using a multi-class Support Vector Machine (SVM). We validated our results on an Arabic/English language identification task, with an accuracy of 100\%. We also evaluated these features in a binary classifier to discriminate between Modern Standard Arabic (MSA) and Dialectal Arabic, with an accuracy of 100\%.  We further reported results using the proposed methods to discriminate between the five most widely used dialects of Arabic: namely Egyptian, Gulf, Levantine, North African, and MSA, with an accuracy of 59.2\%. We discuss dialect identification errors in the context of dialect code-switching between Dialectal Arabic and MSA, and compare the error pattern between manually labeled data, and the output from our classifier. All the data used on our experiments have been released to the public as a language identification corpus. 
classifiers.tex
\label{sec:classifier}
\subsection{Generative Model}
\subsubsection{Trigram Langugage Modelling}
A different trigram model with Kneser-Ney smoothing is built for each dialect using the training set. The language model can be represented by equation~\ref{lm}.
\begin{equation}
  P(w_{1},w_{2},\ldots,w_{n})=\prod_{i=1}^{n} P(w_{i} | w_{i-2},w_{i-1}) \label{lm}
\end{equation}
where, $w_{0}=w_{-1}=*$. $*$ is a special start symbol~\cite{Collins}.

Given a test utterance and its correponding word sequence, perplexity of the word sequence is calculated using each dialectal language model. The one that gives the lowest perplexity is chosen as the dialect of the test utterance.

\subsubsection{Naive Bayes}
We use a standard Naive Bayes classifier as used in text classification, where the feature vector, $x$ is assumed to be made up of discrete features, where each element $x_{i}$, is either $0$ or $1$, indicating the presence or absence of the word at dictionary index $i$. The feature vector dimensionality is equal to the vocabulary size. The model parameters are estimated via Maximum Likelihood Estimation using the training set. For more details and the likelihhod equation, the reader is referred to an excellent tutorial~\cite{Ng}.

Given a test feature vector, $x^{*}$, and the model parameters $\theta$, the class $c$ (dialect) that $x^{*}$ belongs to is given by:

\begin{equation}
  c=\argmax_{c} p(x^{*}|y=c,\theta)p(y=c)
\end{equation}

\subsection{Discriminative Classifiers}
\subsubsection{Maximum Entropy}
Maximum Entropy classification is another popular technique used in document classiciation and we use the same technique for utterance classification to determine the dialect of the utterance. For the lack of space we refer the reader to the seminal work done by~\cite{Nigam}.

\subsubsection{Linear SVM}
We used a multi-class kernel classifier using regularization parameters to trade off margin size and training error. Given a test utterance the SVM otputs a score for each dialect, and the dialect that is given the highest score is chosen as the dialect of the utterance

code-switching.tex
\label{sec:code-switching}
 The detailed error pattern and confusion matrix in Table~\ref{tab:best_system} shows that GLF and LAV very often got confused more than any other pair. We hypothesise that this is related to the greater lexical similarity between LAV and GLF, illustrated in Table~\ref{tab:lexical_difference}.  % We also note that MSA significantly outperforms the rest of dialectal Arabic in classification.  We believe that this is because the acoustic training data mainly comprises MSA data, which can be considered to tune lexical and phonetic features to MSA.

Speaker linking information in the test set has been calculated using~\cite{meignier2010lium}. In our experiments, it is assumed that each segment (belonging to one speaker) is one dialect and speakers will not switch between dialects without reasonable silence (chosen to be 300ms), which seems to be a rather naive assumption. The authors have plans to visit a more fine-grained dialect diarization in the future~\cite{Wu2015} %This assumption has been accepted by the crowdsource community\footnote{It can be argued that in dialectal Arabic speakers can switch between MSA and dialects without silence. The authors have plans to visit a more fine-grained dialect diarization in the future.}.
  \begin{figure}[ht!]
    \centering
    \includegraphics[scale=0.25]{DialectDetectionSpace}
    \caption{LDA projection for All Dialectal Features.}
    \label{fig:best_system}
  \end{figure}
%\subsection{MSA and Dialectal Arabic Code-Switching}
As shown in Figure~\ref{fig:best_system}, we see two dialects LAV and GLF overlap more with MSA compared to NOR or EGY. The crowdsourcing experiment~\cite{samn2015crowdsource} also provided manual annotations of 200 speech segments from each dialect as well as MSA. The input comprises speech segments from speakers with known dialect, and the judges chose freely which dialect each speech segment belongs to. For example, there are 200 samples from potential Gulf speakers. After labeling them, there are 106 segments labeled to be MSA, 82 validated as GLF, and 8 LAV, and 4 segments were not identified with enough confidence to be considered.  The results are shown in Table~\ref{tab:expansion2}, where it can be observed that code-switching happens most often between GLF/MSA and LAV/MSA. It is very interesting to observe that potential GLF speakers in this sample speak MSA more than 50\% of the time, and for LAV speakers, nearly 40\% of their speech is in MSA. This summary could explain that the overlap for MSA, GLF and LAV in Figure~\ref{fig:best_system} mainly occurs in code-switching rather than just an error in dialect identification.

Arabic native speakers will often speak only one dialect but will in many cases switch to MSA when using formal speech. It is therefore reasonable to assume that a detected switch between dialects is likely to be an error. This is a challenge for a subjective human judge according to our finding in some of the crowdsource feedback. %, where the decision is forced between closed dialects such as GLF or LAV based on the acoustics alone.


\begin{table}[ht!]
\centering
\begin{tabular}{p{2.4cm} | r | r | r | r |r}
  \hline
  Expected Dialect & EGY & GLF & LAV & NOR & MSA\\ \hline
  EGY &65\%\cellcolor[gray]{0.9}&&&&32\%\\ \hline
   GLF & &41\%&4\%&&53\%\cellcolor[gray]{0.9}\\ \hline
   LAV &1\%& 1\%& 53\%\cellcolor[gray]{0.9}&&39\%\\ \hline
   NOR & 1\%& &&69\%\cellcolor[gray]{0.9}&28\%\\
  \hline
\end{tabular}
\caption{Expected dialect of the file based on expansion at a 75\% confidence threshold from initial annotation by actual results}
\label{tab:expansion2}
\end{table}

conclusion.tex
This paper presents our efforts on automatic dialect identification for Arabic broadcast speech. We have demonstrated a dialect classifier with an accuracy of 60.2\% using system combination. We also achieved 100\% accuracy on two binary classification tasks; MSA vs Dialectal Arabic and English vs Arabic. We studied the potential code-switching pattern in our classifier and its correlation with the manual annotation. Further work for this research is to study the code-switch between MSA and dialectal Arabic without considering speaker diarization or silence between speech segments in what can be called dialect diarization. We shall also study deep neural network approaches of classification to learn a more complex non-linear decision boundary.

data.tex
\label{sec:data}

 \subsection{Train Data}

The training corpus was collected from the Broadcast News domain in four Arabic dialects (EGY, LAV, GLF, and NOR) as well as MSA. Data recordings were carried out at 16Khz. The recordings were segmented to avoid speaker overlap, removing any non-speech parts such as music and background noise. More details about the training data can be found in~\cite{bahari2014non}. Although the test database came from the same broadcast domain, the recording setup is different. The test data was downloaded directly from the high quality video server for Aljazeera (brightcove) over the period of July 2104 until January 2015, as part of QCRI Advanced Transcription Service (QATS)~\cite{ali2014qcri}.
\begin{table}[pht!]
\centering
\ra{1.3}
\resizebox{48mm}{7mm}{%
\begin{tabular}{@{}llllllll@{}} \toprule Data & EGY & GLF & LAV & NOR & MSA & ENG\\ \midrule Train& 13  & 9.5  & 11 & 9 & 10 & 10 \\
 Test& 2  & 2 & 2 &  2 & 2 & 2\\
\bottomrule \end{tabular}%
}
\caption{\textit{Number of hours of speech available for each dialect.}}\label{tab:data1}
\end{table}
\subsection{Test Data}
\label{testset}
The test set was labeled using the crowdsource platform CrowdFlower, with the criteria to have a minimum of three judges per file and up to nine judges, or 75\% inter-annotator agreement (whichever comes first). More details about the test set and crowdsourcing experiment can be found in~\cite{samn2015crowdsource}. The test set used in this paper differs from that used in~\cite{bahari2014non} for two reasons: First, the crowdsourced data is available to reproduce the results, and thus can be used as a standard test set for Arabic DID; second, the new test set has been collected using different channels, and recording setup compared to the training data, which makes our experiments less sensitive to channel/speaker characteristics.

The train and test data can be found on the QCRI web portal\footnote{http://alt.qcri.org/resources/ArabicDialectIDCorpus/}. Table~\ref{tab:data1} and Table~\ref{tab:data2} present some statistics about the train and the test data.

\begin{table}[pht!]
\centering
\ra{1.3}
\resizebox{48mm}{7mm}{%
\begin{tabular}{@{}llllllll@{}} \toprule Data & EGY & GLF & LAV & NOR & MSA & ENG\\ \midrule Train& 1720  & 1907  & 1059 & 1934 & 1820 & 1649 \\
 Test& 315  & 348  & 238 &  355 & 265 & 452 \\
\bottomrule \end{tabular}%
}
\caption{\textit{Number of speech utterances for each dialect.}}\label{tab:data2}
\end{table}

dialect_challenges.tex
\label{sec:dialect_challenges}
 
An objective comparison of the varieties of Arabic dialects, especially at the edges of the Arabic-speaking area, might lead someone to conclude that they were historically related, but are synchronically and mutually unintelligible languages like English and Dutch; normal vernacular can be difficult to understand across different Arabic Dialects~\cite{holes2004modern}. 

%This inspired us to look at speech DID  in Arabic dialects similarly to LID task in other languages, which makes this task unique and interesting in itself. One can look at some examples across different dialects, as well as in MSA that shows significant lexical variates as shown in Table~\ref{tab:lexical_difference}.
Arabic dialects are thus sufficiently distinctive, that it is reasonable to regard the DID task in Arabic as similar to the LID task in other languages.  Table~\ref{tab:lexical_difference} shows examples of two phrases across some different dialects, as well as MSA, and makes clear the lexical variation, which is  greater than when comparing differences across dialects in many other languages such as English.    
 
\FloatBarrier   
\begin{table}[ht]
\label{lexicalExamples}
\caption{Lexical examples showed in Arabic and Buckwalter format}
\resizebox{0.5\textwidth}{!}{%
\begin{tabular}{|l|l|l|l|l|l|l|}
\hline
         & EGY     & GLF    & LAV           & MSA      & NOR     & English Translation \\ \hline
Example1 &   \begin{tabular}[x]{@{}c@{}} \novocalize \RL{AzAyk} \\AzAYk\end{tabular}                                                               & 
\begin{tabular}[x]{@{}c@{}} \novocalize \RL{A^slwnk} \\ A\$lwnk\end{tabular}                             & \begin{tabular}[x]{@{}c@{}} \novocalize \RL{A^slwnk / kyfk} \\ kyfk / A\$lwnk \end{tabular}      & \begin{tabular}[x]{@{}c@{}} \novocalize \RL{kyf .hAlk} \\ kyf HAlk\end{tabular}                            &  \begin{tabular}[x]{@{}c@{}} \novocalize \RL{wA^s rAk} \\ wA\$ rAk\end{tabular}                          & 
How are you?        \\ \hline

Example2 &   \begin{tabular}[x]{@{}c@{}} \novocalize \RL{Ant fen} \\Ant fyn\end{tabular}       &  
\begin{tabular}[x]{@{}c@{}} \novocalize \RL{wynk} \\wynk\end{tabular}                       				  & 
\begin{tabular}[x]{@{}c@{}} \novocalize \RL{wynk} \\wynk\end{tabular}                        				  & 
\begin{tabular}[x]{@{}c@{}} \novocalize \RL{Ayn Ant} \\ Ayn Ant\end{tabular}           					 &  
\begin{tabular}[x]{@{}c@{}} \novocalize \RL{wyn rAk} \\ wyn rAk\end{tabular}             				&
Where are you?        \\ \hline
\end{tabular}
}
\label{tab:lexical_difference}
\end{table}
\FloatBarrier

\noindent
%The two aforementioned examples in Table~\ref{tab:lexical_difference} show major lexical differences between Arabic dialects, and with MSA as well, %with an exception of LAV and GLF.
%clearly this is not the case in other languages such as when comparing American, British, and Australian English dialects. In this paper, when dialectal Arabic is mentioned, it does not include MSA. In this study, we depict lexical, phonetic, and acoustic features; namely four discriminative categories: phoneme sequence, phoneme duration, lexical, and acoustic features.


dialect_features.tex
\label{sec:dialectfeatures}
\subsection{Senone based Utterance VSM}
\label{sec:svsm}
\textbf{Senone} refers to an n-gram phone sequence. In our case $n \le 4$.
VSM construction takes place in two steps:
%\begin{itemize}[noitemsep,topsep=3pt]
%  \item 
first, a phoneme recognizer is used to extract the senone~\cite{Hwang} sequence for a given speech utterance. The phoneme sequence is obtained by automatic vowelization of the training text, followed by vowelization to phonetization (V2P). The 36 chosen phonemes cover all the dialectal Arabic sounds. Further details about the speech recognition pipeline, training data, and phoneme set is given in~\cite{ali2014complete}. For the phoneme sequence, we process the phoneme lattice, and obtain the one-best transcription, ignoring silences as well as noisy silences.
%  \item 
Each speech utterance ($\mathbf{u}$) is then represented as a high dimensional sparse vector ($\mathbf{\vv{u}}$):% as given by equation~\ref{sv}
    \begin{equation}
      \mathbf{\vv{u}} = \left(A(f(u,s_{1})), A(f(u,s_2)), \ldots, A(f(u,s_d))\right), \label{sv}
    \end{equation}
where $f(u,s_{i})$ is the number of times a senone $s_{i}$ occurs in the speech utterance $u$, and  $A$ is the scaling function. We experiment with both an identity scaling function and $tf.idf$ scaling function, commonly used in the field of Natural Language Processing~\cite{Ramos} to downweight the contribution of the words (in our case senones) that occur in almost all documents (in our case utterances), as these words (senones) do not provide any discriminative information about the documents (utterances).  

%\end{itemize}

The vector space is then represented by the matrix, $\mathbf{U_{s}} \in \mathbb{R}^{d \times N}$ (see Fig~\ref{table-semspace}).  This approach and the notation used to define a VSM is directly inspired by the seminal works in the area of VSM of Natural Language in~\cite{Salton1975,Lowe2000,Pado2007} and in LID~\cite{Li2007}.
\begin{figure}[H]
\centering
\small
\[
  \text{$\mathbf{U_{s}}$} = \kbordermatrix{
    & u_1 & u_2 & \dots & u_N \\
    s_1 & A(f(s_1,u_1) & A(f(s_1,u_2) & \ldots & A(f(s_1,u_N) \\
    s_2 & A(f(s_2,u_1) & A(f(s_2,u_2) & \dots & A(f(s_2,u_N) \\
    \vdots & \vdots  & \vdots &\ddots & \vdots \\
    s_d & A(f(s_d,u_1) & A(f(s_d,u_2) & \dots  & A(f(s_d,u_N)
  }
\]
\normalsize
\caption{\textit{Senone-based utterance VSM. Column vectors of the matrix correspond to the speech utterance vector representation formed using equation~\ref{sv}. $d$ is the size of the senone dictionary, and $N$ is the total number of speech utterances in the dialectal speech database.}}
\label{table-semspace}
\end{figure}

\subsection{Word based Utterance VSM}
\label{sec:wordvsm}
The word-based utterance VSM ($\mathbf{U_{w}}$) is constructed in two steps in a manner similar to the senone features:
%\begin{itemize}[noitemsep,topsep=2pt]
%  \item 
An ASR system is used to extract the word sequence for each utterance in the speech database. Details about the ASR system can be found in~\cite{ali2014complete}.
%  \item 
Each speech utterance ($\mathbf{u}$) is then represented as a high-dimensional sparse vector ($\mathbf{\vv{u}}$):% as given by euqation~\ref{wv}
    \begin{equation}
      \mathbf{\vv{u}} = \left(A(f(u,w_{1})), A(f(u,w_2)), \ldots, A(f(u,w_{d^{\prime}}))\right) , \label{wv}
    \end{equation}
where $f(u,w_{i})$ is the number of times a word $w_{i}$ occurs in the speech utterance $u$ and $A$ is the scaling function which has the same interpretation as for $\mathbf{U_{s}}$ (above). Vocabulary size was 55k. The tri-gram dictionary size was 580k which we used to construct the word based VSM

%\end{itemize}

\subsection{i-vector-based Utterance VSM}
\label{sec:ivsm}
\subsubsection{Bottleneck Features (BN)}
Recently, bottleneck features extracted from an ASR DNN-based model were applied successfully to language identification~\cite{Ysong2013,Pavel2014,Fred2015}. In this paper, we used a similar bottleneck features configurations as in our previous ASR-DNN system for MSA speech recognition~\cite{Card2015}. This system is based on two successive DNN models. Both DNNs use the same setup of 5 hidden sigmoid layers and 1 linear BN layer, and they were both based on tied-states as target outputs. The senone labels of dimension 3040 are generated by a forced alignment from an HMM-GMM baseline trained on 60 hours of manually transcribed Al-Jazeera MSA news recordings ~\cite{ali2014complete}.
The input to the first DNN consists of 23 critical-band energies that are obtained from Mel filter-bank. Pitch and voicing probability are then added. 11 consecutive frames are then stacked together. The second DNN is used for correcting the posterior outputs of the first DNN. In this architecture, the input features of the second DNN are the outputs of the BN layer from the first DNN. Context expansion is achieved by concatenating frames with time offsets of -10, -5, 0, 5, and 10. Thus, the overall time context seen by the second DNN is 31 frames.

\subsubsection{Modeling}
An effective and well-studied method in language and dialect recognition is the i-vector approach ~\cite{bahari2014non,dehak2011front,dehak2011language}. The i-vector involves  modeling speech using a universal background model (UBM) -- typically a large GMM -- trained on a large amount of data to represent general feature characteristics, which plays a role of a prior on how all dialects look like. The i-vector approach is a powerful technique that summarizes all the updates happening during the adaptation of the UBM mean components to a given utterance. All this information is modeled in a low dimensional subspace referred to as the total variability space. In the i-vector framework, each speech utterance can be represented by a GMM supervector, which is assumed to be generated as follows:
\[M = u + Tv \]
Where $u$ is the channel and dialect independent supervector (which can be taken to be the UBM supervector),
$T$ spans a low-dimensional subspace and $v$ are the factors that best describe the utterance-dependent mean offset. The vector $v$ is treated as a latent variable with the  i-vector being its maximum-a-posteriori (MAP) point estimate. The subspace matrix $T$ is estimated using maximum likelihood on large training dataset. An efficient procedure for training and for MAP adaptation of i-vector can be found in~\cite{kenny2008study}. In this approach, the i-vector is the low-dimensional representation of an audio recording that can be used for classification and estimation purposes. In our experiments, the UBM was a GMM with 2048 components, BN features were used, and the i-vectors were 400-dimensional.

In order to maximize the discrimination between the different dialect classes in the i-vector space, we combine Linear Discriminant Analysis (LDA) and Within Class Co-variance Normalization~\cite{dehak2011language}. This intersession compensation method has been used with both SVM~\cite{dehak2011language} and cosine scoring~\cite{bahari2014non}.
did.tex
\label{sec:did}
\subsection{Choosing the Best Classifier}

We first studied the best classification approach for the DID task from a set of two generative models: n-gram language model~\cite{Collins} and Naive Bayes~\cite{Ng}, and two discriminative classifiers: linear SVM~\cite{drucker1999support} and Maximum Entropy~\cite{Nigam}.  We measured the performance of each model on the DID task, in the word or lexical-based utterance vector space, which is constructed using the approach mentioned in section~\ref{sec:dialectfeatures}, using identity scaling function $A$, and performing no dimensionality reduction. Hence, the dimensionality of an utterance vector, $\vv{u}$, is the same as the size of the lexicon, which in our case was $55$k. Results can be seen in table~\ref{tab:classifiers}.  As the linear SVM performs the best, it is our choice of classifier for the rest of the experiments.

\begin{table}
\centering
\ra{1.3}
\resizebox{70mm}{13mm}{%
\begin{tabular}{@{}llllll@{}} \toprule Model & ACC & PRC & RCL\\ \midrule
n-gram Language Model& 40.4\%  & 40.2\% & 41.3\% \\
Naive Bayes& 37.9\%  & 37.5\% &  50.2\% \\
Max Ent& 40\%  & 40\% &  40.6\% \\
SVM & {\bf 45.2}\%   & 44.8\%  & 45.4\% \\
\bottomrule \end{tabular}%
}
\caption{\textit{Performance of different classifiers using lexical features, with lexicon size of 55K. ACC, PRC and RCL correspond to accuracy, precision and recall on the test set.}}\label{tab:classifiers}
\end{table}

\subsection{Feature Selection Study}
\label{fab}
 \begin{table*}\centering
 \ra{1.5}
 \resizebox{170mm}{19mm}{%
 \begin{tabular}{@{}rrrrcrrrcrrrcrrr@{}}\toprule
 & \multicolumn{3}{c}{$d = 300$} & \phantom{abc}& \multicolumn{3}{c}{$d = 600$} &
 \phantom{abc} & \multicolumn{3}{c}{$d = 1200$} & \phantom{abc}& \multicolumn{3}{c}{$d = 1600$} \\ \cmidrule{2-4} \cmidrule{6-8} \cmidrule{10-12} \cmidrule{14-16}
 & $ACC$ & $PRC$ & $RCL$ && $ACC$ & $PRC$ & $RCL$ && $ACC$ & $PRC$ & $RCL$  && $ACC$ & $PRC$ & $RCL$\\ \midrule
 $\mathbf{U_{w}^{i}}$  & 38.3 & 41.9 & 39.4 && 41.7 & 44.1 & 42.8 && \textbf{42.9} & 45.6 & 44 && 42.9 & 45 & 43.8\\
 $\mathbf{U_{w}^{tfidf}}$ & 43.3 & 42.7 & 43.5 && 44.6 & 44 & 44.9 && \textbf{45.5} & 45.1 & 45.8 && 21.9 & 20.9 & 21.9\\
 $\mathbf{U_{s}^{i}}$  & 45.2& 44.8& 45.9&& \textbf{45.8} & 45.1 & 46.5 && 45.2 & 44.7 & 45.8\\
 $\mathbf{U_{s}^{tfidf}}$& \textbf{44} & 43.9& 44.7&& 44 & 44.2 & 44.6 && 43.9 & 44 & 44.3\\
 Feature Combination & \textbf{44.8} & 44.2& 45.6&& 44.1& 43.4& 44.8 && 44.8& 44.1& 45.4\\
 \\ \bottomrule
 \end{tabular}%
 }
 \caption{\textit{Accuracy, Precision and Recall for different senone and lexical feature based Vector Spaces. $d$ is the dimensionality of the Vector Space. \textbf{Boldfaced} numbers are the best accuracy for the corresponding vector space, for a corresponding vector space dimensionality $d$. A detailed explanation of feature spaces is given in the feature selection study (section~\ref{fab})}}\label{results-1}
 \end{table*}
 
 \begin{table}[pht!]
 \centering
 \ra{1.5}
 \resizebox{58mm}{10mm}{%
 \begin{tabular}{@{}llllll@{}} \toprule Feature Space & d & ACC & PRC & RCL\\ \midrule $\mathbf{U_{iVec}^{bnf}}$ & 400 &55.3 & 61 & 55.9 \\
  $\mathbf{U_{iVec+LDA+WCNN}^{bnf}}$ &4 & 58.5 & 62.3 & 58.9 \\
  $\mathbf{U_{iVec+LDA+WCNN+LNORM}^{bnf}}$&4 & 58.7 & 61.9 & 59.3 \\
  $\mathbf{U_{iVec+LDA+WCNN}^{bnf} + U_{s}^{i}(600d)}$&604 & 59.2 & 62.7 & 59.5\\
 \bottomrule \end{tabular}%
 }
 \caption{\textit{Accuracy, Precision and Recall for different i-vector based feature spaces. $d$ refers to the dimensionality of the Vector Space. A detailed explanation of feature spaces is given in the feature selection study (section~\ref{fab}).}}\label{tab:results2}
 \end{table}

Here we examine the dialect information captured by the three utterance VSMs explained in section~\ref{sec:dialectfeatures}. We also explore the concatenation of the utterance vector representations, and report the results in Tables~\ref{results-1} and~\ref{tab:results2}. Details about the terms in the results table are given below:
\begin{itemize}
  \itemsep0em
  \item $\mathbf{U_{w}^{i}}$: Refers to the utterance VSM in which each utterance is represented by a vector given by equation~\ref{wv}, where $A$ is chosen to be the identity function. The bases of the vectors are the words in the lexicon. SVD is used to reduce the dimensionality of the utterance Vector Space from $55$k originally, to $300$, $600$, $1200$, $1600$ at which point increase the gain in the classification performance tends to saturate.
  \item $\mathbf{U_{w}^{tf.idf}}$: Same as the previous Utterance VSM, except that $A$ is chosen via $tf.idf$~\cite{Ramos} instead of identity function, which gives us significant improvement in accuracy over the previous vector space.
  \item $\mathbf{U_{s}^{i}}$: Refers to the utterance VSM in which each utterance is represented by a vector given by equation~\ref{sv}, where $A$ is chosen to be the identity function. Utterance vector bases corresponds to senones. Just as with the word-based utterance VSM, we use SVD on the vector space and experiment with different dimensions. The utterance Vector Space constructed using senone features is more discriminative than word-based Vector space.
  \item $\mathbf{U_{s}^{tf.idf}}$: Refers to the same vector space as the previous one, except that $A$ is chosen to be the $tf.idf$ function. $tf.idf$, does not help in the case of senone features.
  \item \textbf{Feature Combination}: Combining the best senone-based utterance VSM, $U_{s}^{i}(600d)$, and the best lexical-based utterance VSM, $U_{w}^{tfidf}(1200d)$, to form a concatenated feature vector representation. SVD is performed to reduce the dimensions of the feature space. Feature combination does not help and hence we conclude that the two vector spaces are capturing similar information.
  \item $\mathbf{U_{iVec}^{bnf}}$: Refers to the utterance VSM, where each utterance is represented by a compact $400d$ i-vector (section~\ref{sec:ivsm}). We use the bottleneck features to train the UBM, which is then used to extract the i-vector. We do not experiment with different i-vector dimensions and take the best dimension reported in~\cite{dehak2011language} for the LID task. The i-vector feature space is significantly more discriminative than previously defined feature spaces.
  \item $\mathbf{U_{iVec+LDA+WCNN}^{bnf}}$: Reducing the dimensionality of the i-vector space using LDA and performing WCNN has been reported to do well in LID tasks~\cite{dehak2011language} and we use the same technique and see a significant improvement in the DID results.
  \item $\mathbf{U_{iVec+LDA+WCNN}^{bnf} + U_{s}^{i}(600d)}$: Finally we concatenate the best senone-based VSM with the best i-vector-based VSM, to form a concatenated vector representation for each utterance and see slight improvements in the results. As the lexical and senone-based representations encode the same information about the dialect, we do not experiment with concatenated lexical and i-vector representations.
\end{itemize}
  %There are only negligible differences in average across different Arabic dialects. However, average phoneme duration is a more useful feature, when English and Arabic (both MSA, and dialectal Arabic) are compared (as shown in the above link). More interestingly, phoneme duration does differ between MSA and dialectal Arabic. One plausible explanation for this is that dialectal Arabic very often occurs in debates and conversational speech, while MSA often appears in news broadcast and reports, where different patterns in the speech with less hesitation or unintelligible speech is expected. This has been verified by studying LDC GALE Arabic corpus~\cite{gale_arabic} which has both reports and conversational segments for the same program, where conversational speech is mainly dialectal data and reports are MSA.

  %We also investigated invididual phoneme duration features, in which a feature vector could potentially look like: g\_10 T\_20 S\_200 a\_30 A\_50.  We binned the durations using 40 bins from 0--400~ms with a 10~ms window, thus avoiding a long tail of phonemes with longer durations, resulting in 1440 unique phoneme features in total (40 per phone for 36 phones).  We have also investigated using unlabelled phone durations (so the previous feature vector will look like: \_10 \_20 \_200 \_30 \_50), resulting in 40 unique features. Classification performance using phoneme duration as features can be seen in table~\ref{tab:phonedur}.

  %\FloatBarrier
  %\begin{table}[pht!]
  %\centering
  %\ra{1.3}
  %\resizebox{60mm}{8mm}{%
  %\begin{tabular}{@{}lllll@{}} \toprule Features & ACC & PRC & RCL\\ \midrule
  %Phoneme Duration & 37.9\%  & 37.6\% & 36.2\% \\
  %\bottomrule \end{tabular}%
  %}
  %\caption{Performance of SVM classifier on phoneme duration features. ACC, PRC and RCL correspond to accuracy, precision and recall on the test set}\label{tab:phonedur}
  %\end{table}
  %\FloatBarrier

\subsection{One Vs All classification (Sanity Check)}
We  constructed a senone-based utterance VSM (section~\ref{sec:svsm}) based on 20 hours of speech; 10 hours English (which we got from~\cite{glass2004analysis}) and 10 hours Arabic (randomly sampled from our training data, section~\ref{sec:data}). Binary classification (English vs Arabic) using an SVM classifier, was then performed and it yielded 100\% accuracy on the 1.5 hour test set. The reason to choose the senone-based feature space and not the i-vector-based feature space for classification is to avoid channel mismatch, as the English data came from a different source domain. We did a similar experiment to classify MSA versus all dialectal Arabic and again obtained 100\% classification accuracy.

\subsection{System Output Combination}
We fused the scores of the best senone system and the SVM-based i-vector system. In the fusion steps, the original scores of each system were normalized and combined using the same fusion weights for both systems. This approach yielded a final accuracy of 60.2\%, which is the best performance we achieved. One explanation for this gain is that the error patterns for the two feature spaces are quite different, and we were able to confirm that by analyzing the confusion matrix for each system.


discussion.tex
\label{cs}

We infer from the confusion matrix in Table~\ref{tab:cm} that GLF and LAV are the most confusable dialect pair. We believe that this is related to the greater lexical similarity between these two dialects (see Table~\ref{tab:lexical_difference}). Note, the confusion matrix is from the best DID system.
We borrowed Table~\ref{tab:expansion2} from previous work~\cite{samn2015crowdsource} on the test set, which shows the amount of time the same speakers switch between dialect and another (mainly MSA, and their own native dialect). For example, in the second row of Table~\ref{tab:expansion2}, there are 200 samples from potential Gulf speakers. After manually labeling, there were 106(53\%) segments labeled as MSA, 82(41\%) validated as GLF, 8(4\%) as LAV, and 4 segments were not identified with enough confidence to be considered. This means more than 50\% of the random GLF speakers data is in-fact MSA speech segments. This is strong evidence for the amount of code-switching between one dialect and MSA from the same speaker.



% We also note that MSA significantly outperforms the rest of dialectal Arabic in classification.  We believe that this is because the acoustic training data mainly comprises MSA data, which can be considered to tune lexical and phonetic features to MSA.

%Speaker linking information in the test set has been calculated using~\cite{meignier2010lium}. In our experiments, it is assumed that each segment (belonging to one speaker) is one dialect and speakers will not switch between dialects without reasonable silence (chosen to be 300ms), which seems to be a rather naive assumption. The authors have plans to visit a more fine-grained dialect diarization in the future~\cite{Wu2015} %This assumption has been accepted by the crowdsource community\footnote{It can be argued that in dialectal Arabic speakers can switch between MSA and dialects without silence. The authors have plans to visit a more fine-grained dialect diarization in the future.}.

\begin{table}
\centering
\ra{1.3}
\resizebox{80mm}{16mm}{%
\begin{tabular}{@{}lllllllll@{}} \toprule  & EGY & GLF & LAV & MSA & NOR & Total Truth & PRC\\ \midrule
EGY & 221  & 15 & 57 & 13 & 9 & 315 & 50.3\% \\
GLF & 45 & 121 & 82 & 12 & 5 & 265 & 55.8\% \\
LAV & 74 & 43 & 199 & 18 & 14 & 348 & 46.9\% \\
MSA & 19 & 17 & 20 & 218 & 5 & 279 & 77\% \\
NOR & 80 & 21 & 66 & 22 & 166 & 355 & 83.4\% \\
\#class & 439 & 217 & 424 & 283 & 199 \\
RCL & 70.2\% & 45.7\% & 57.2\% & 78.1\% & 46.8\% \\
\bottomrule \end{tabular}%
}
\caption{\textit{Confusion Matrix for DID.}}\label{tab:cm}
\end{table}


%As shown in Table~\ref{tab:cm}, we see two dialects LAV and GLF overlap more with MSA compared to NOR or EGY. The crowdsourcing experiment~\cite{samn2015crowdsource} also provided manual annotations of 200 speech segments from each dialect as well as MSA. The input comprises speech segments from speakers with known dialect, and the judges chose freely which dialect each speech segment belongs to. For example, there are 200 samples from potential Gulf speakers. After labeling them, there are 106 segments labeled to be MSA, 82 validated as GLF, and 8 LAV, and 4 segments were not identified with enough confidence to be considered. The results are shown in Table~\ref{tab:expansion2}, where it can be observed that code-switching happens most often between GLF/MSA and LAV/MSA. It is very interesting to observe that potential GLF speakers in this sample speak MSA more than 50\% of the time, and for LAV speakers, nearly 40\% of their speech is in MSA. This summary could explain that the overlap for MSA, GLF and LAV in Table~\ref{tab:cm} mainly occurs in code-switching rather than just an error in dialect identification.

\FloatBarrier
\begin{table}[H]
\centering
\ra{1.3}
\resizebox{50mm}{10.5mm}{%
\begin{tabular}{@{}lllllll@{}} \toprule Expected Dialect & EGY & GLF & LAV & NOR & MSA \\ \midrule
EGY & 65\% & & & & 32\% \\
GLF & & 41\% & 4\% &  & 53\%  \\
LAV & 1\% & 1\% & 53\% &  & 39\%  \\
NOR & 1\% &  &  & 69\% &  28\% \\
\bottomrule \end{tabular}%
}
\caption{\textit{Expected dialect of each speech segment from particular dialectal speakers.}}\label{tab:expansion2}
\end{table}

featureab.tex
\label{sec:fab}

testflow.tex
%% testflow.tex 
%% V1.1
%% 2007/01/10
%% Copyright (c) 2002-2007 by Michael Shell and The LaTeX3 Project
%% See: http://www.michaelshell.org/
%% for current contact information.
%% 
%% A test page based on Rainer Schoepf's testpage.tex which is part of the
%% LaTeX2e system with added modifications to add tests to verify PS/PDF
%% output by Michael Shell.
%%
%% For complete usage documentation, see the file testflow_doc.pdf
%%
%% Support sites:
%% http://www.michaelshell.org/tex/testflow/
%% http://www.ctan.org/tex-archive/macros/latex/contrib/IEEEtran/testflow/
%%
%%*************************************************************************
%% Legal Notice:
%% This code is offered as-is without any warranty either expressed or
%% implied; without even the implied warranty of MERCHANTABILITY or
%% FITNESS FOR A PARTICULAR PURPOSE! 
%% User assumes all risk.
%% In no event shall IEEE or any contributor to this code be liable for
%% any damages or losses, including, but not limited to, incidental,
%% consequential, or any other damages, resulting from the use or misuse
%% of any information contained here.
%%
%% All comments are the opinions of their respective authors and are not
%% necessarily endorsed by the IEEE.
%%
%% This work is distributed under the LaTeX Project Public License (LPPL)
%% ( http://www.latex-project.org/ ) version 1.3, and may be freely used,
%% distributed and modified. A copy of the LPPL, version 1.3, is included
%% in the base LaTeX documentation of all distributions of LaTeX released
%% 2003/12/01 or later.
%% Retain all contribution notices and credits.
%% ** Modified files should be clearly indicated as such, including  **
%% ** renaming them and changing author support contact information. **
%%
%% File list of work: testflow.tex, testflow_ctl_A4.ps, testflow_ctl_A4.pdf,
%%                    testflow_ctl_LTR.ps, testflow_ctl_LTR.pdf, 
%%                    testflow_doc.pdf
%%*************************************************************************
%
% Changelog:
%
% 1.1 (2007/01/10)
%  1. new duplex test
%  2. tests for Palladio hinting
%
% 1.0 (2002/04/08) Initial release




% ***** To prevent testflow from prompting user on the console, *****
% ***** uncomment one each of the two pairs below:              *****
%
% *** paper size (choose one) ***
%\def\papertype{letterpaper}
%\def\papertype{a4paper}
%
% *** duplex page test (choose one) ***
%\def\makeduplexpage{yes}
%\def\makeduplexpage{no}




\def\docversion{1.1}
\def\docbuildcode{0000}

\newif\ifmakeduplextest
\makeduplextestfalse
\newif\ifneeduserresponse
\needuserresponsetrue

\def\useryesstring{yes}
\def\usernostring{no}
\def\userystring{y}
\def\usernstring{n}

\ifx\papertype\TESTFLOWundefined
\typeout{}
\typeout{ This is the testflow.tex (V\docversion) test page generator.}
\typeout{ See the testflow_doc.pdf file for full documentation.}

\typeout{}
\typeout{**********************************************************}
\typeout{*  Enter paper type in form of document class option,%
\space\space\space\space\space*}
\typeout{*  e.g., `a4paper' or `letterpaper' (without the quotes).\space*}
\typein[\papertype]{******************************************%
****************}
\fi

\ifx\makeduplexpage\TESTFLOWundefined
\loop
\typeout{}
\typeout{**************************************************************}
\typeout{*  Do you wish to make a second page to test duplex printing?%
\space*}
\typeout{*  Enter `yes' or `no' (without the quotes).%
\space\space\space\space\space\space\space\space\space\space\space\space\space
\space\space\space\space\space*}
\typein[\makeduplexpage]{******************************************%
********************}
\ifx\makeduplexpage\useryesstring\makeduplextesttrue\needuserresponsefalse\fi
\ifx\makeduplexpage\usernostring\makeduplextestfalse\needuserresponsefalse\fi
\ifx\makeduplexpage\userystring\makeduplextesttrue\needuserresponsefalse\fi
\ifx\makeduplexpage\usernstring\makeduplextestfalse\needuserresponsefalse\fi
\ifneeduserresponse
\repeat
\else
\ifx\makeduplexpage\useryesstring\makeduplextesttrue\fi
\ifx\makeduplexpage\userystring\makeduplextesttrue\fi
\fi



% declare symbols for lasy test
\DeclareSymbolFont{lasy}{U}{lasy}{m}{n}
\SetSymbolFont{lasy}{bold}{U}{lasy}{b}{n}
\DeclareMathSymbol\lasymho     {\mathord}{lasy}{"30}
\DeclareMathSymbol\lasyJoin    {\mathrel}{lasy}{"31}
\DeclareMathSymbol\lasyBox     {\mathord}{lasy}{"32}
\DeclareMathSymbol\lasyDiamond {\mathord}{lasy}{"33}
\DeclareMathSymbol\lasyleadsto {\mathrel}{lasy}{"3B}
\DeclareMathSymbol\lasysqsubset{\mathrel}{lasy}{"3C}
\DeclareMathSymbol\lasysqsupset{\mathrel}{lasy}{"3D}
\DeclareMathSymbol\lasylhd     {\mathbin}{lasy}{"01}
\DeclareMathSymbol\lasyunlhd   {\mathbin}{lasy}{"02}
\DeclareMathSymbol\lasyrhd     {\mathbin}{lasy}{"03}
\DeclareMathSymbol\lasyunrhd   {\mathbin}{lasy}{"04}



\documentclass[\papertype, 10pt]{article}

\typeout{-- Option summary:}
\typeout{-- Paper size: \papertype}
\ifmakeduplextest
\typeout{-- With duplex test page}
\else
\typeout{-- No duplex test page}
\fi


% enable Times fonts
\renewcommand{\sfdefault}{phv}
\renewcommand{\rmdefault}{ptm}
\renewcommand{\ttdefault}{pcr}

\def\fontsubfuzz{.9pt}

\def\tenptsize{\fontsize{10}{12pt}\selectfont}
\def\nineptsize{\fontsize{9}{11pt}\selectfont}
\def\eightptsize{\fontsize{8}{10pt}\selectfont}
\def\sevenptsize{\fontsize{7}{9pt}\selectfont}
\def\sixptsize{\fontsize{6}{8pt}\selectfont}
\def\fiveptsize{\fontsize{5}{7pt}\selectfont}


\pagestyle{empty}
\thispagestyle{empty}

\nofiles

\setlength{\oddsidemargin}{0pt}
\setlength{\evensidemargin}{0pt}
\setlength{\marginparwidth}{1in}
\setlength{\marginparsep}{0pt}

\setlength{\topmargin}{0pt}
\setlength{\headheight}{0pt}
\setlength{\headsep}{0pt}
\setlength{\topskip}{0pt}

\setlength{\footskip}{0pt}

\setlength{\textwidth}{\paperwidth}
\addtolength{\textwidth}{-2in}
\setlength{\textheight}{\paperheight}
\addtolength{\textheight}{-2in}

\setlength{\parindent}{0pt}
\setlength{\parskip}{0pt}

\setlength{\unitlength}{1sp}


\newcounter{textheight}
\newcounter{textwidth}

\setcounter{textheight}{\textheight}
\setcounter{textwidth}{\textwidth}



\newlength{\help}

\newcounter{help}

\newcounter{in}
\newcounter{halfin}
\newcounter{fifthin}
\newcounter{tenthin}
\newcounter{twtin}

\newcounter{mm}
\newcounter{tmm}
\newcounter{frmm}
\newcounter{fvmm}
\newcounter{tenmm}

\newcounter{foo}

\newcounter{x}
\newcounter{y}

\newcommand{\addtox}{\addtocounter{x}}
\newcommand{\addtoy}{\addtocounter{y}}

\newcommand{\putxy}{\put(\value{x},\value{y})}
\newcommand{\multiputxy}{\multiput(\value{x},\value{y})}

\newcommand{\sethelpcounter}[2]{%
   \setlength{\help}{#2}\setcounter{#1}{\help}}


\newenvironment{testflowpage}{\setlength{\help}{1in}
\setcounter{in}{\help}

\setlength{\help}{0.5in}
\setcounter{halfin}{\help}

\setlength{\help}{0.2in}
\setcounter{fifthin}{\help}

\setlength{\help}{0.1in}
\setcounter{tenthin}{\help}

\setlength{\help}{0.05in}
\setcounter{twtin}{\help}


\setlength{\help}{1mm}
\setcounter{mm}{\help}

\setlength{\help}{2mm}
\setcounter{tmm}{\help}

\setlength{\help}{4mm}
\setcounter{frmm}{\help}

\setlength{\help}{5mm}
\setcounter{fvmm}{\help}

\setlength{\help}{10mm}
\setcounter{tenmm}{\help}


\begin{picture}(0,0)
\scriptsize

\put(0,-\value{textheight}){%
     \framebox(\value{textwidth},\value{textheight}){}}

% left mm ruler
\setcounter{x}{0}
\sethelpcounter{y}{-0.45\textheight}
\putxy{\line(-1,0){\value{in}}}


\addtox{-\value{fvmm}}
\addtoy{-\value{tmm}}
\multiputxy(-\value{fvmm},0){5}{\line(0,1){\value{frmm}}}


\addtoy{\value{mm}}
\multiput(-\value{mm},\value{y})(-\value{mm},0){25}%
          {\line(0,1){\value{tmm}}}

\addtoy{\value{frmm}}
\setcounter{foo}{5}
\multiput(-\value{fvmm},\value{y})(-\value{fvmm},0){4}{%
    \makebox(0,0){\arabic{foo}}\addtocounter{foo}{5}}


% left in ruler
\setcounter{x}{0}
\sethelpcounter{y}{-0.55\textheight}
\putxy{\line(-1,0){\value{in}}}

\addtox{-\value{tenthin}}
\addtoy{-\value{tenthin}}
\multiputxy(-\value{tenthin},0){10}{\line(0,1){\value{fifthin}}}

\addtox{\value{twtin}}
\addtoy{\value{twtin}}
\multiputxy(-\value{tenthin},0){10}{\line(0,1){\value{tenthin}}}

\setcounter{foo}{1}

\addtox{-\value{twtin}}
\addtoy{\value{tenthin}}
\addtoy{\value{tenthin}}
\multiputxy(-\value{tenthin},0){9}{%
    \makebox(0,0){\arabic{foo}}\addtocounter{foo}{1}}

% right mm ruler
\sethelpcounter{x}{\textwidth}
\sethelpcounter{y}{-0.45\textheight}
\putxy{\line(1,0){\value{in}}}

\addtox{\value{mm}}
\addtoy{-\value{mm}}
\multiputxy(\value{mm},0){25}{\line(0,1){\value{tmm}}}

\addtox{\value{frmm}}
\addtoy{-\value{mm}}
\multiputxy(\value{fvmm},0){5}{\line(0,1){\value{frmm}}}

\addtoy{\value{fvmm}}
\setcounter{foo}{5}
\multiputxy(\value{fvmm},0){4}{%
    \makebox(0,0){\arabic{foo}}\addtocounter{foo}{5}}

% right in ruler
\sethelpcounter{x}{\textwidth}
\sethelpcounter{y}{-0.55\textheight}
\putxy{\line(1,0){\value{in}}}

\addtox{\value{tenthin}}
\addtoy{-\value{tenthin}}
\multiputxy(\value{tenthin},0){10}{%
   \line(0,1){\value{fifthin}}}

\addtox{-\value{twtin}}
\addtoy{\value{twtin}}
\multiputxy(\value{tenthin},0){10}{%
   \line(0,1){\value{tenthin}}}

\setcounter{foo}{1}
\addtox{\value{twtin}}
\addtoy{\value{tenthin}}
\addtoy{\value{tenthin}}
\multiputxy(\value{tenthin},0){9}{%
    \makebox(0,0){\arabic{foo}}\addtocounter{foo}{1}}


% top mm ruler
\sethelpcounter{x}{0.45\textwidth}
\setcounter{y}{0}
\putxy{\line(0,1){\value{in}}}

\addtox{-\value{tmm}}
\addtoy{\value{fvmm}}
\multiputxy(0,\value{fvmm}){5}{\line(1,0){\value{frmm}}}

\addtox{\value{mm}}
\addtoy{-\value{frmm}}
\multiputxy(0,\value{mm}){25}{\line(1,0){\value{tmm}}}

\setcounter{foo}{5}
\addtox{-\value{tmm}}
\addtoy{-\value{mm}}
\addtoy{\value{fvmm}}
\multiputxy(0,\value{fvmm}){4}{%
  \makebox(0,0){\arabic{foo}\rule{5pt}{0pt}}\addtocounter{foo}{5}}

% top in ruler
\sethelpcounter{x}{0.55\textwidth}
\setcounter{y}{0}
\putxy{\line(0,1){\value{in}}}

\addtox{-\value{tenthin}}
\addtoy{\value{tenthin}}
\multiputxy(0,\value{tenthin}){10}{\line(1,0){\value{fifthin}}}

\addtox{\value{twtin}}
\addtoy{-\value{twtin}}
\multiputxy(0,\value{tenthin}){10}{\line(1,0){\value{tenthin}}}

\setcounter{foo}{1}
\addtox{-\value{tenthin}}
\addtoy{\value{twtin}}
\multiputxy(0,\value{tenthin}){9}{%
   \makebox(0,0){\arabic{foo}}\addtocounter{foo}{1}}

% bottom mm ruler
\sethelpcounter{x}{0.45\textwidth}
\setcounter{y}{-\textheight}
\putxy{\line(0,-1){\value{in}}}

\addtox{-\value{tmm}}
\addtoy{-\value{fvmm}}
\multiputxy(0,-\value{fvmm}){5}{\line(1,0){\value{frmm}}}

\addtox{\value{mm}}
\addtoy{\value{frmm}}
\multiputxy(0,-\value{mm}){25}{\line(1,0){\value{tmm}}}

\setcounter{foo}{5}
\addtox{-\value{tmm}}
\addtoy{\value{mm}}
\addtoy{-\value{fvmm}}
\multiputxy(0,-\value{fvmm}){4}{%
   \makebox(0,0){\arabic{foo}\rule{5pt}{0pt}}\addtocounter{foo}{5}}


% bottom in ruler
\sethelpcounter{x}{0.55\textwidth}
\setcounter{y}{-\textheight}
\putxy{\line(0,-1){\value{in}}}

\addtox{-\value{tenthin}}
\addtoy{-\value{tenthin}}
\multiputxy(0,-\value{tenthin}){10}{\line(1,0){\value{fifthin}}}

\addtox{\value{twtin}}
\addtoy{\value{twtin}}
\multiputxy(0,-\value{tenthin}){10}{\line(1,0){\value{tenthin}}}

\setcounter{foo}{1}
\addtox{-\value{tenthin}}
\addtoy{-\value{twtin}}
\multiputxy(0,-\value{tenthin}){9}{%
   \makebox(0,0){\arabic{foo}}\addtocounter{foo}{1}}


\end{picture}

\setlength{\help}{\textwidth}
\addtolength{\help}{-1.25in}

\vspace*{\baselineskip}
\mbox{}\hfill
\begin{minipage}{\help}\relax}
% end of testpage environment
{\end{minipage}\hfill\mbox{}\vfill\mbox{}}


\begin{document}


\begin{testflowpage}

\centerline{\Huge A Test for \LaTeX\ PS/PDF Printing}
\vspace{0.75\baselineskip}
\centerline{\large Version \docversion\hspace{0.2in} Build Code: \docbuildcode}
\vspace{0.75\baselineskip}
\centerline{\large See the testflow home page for the latest news and FAQ:}
\centerline{\texttt{\footnotesize http://www.michaelshell.org/tex/testflow/}} 
\vspace{1.5\baselineskip}
\centerline{\large\bfseries\scshape Notes}
\vspace{0.5\baselineskip}

Document paper type selected under \LaTeX: \mbox{\ttfamily\papertype}\\
Depends only on the base article.cls --- no other external packages are loaded.\\
The main text font is Times Roman, the math font is Computer Modern.\\
Imperial (0.1in) and metric (mm) rulers are provided to measure centering.\\
The frame on this page should be centered on the paper and 1in (25.4mm) from the edges.\\
To maintain accurate dimensions, do not scale page when printing.\\
(i.e., deselect any ``fit to page" or ``shrink/expand page" options.)\\
For complete usage information, read the \texttt{testflow\_doc.pdf} file.

\vspace{1.0\baselineskip}
\centerline{\large\bfseries\scshape Palladio Font Hinting Test}
\vspace{0.5\baselineskip}

\centerline{\renewcommand{\rmdefault}{ppl}\rmfamily\bfseries This is in bold Palatino/Palladio.}


\vspace{1.0\baselineskip}
\centerline{\large\bfseries\scshape Ligature Test}
\vspace{0.5\baselineskip}

\makebox[0.85in][l]{\textbf{Ligatures}} ---~~~The office was affected by the five flawed mufflers.

\makebox[0.85in][l]{\textbf{No Ligatures}} ---~~~The of\/f\/ice was a\/f\/fected by the f\/ive f\/lawed muf\/f\/lers.


\vspace{1.0\baselineskip}
\centerline{\large\bfseries\scshape Math Tests}
\vspace{0.5\baselineskip}

\centerline{\normalsize\bfseries Large Delimiter and Operator Test}
\begin{center}
\begin{math}
I = \left[ 
    \begin{array}{cc}
    1 & 0\\
    0 & 1
   \end{array}
   \right]\quad \mbox{and}   
\quad\Bigg(\sum\limits_{i=0}^{2} 2^i = 7 \Bigg)
\end{math}
\end{center}

\vspace{0.25\baselineskip}

\centerline{\normalsize\bfseries Minus Sign Test}
\begin{center}
\begin{math}
\mbox{If } a = 4\mbox{, then: } 2^{-a + 7} - 2^{a-3} = 2^{3} - 2^{1} = 6
\end{math}
\end{center}

\vspace{0.1\baselineskip}
\centerline{\normalsize\bfseries Problem Character, Times Roman and Font Kerning Tests}
\vspace{0.5\baselineskip}
% here is how we would directly access them within cmmi10
%{\fontencoding{OML}\fontfamily{cmm}\fontshape{it}\selectfont
%\symbol{0}\hspace{1ex}\symbol{9}\hspace{1ex}\symbol{10}\hspace{1ex}\symbol{13}
%\hspace{1ex}\symbol{32}\hspace{1ex}\symbol{127}}
\centerline{\normalfont Math italic glyphs: $\mathnormal{\Gamma,~\Psi,~\Omega,~\gamma,~\psi,~\mbox{\t{\mbox{}\mbox{}}}}$}
\vspace{0.25\baselineskip}
\centerline{Large Times Roman italic: {\LARGE \textit{z}}\qquad GS kerning test: {\footnotesize The ``Problematic" little quotes.}} 

\vspace{0.25\baselineskip}
{\footnotesize\textbf{Note:} The math italic glyphs are in the control character positions 
0, 9, 10, 13, 32 and 127.\hfill\\
i.e., \texttt{\string\Gamma}, \texttt{\string\Psi}, \texttt{\string\Omega}, 
\texttt{\string\gamma}, \texttt{\string\psi}, and the \texttt{\string\t\{xx\}} tie-after accent, respectively.} 


\vspace{1.0\baselineskip}
\centerline{\large\bfseries\scshape Picture and Lasy Fonts Test}
\vspace{0.5\baselineskip}

\def\lasyglyphs{\lasymho, \lasyJoin, \lasyBox,
\lasyDiamond, \lasyleadsto, \lasysqsubset, \lasysqsupset, 
\lasylhd, \lasyunlhd, \lasyrhd, \lasyunrhd}

\setlength{\unitlength}{1cm}

\begin{center}

\begin{picture}(12,2.4)
\qbezier(0,2.4)(2.25,2.8)(4.5,2.4)
\put(1,1.8){\oval(1,1)[tl]}
\put(3.5,1.8){\oval(1,1)[tr]}
\put(2.25,1.9){\oval(4,1)[t]}
\put(.2,1.5){\oval(1,2.2)[l]}
\put(1.5,1.5){\circle*{0.1}}
\put(1.5,1.5){\circle{1.2}}
\put(1.5,1.5){\vector(0,1){0.6}}
\put(1.5,1.5){\vector(1,0){0.6}}
\put(1.5,1.5){\vector(0,-1){0.6}}
\put(1.5,1.5){\vector(-1,0){0.6}}
\thicklines
\qbezier(0,0.6)(2.25,0.2)(4.5,0.6)
\put(0,0){\vector(1,0){11.5}}
\put(3,1.5){\circle*{0.1}}
\put(3,1.5){\circle{1.2}}
\put(3,1.5){\vector(0,1){0.6}}
\put(3,1.5){\vector(1,0){0.6}}
\put(3,1.5){\vector(0,-1){0.6}}
\put(3,1.5){\vector(-1,0){0.6}}
\put(1,1.2){\oval(1,1)[bl]}
\put(3.5,1.2){\oval(1,1)[br]}
\put(2.25,1.1){\oval(4,1)[b]}
\put(4.3,1.5){\oval(1,2.2)[r]}
\put(4.9,2.4){\makebox[0.75in][r]{10pt bold:\hspace{0.1in}}\tenptsize\boldmath$\lasyglyphs$}
\put(4.9,2.0){\makebox[0.75in][r]{10pt:\hspace{0.1in}}\tenptsize$\lasyglyphs$}
\put(4.9,1.65){\makebox[0.75in][r]{\nineptsize9pt:\hspace{0.1in}}\nineptsize$\lasyglyphs$}
\put(4.9,1.35){\makebox[0.75in][r]{\eightptsize8pt:\hspace{0.1in}}\eightptsize$\lasyglyphs$}
\put(4.9,1.1){\makebox[0.75in][r]{\sevenptsize7pt:\hspace{0.1in}}\sevenptsize$\lasyglyphs$}
\put(4.9,0.875){\makebox[0.75in][r]{\sixptsize6pt:\hspace{0.1in}}\sixptsize$\lasyglyphs$}
\put(4.9,0.675){\makebox[0.75in][r]{\fiveptsize5pt:\hspace{0.1in}}\fiveptsize$\lasyglyphs$}
\end{picture}
\end{center}
\end{testflowpage}


% Optional duplex test page
\ifmakeduplextest
\newpage
\begin{testflowpage}
\centerline{\Huge A Test for \LaTeX\ PS/PDF Printing}
\vspace{0.75\baselineskip}
\centerline{\large Version \docversion\hspace{0.2in} Build Code: \docbuildcode}
\vspace{1.0\baselineskip}
\centerline{\large\bfseries\scshape Duplex Alignment Test Side}
\mbox{}\vfill\mbox{}
\end{testflowpage}
\fi

\end{document}


intro.tex
\label{sec:intro}

The task of Dialect Identification (DID) is a special case of the more general problem of Language Identification (LID). LID refers to the process of automatically identifying the language class for given speech segment or text document.  DID is arguably a more challenging problem than LID, since it consists of identifying the different dialects within the same language class. The importance of addressing DID can be gauged from its growing interest in the Automatic Speech Recognition (ASR) community~\cite{liu2010dialect}. A good DID system can facilitate the identification of dialectal segments from an untranscribed mixed-speech dataset. This process can help reduce the ASR word error rate (WER) for dialectal data by training ASR systems for each dialect, or by adapting the ASR models to a particular dialect.

The natural language processing (NLP) community has aggregated dialectal Arabic into five regional language groups: Egyptian (EGY), North African or Maghrebi (NOR), Gulf or Arabian Peninsula (GLF), Levantine (LAV), and Modern Standard Arabic (MSA). An objective comparison of the varieties of Arabic dialects could potentially lead to the conclusion that Arabic dialects are historically related, but not synchronically, and are mutually unintelligible languages like English and Dutch. Normal vernacular can be difficult to understand across different Arabic dialects~\cite{holes2004modern}. Arabic dialects are thus sufficiently distinctive, and it is reasonable to regard the DID task in Arabic as similar to the LID task in other languages.  Table~\ref{tab:lexical_difference} shows two phrases across the different dialects, it is clear from this example that there are lexical variations across the different dialects which motivates us to consider it.

Two broad LID approaches have been investigated in the literature:  low-level acoustic features, and high-level phonetic and lexical features. In the lexical area, words, roots, morphology, and grammars~\cite{reynolds2008lid,ambikairajah2011language} have been studied. Acoustic features such as shifted delta cepstral coefficients~\cite{dehak2011language} and prosodic features ~\cite{martinez2012ivector} using Gaussian mixture models (GMMs), i-vector representations and support vector machine (SVM) classifiers ~\cite{dehak2011language} have been shown to be effective for LID. More recent work explored the use of frame-by-frame phone posteriors (PLLRs) ~\cite{plchot2014pllr} as new features for LID. New subspace approaches based on non-negative factor analysis (NFA) for GMM weight decomposition and adaptation~\cite{bahari2014non} were also applied to both LID and DID tasks. GMM weight adaptation subspaces seem to provide complementary information to the classical i-vector framework. Finally, phoneme sequence modeling and its n-gram subspace have been studied for both Arabic DID ~\cite{soltau2011modern} and LID ~\cite{soufifar2012discriminative}.

\FloatBarrier
\begin{table}[ht]
\label{lexicalExamples}
\caption{\textit{Lexical examples in Arabic and Buckwalter format.}}
\resizebox{0.5\textwidth}{!}{%
\begin{tabular}{|l|l|l|l|l|l|}
\hline
EGY     & GLF    & LAV           & MSA      & NOR     & Translation \\ \hline
\begin{tabular}[x]{@{}c@{}} \novocalize \RL{AzAyk} \\AzAYk\end{tabular}                                                               &
\begin{tabular}[x]{@{}c@{}} \novocalize \RL{A^slwnk} \\ A\$lwnk\end{tabular}                             & \begin{tabular}[x]{@{}c@{}} \novocalize \RL{A^slwnk / kyfk} \\ kyfk / A\$lwnk \end{tabular}      & \begin{tabular}[x]{@{}c@{}} \novocalize \RL{kyf .hAlk} \\ kyf HAlk\end{tabular}                            &  \begin{tabular}[x]{@{}c@{}} \novocalize \RL{wA^s rAk} \\ wA\$ rAk\end{tabular}                          &
How are you?        \\ \hline

\begin{tabular}[x]{@{}c@{}} \novocalize \RL{Ant fen} \\Ant fyn\end{tabular}       &
\begin{tabular}[x]{@{}c@{}} \novocalize \RL{wynk} \\wynk\end{tabular}                       				  &
\begin{tabular}[x]{@{}c@{}} \novocalize \RL{wynk} \\wynk\end{tabular}                        				  &
\begin{tabular}[x]{@{}c@{}} \novocalize \RL{Ayn Ant} \\ Ayn Ant\end{tabular}           					 &
\begin{tabular}[x]{@{}c@{}} \novocalize \RL{wyn rAk} \\ wyn rAk\end{tabular}             				&
Where are you?        \\ \hline
\end{tabular}
}
\label{tab:lexical_difference}
\end{table}
\FloatBarrier

In this paper we investigate three Vector Subspace Models (VSMs) for Arabic DID based on 1) lexical, 2) phonetic, and 3) i-vectors.  We conduct a thorough feature selection study of these models to better understand their interaction.  A further contribution of this work is the release of an Arabic DID system so others can extend and improve DID performance on this task.\footnote{https://github.com/Qatar-Computing-Research-Institute/dialectID}

main_old.tex
% Template for ASRU-2015 paper; to be used with:
%          spconf.sty  - ICASSP/ICIP LaTeX style file, and
%          IEEEbib.bst - IEEE bibliography style file.
% --------------------------------------------------------------------------
\documentclass{article}
\usepackage{INTERSPEECH2016}
\usepackage{amsmath,graphicx}
\usepackage{graphicx}
\usepackage{amssymb,amsmath,bm}
\usepackage{textcomp}
\usepackage{enumitem}
\usepackage{arabtex}
\usepackage{url}
\usepackage[table,xcdraw]{xcolor}
\usepackage{placeins}
\usepackage{float}
\usepackage{esvect}
\restylefloat{table}
\usepackage{placeins}
\usepackage[export]{adjustbox}
\usepackage{color,soul}
\usepackage{fixltx2e}
\usepackage{hyperref} % For hyperlinks in the PDF
\hypersetup{
  colorlinks   = true, %Colours links
  urlcolor     = blue, % Colour for externalhyperlinks
  linkcolor    = red, % Colour of internal links
  citecolor   = blue % Colour of citations
}
\usepackage{kbordermatrix}
\usepackage{booktabs}

\usepackage{graphics}
\usepackage{graphicx}
\newcommand\scalemath[2]{\scalebox{#1}{\mbox{\ensuremath{\displaystyle #2}}}}

\newcommand{\ra}[1]{\renewcommand{\arraystretch}{#1}}
\DeclareMathOperator*{\argmin}{arg\,min}
\DeclareMathOperator*{\argmax}{arg\,max}

% Example definitions.
% --------------------
\def\x{{\mathbf x}}
\def\L{{\cal L}}

% Title.
% ------
\title{Automatic Dialect Detection in Arabic Broadcast Speech}
%
% Single address.
% ---------------
\name{{\em Ahmed Ali$^{1,2}$, Najim Dehak, Patrick Cardinal, Sameer Khurana$^{1}$, Sree Harsha Yella, Peter Bell$^2$, Steve Renals$^2$}}
\address{$^1$Qatar Computing Research Institute, HBKU, Doha, Qatar \\$^2$Centre for Speech Technology Research,School of Informatics \\
University of Edinburgh, Edinburgh EH8 9AB, UK \\
{\small \tt amali@qf.org.qa, \{peter.bell, s.renals\}@ed.ac.uk
}}
%
% For example:
% ------------
%\address{School\\
%	Department\\
%	Address}
%
% Two addresses (uncomment and modify for two-address case).
% ----------------------------------------------------------
%\twoauthors
%  {A. Author-one, B. Author-two\sthanks{Thanks to XYZ agency for funding.}}
%	{School A-B\\
%	Department A-B\\
%	Address A-B}
%  {C. Author-three, D. Author-four\sthanks{The fourth author performed the work
%	while at ...}}
%	{School C-D\\
%	Department C-D\\
%	Address C-D}
%
\begin{document}
%\ninept
%
\maketitle


\begin{abstract}
\input{abstract}
\end{abstract}

%
\begin{keywords}
Dialect Identification, Vector Space Modelling
\end{keywords}
%

\section{Introduction}
\input{intro}

\section{Vector Space Models}
\input{dialect_features}

\section{Dataset}
\input{data}

\section{Classifiers}
\input{classifiers}

\section{Experiments}
\input{did}

\section{Conclusions}
\input{conclusion}




% References should be produced using the bibtex program from suitable
% BiBTeX files (here: strings, refs, manuals). The IEEEbib.bst bibliography
% style file from IEEE produces unsorted bibliography list.
% -------------------------------------------------------------------------
\bibliographystyle{IEEEbib}
\bibliography{refs}

\end{document}

template.tex
\documentclass[a4paper]{article}

\usepackage{INTERSPEECH2016}

\usepackage{graphicx}
\usepackage{amssymb,amsmath,bm}
\usepackage{textcomp}
\usepackage{enumitem}
\usepackage{arabtex}
\usepackage{url}
\usepackage[table,xcdraw]{xcolor}
\usepackage{placeins}
\usepackage{float}
\usepackage{esvect}
\restylefloat{table}
\usepackage{placeins}
\usepackage[export]{adjustbox}
\usepackage{color,soul}
\usepackage{fixltx2e}
\usepackage{hyperref} % For hyperlinks in the PDF
\hypersetup{
  colorlinks   = true, %Colours links
  urlcolor     = black, % Colour for externalhyperlinks
  linkcolor    = black, % Colour of internal links
  citecolor   = black % Colour of citations
}
\usepackage{kbordermatrix}
\usepackage{booktabs}

\usepackage{graphics}
\newcommand\scalemath[2]{\scalebox{#1}{\mbox{\ensuremath{\displaystyle #2}}}}

\newcommand{\ra}[1]{\renewcommand{\arraystretch}{#1}}
\DeclareMathOperator*{\argmin}{arg\,min}
\DeclareMathOperator*{\argmax}{arg\,max}

% Example definitions.
% --------------------
\def\x{{\mathbf x}}
\def\L{{\cal L}}
\def\vec#1{\ensuremath{\bm{{#1}}}}
\def\mat#1{\vec{#1}}


\sloppy % better line breaks
\ninept

\title{Automatic Dialect Detection in Arabic Broadcast Speech}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% If multiple authors, uncomment and edit the lines shown below.       %%
%% Note that each line must be emphasized {\em } by itself.             %%
%% (by Stephen Martucci, author of spconf.sty).                         %%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%\makeatletter
%\def\name#1{\gdef\@name{#1\\}}
%\makeatother
%\name{{\em Firstname1 Lastname1, Firstname2 Lastname2, Firstname3 Lastname3,}\\
%      {\em Firstname4 Lastname4, Firstname5 Lastname5, Firstname6 Lastname6,
%      Firstname7 Lastname7}}
%%%%%%%%%%%%%%% End of required multiple authors changes %%%%%%%%%%%%%%%%%

\makeatletter
\def\name#1{\gdef\@name{#1\\}}
\makeatother \name{\small{{\em Ahmed Ali$^{1,5}$, Najim Dehak$^{2,3}$, Patrick Cardinal$^{4}$, Sameer Khurana$^{1}$, Sree Harsha Yella$^{2}$, James Glass$^2$, Peter Bell$^5$, Steve Renals$^5$}}}
\address{$^1$Qatar Computing Research Institute, HBKU, Doha, Qatar \\
  $^2$ MIT Computer Science and Artificial Intelligence Laboratory (CSAIL), Cambridge, MA, USA\\
  $^3$ JHU Center for Language and Speech Processing (CLSP), Baltimore, MD, USA\\
  $^4$ \'Ecole de technologie sup\'erieure, D\'epartement de G\'enie Logiciel et des TI, Montr\'eal, Canada\\
  $^5$Centre for Speech Technology Research, University of Edinburgh, UK \\
  {\small \tt \{amali,skhurana\}@qf.org.qa, najim@jhu.edu}
 %patrick.cardinal@etsmtl.ca,shyella@csail.mit.edu, galss@mit.edu,\{peter.bell, s.renals\}@ed.ac.uk}
}

%\twoauthors{Karen Sp\"{a}rck Jones.}{Department of Speech and Hearing \\
%  Brittania University, Ambridge, Voiceland \\
%  {\small \tt Karen@sh.brittania.edu} }
%  {Rose Tyler}{Department of Linguistics \\
%  University of Speechcity, Speechland \\
%  {\small \tt RTyler@ling.speech.edu} }

%
\begin{document}

  \maketitle
  %
  \begin{abstract}
  \input{abstract}
  \end{abstract}
  \noindent{\bf Index Terms}: Dialect Identification, Vector Space Modelling

  \section{Introduction}
  \input{intro}

  \section{Vector Space Models}
  \input{dialect_features}

  \section{Dataset}
  \input{data}

  %\section{Classifiers}
  %\input{classifiers}

  \section{Experiments}
  \input{did}

  \section{Discussion}
  \input{discussion}

  \section{Conclusions}
  \input{conclusion}

  \newpage
  \eightpt
  \bibliographystyle{IEEEtran}

  \bibliography{mybib}

%  \begin{thebibliography}{9}
%    \bibitem[1]{Davis80-COP}
%      S.\ B.\ Davis and P.\ Mermelstein,
%      ``Comparison of parametric representation for monosyllabic word recognition in continuously spoken sentences,''
%      \textit{IEEE Transactions on Acoustics, Speech and Signal Processing}, vol.~28, no.~4, pp.~357--366, 1980.
%    \bibitem[2]{Rabiner89-ATO}
%      L.\ R.\ Rabiner,
%      ``A tutorial on hidden Markov models and selected applications in speech recognition,''
%      \textit{Proceedings of the IEEE}, vol.~77, no.~2, pp.~257-286, 1989.
%    \bibitem[3]{Hastie09-TEO}
%      T.\ Hastie, R.\ Tibshirani, and J.\ Friedman,
%      \textit{The Elements of Statistical Learning -- Data Mining, Inference, and Prediction}.
%      New York: Springer, 2009.
%    \bibitem[4]{YourName16-XXX}
%      F.\ Lastname1, F.\ Lastname2, and F.\ Lastname3,
%      ``Title of your INTERSPEECH 2016 publication,''
%      in \textit{Interspeech 2016 -- 16\textsuperscript{th} Annual Conference of the International Speech Communication Association, September 8–12, San Francisco, California, USA, Proceedings, Proceedings}, 2016, pp.~100--104.
%  \end{thebibliography}

\end{document}

template.bbl
% Generated by IEEEtran.bst, version: 1.13 (2008/09/30)
\begin{thebibliography}{10}
\providecommand{\url}[1]{#1}
\csname url@samestyle\endcsname
\providecommand{\newblock}{\relax}
\providecommand{\bibinfo}[2]{#2}
\providecommand{\BIBentrySTDinterwordspacing}{\spaceskip=0pt\relax}
\providecommand{\BIBentryALTinterwordstretchfactor}{4}
\providecommand{\BIBentryALTinterwordspacing}{\spaceskip=\fontdimen2\font plus
\BIBentryALTinterwordstretchfactor\fontdimen3\font minus
  \fontdimen4\font\relax}
\providecommand{\BIBforeignlanguage}[2]{{%
\expandafter\ifx\csname l@#1\endcsname\relax
\typeout{** WARNING: IEEEtran.bst: No hyphenation pattern has been}%
\typeout{** loaded for the language `#1'. Using the pattern for}%
\typeout{** the default language instead.}%
\else
\language=\csname l@#1\endcsname
\fi
#2}}
\providecommand{\BIBdecl}{\relax}
\BIBdecl

\bibitem{liu2010dialect}
G.~Liu, Y.~Lei, and J.~H. Hansen, ``Dialect identification: Impact of
  differences between read versus spontaneous speech,'' \emph{EUSIPCO-2010:
  European Signal Processing Conference, Aalborg, Denmark}, 2010.

\bibitem{reynolds2008lid}
D.~A. Reynolds, W.~M. Campbell, W.~Shen, and E.~Singer, ``Automatic language
  recognition via spectral and token based approaches,'' in \emph{Springer
  Handbook of Speech Processing}, J.~Benesty, M.~M. Sondhi, and Y.~Huang,
  Eds.\hskip 1em plus 0.5em minus 0.4em\relax Springer, 2008.

\bibitem{ambikairajah2011language}
E.~Ambikairajah, H.~Li, L.~Wang, B.~Yin, and V.~Sethu, ``Language
  identification: A tutorial,'' \emph{Circuits and Systems Magazine, IEEE},
  vol.~11, no.~2, pp. 82--108.

\bibitem{zissman1996}
M.~Zissman, ``Comparison of four approaches to automatic language
  identification of telephone speech,'' \emph{IEEE Transactions on Speech and
  Audio Processing}, vol.~4, no.~1, pp. 31--44, 1996.

\bibitem{martinez2012ivector}
D.~Mart{\'\i}nez, L.~Burget, L.~Ferrer, and N.~Scheffer, ``ivector-based
  prosodic system for language identification,'' in \emph{ICASSP}, 2012, pp.
  4861--4864.

\bibitem{plchot2014pllr}
O.~Plchot, M.~Diez, M.~Soufifar, and L.~Burget, ``Pllr features in language
  recognition system for rats,'' in \emph{Fifteenth Annual Conference of the
  International Speech Communication Association}, 2014.

\bibitem{bahari2014non}
M.~H. Bahari, N.~Dehak, L.~Burget, A.~Ali, J.~Glass \emph{et~al.},
  ``Non-negative factor analysis for gmm weight adaptation,'' \emph{IEEE
  Transactions on Audio Speech and Language Processing}, 2014.

\bibitem{soltau2011modern}
H.~Soltau, L.~Mangu, and F.~Biadsy, ``From modern standard arabic to levantine
  asr: Leveraging gale for dialects,'' in \emph{ASRU}, 2011, pp. 266--271.

\bibitem{soufifar2012discriminative}
M.~Soufifar, S.~Cumani, L.~Burget, and J.~{\v{C}}ernocky, ``Discriminative
  classifiers for phonotactic language recognition with ivectors,'' in
  \emph{ICASSP}, 2012, pp. 4853--4856.

\bibitem{Hwang}
M.-y. Hwang, ``{Subphonetic Modeling for Speech Recognition},'' pp. 174--179.

\bibitem{ali2014complete}
A.~Ali, Y.~Zhang, P.~Cardinal, N.~Dahak, S.~Vogel, and J.~Glass, ``A complete
  kaldi recipe for building arabic speech recognition systems,'' in \emph{SLT},
  2014.

\bibitem{Ramos}
J.~Ramos, J.~Eden, and R.~Edu, ``{Using TF-IDF to Determine Word Relevance in
  Document Queries}.''

\bibitem{Salton1975}
G.~Salton, a.~Wong, and C.~S. Yang, ``{A Vector Space Model for Automatic
  Indexing},'' \emph{Magazine Communications of the ACM}, vol.~18, no.~11, pp.
  613--620, 1975.

\bibitem{Lowe2000}
W.~Lowe, S.~Mcdonald, W.~Lowe, and S.~Mcdonald, ``{Division of Informatics ,
  University of Edinburgh Institute for Adaptive and Neural Computation The
  Direct Route : Mediated Priming in Semantic Space by The Direct Route :
  Mediated Priming in Semantic Space},'' no. April, 2000.

\bibitem{Pado2007}
S.~Pad{\'{o}} and M.~Lapata, ``{Dependency-Based Construction of Semantic Space
  Models},'' \emph{Computational Linguistics}, vol.~33, no. December 2004, pp.
  161--199, 2007.

\bibitem{dehak2011front}
N.~Dehak, P.~Kenny, R.~Dehak, P.~Dumouchel, and P.~Ouellet, ``Front-end factor
  analysis for speaker verification,'' \emph{Audio, Speech, and Language
  Processing, IEEE Transactions on}, pp. 788--798, 2011.

\bibitem{dehak2011language}
N.~Dehak, P.~A. Torres-Carrasquillo, D.~A. Reynolds, and R.~Dehak, ``Language
  recognition via i-vectors and dimensionality reduction,'' in
  \emph{INTERSPEECH}, 2011, pp. 857--860.

\bibitem{kenny2008study}
P.~Kenny, P.~Ouellet, N.~Dehak, V.~Gupta, and P.~Dumouchel, ``A study of
  interspeaker variability in speaker verification,'' \emph{Audio, Speech, and
  Language Processing, IEEE Transactions on}, pp. 980--988, 2008.

\bibitem{ali2014qcri}
A.~Ali, Y.~Zhang, and S.~Vogel, ``{QCRI} advanced transcription ssystem
  ({QATS}),'' in \emph{SLT}, 2014.

\bibitem{samn2015crowdsource}
S.~Wray and A.~Ali, ``Crowdsource a little to label a lot: Labeling a speech
  corpus of dialectal arabic‏,'' in \emph{INTERSPEECH}, 2015.

\bibitem{Collins}
M.~Collins, ``{Language Modeling}.''

\bibitem{Ng}
A.~Ng, ``{CS229 Lecture notes Generative Learning algorithms},'' no.~0, pp.
  1--14.

\bibitem{drucker1999support}
H.~Drucker, D.~Wu, and V.~N. Vapnik, ``Support vector machines for spam
  categorization,'' \emph{Neural Networks, IEEE Transactions on}, vol.~10,
  no.~5, pp. 1048--1054, 1999.

\bibitem{Nigam}
K.~Nigam, J.~Lafferty, and A.~Mccallum, ``{Using Maximum Entropy for Text
  Classification}.''

\bibitem{meignier2010lium}
S.~Meignier and T.~Merlin, ``Lium spkdiarization: an open source toolkit for
  diarization,'' in \emph{CMU SPUD Workshop}, 2010.

\bibitem{Wu2015}
C.-h. Wu, H.-p. Shen, and C.-s. Hsu, ``{Code-Switching Event Detection by Using
  a Latent Language Space Model and the Delta-Bayesian},'' vol.~23, no.~11, pp.
  1892--1903, 2015.

\bibitem{Card2015}
P. Cardinal, N. Dehak, Y. Zhang, and J. Glass, ``Speaker Adaptation Using the I-Vector Technique for Bottleneck Features,'' in \emph{INTERSPEECH}, 2015.

\bibitem{Fred2015}
F.~Richardson, D.~Reynolds, and N.~Dehak, ``A Unified Deep Neural Network for Speaker and Language Recognition,'' in \emph{INTERSPEECH}, 2015.

\bibitem{Ysong2013}
Y.~Song, B.~Jiang, Y.~Bao, S.~Wei, and L.~R. Dai,  ``I-vector representation based on bottleneck features for language identification,'' \emph{IEEE Electronics Letters}, 2013.

\bibitem{Pavel2014}
P.~Matejka, L.~Zhang, T.~Ng, H.~S. Mallidi, O.~Glembek, J.~Ma, and B.~Zhang, ``Neural network bottleneck features for language identification,'' in Proc. of \emph{Odyssey}, 2014.

\end{thebibliography}
