



\subsection{Evaluation Metric}
In order to evaluate the model after training, we selected three metrics to comprehensively assess the model performance: the  Receiver  Operating  Characteristic (ROC) curve, the  Area  Under  Curve (AUC) value, and the Accuracy (Other metrics, see Appendix A). 

We first calculate the confusion matrix \cite{sokolova2009systematic} to more conveniently define the evaluation metrics and visualize model performance. In the confusion matrix, see Figure \ref{fig:ConfusionMatrix}, each column represents the predicted class, and each row represents the actual class. TP (True Positive) is the fraction of positive samples those got correctly predicted by the model, and TN (True Negative) is for the correctly predicted negative samples. FP (Fasle Positive) means the incorrectly classified positive samples those should be negative actually, and FN (False Negative) is for the incorrectly predicted negative samples.
\begin{figure}[htbp]
    \centering
    \includegraphics[width=0.45\textwidth,height=45mm]{figures/ConfusionMatrix.pdf}
    \caption{The Confusion Matrix}
    \label{fig:ConfusionMatrix}
\end{figure}

Accuracy is a common evaluation metric for classification problems, defined as:
\begin{equation}
Accuracy = \frac{{TP + TN}}{{TP + TN + FP + FN}}  
\end{equation}

The ROC \cite{bradley1997use} curve is a line connecting each point with the False Positive Rate (FPR) as the horizontal axis and the True Positive Rate (TPR) as the vertical axis. The definition of FPR and TPR is as follows:
\begin{equation}
FPR = \frac{{FP}}{{FP + TN}} 
\end{equation}
\begin{equation}
TPR = \frac{{TP}}{{FP + FN}}    
\end{equation}

The AUC \cite{bradley1997use} is defined as the area under the ROC curve. In general, the ROC curve is above the straight line $y = x$ and thus $AUC \in [0.5,1]$. Usually the ROC curve and the AUC value are introduced for the binary classification task. In this work, however, by transforming the label we calculate the AUC value and ROC curve in the four-class classification. The specific method is as follows:
\begin{itemize}
    \item Assuming the number of test samples is $j$, the number of classes is $k$.
    \item After training, the probability of each test sample in each class is calculated, and a probability matrix ${P_{jk}}$ is obtained.
    \item The label of each test sample is converted by One-Hot encoding ($0 \to 1000,1 \to 0100,2 \to 0010,3 \to 0001$). Each position is used to mark whether it belongs to the corresponding class. The label matrix ${L_{jk}}$ is obtained.
\end{itemize}

The label matrix ${L_{jk}}$ and probability matrix ${P_{jk}}$ are expanded by rows. Then two columns are obtained after transposition. Finally, the results of binary classification are obtained. Therefore, the final ROC curve and AUC value can be directly obtained by this method. The model results are more intuitively shown through the ROC curve. The larger the AUC value, the better the model performance.

\subsection{Results}
\subsubsection{Results and comparisons with XGBoost}
This subsection focuses on the recognition performance of cavitation detection using our method. In Tabel \ref{tab:NoDataAugmentation} we show the comparison of the cavitation detection performance between the traditional XGBoost method (without NOSW and ASFE) and XGBoost with NOSW plus ASFE. 
\input{tables/results}
\begin{figure*}
\centering
\subfigure[Binary Classification]{
\begin{minipage}[t]{0.5\linewidth}
\centering
\includegraphics[width=0.95\textwidth,height=60mm]{figures/TwoConfusionMatrix.pdf}
\end{minipage}%
}%
\subfigure[Four-class Classification]{
\begin{minipage}[t]{0.5\linewidth}
\centering
\includegraphics[width=0.95\textwidth,height=60mm]{figures/FourConfusionMatrix.pdf}
\end{minipage}%
}%
\centering
\caption{The confusion matrix of our method for binary classification of cavitation/ non-cavitation (a), and four-class classification of choked-flow-cavitation/ contant-cavitation/ incipient-cavitation/ non-cavitation (b).}
\label{fig:Two-Four COnfusion Matrix}
\end{figure*}

For binary classification, the accuracy and AUC value of our method of XGBoost+NOSW+ASFE are increased by 4.67$\%$ and 5.91$\%$, compared to the traditional XGBoost, and for four-class classification the accuracy and AUC value are increased by 11.11$\%$ and 5.39$\%$, compared to the traditional XGBoost method, respectively. These clearly demonstrated the effectiveness of the combination of ASFE and NOSW into XGBoost algorithm to identify the cavitation of flow in valve.

To show the classification results of cavitation detection in detail, we calculate the confusion matrices of our method and show in Figure \ref{fig:Two-Four COnfusion Matrix}. For binary classification, it can be seen that cavitation and no cavitation are easy within our method to be distinguished with a high accuracy. For four-class classification, we see that the constant cavitation is more easily to be recognized with higher accuracy than other cavitation-stages intensity (Cavitation choked flow and incipient cavitation). Comparatively, cavitation choked flow can be recognized with higher accuracy compared to incipient cavitation. Therefore, both choked flow cavitation and constant cavitation are cast as states in the phenomenon of cavitation hysteresis. They have more obvious cavitation characteristics. Being differently, the incipient cavitation is challenging to be identified, because in physics it is the critical state between cavitation and no cavitation. Technically another reason is that the amount of incipient cavitation data is very small. In the study of Lehmann and Young\cite{lehman1964experimental} it also shows that the end stages of cavitation can be more easily detected than incipient cavitation. From a practical perspective, it's acceptable that the incipient cavitation is recognized as the constant cavitation to give rise to process/plant alarm. With this consideration, the accuracy of the incipient cavitation in our method can reach 80$\%$.

The Receiver Operator Characteristic (ROC) curve of our method on the testing set is shown in Figure \ref{fig:Two-Four ROC}. For binary classification, the ROC area of our method is 0.981. For four-class classification, the average ROC area of our method for cavitation detection is 0.954. We see that cavitation choked flow, constant cavitation and no-cavitation can be easily identified with higher ROC curve area.
\begin{figure*}
\centering
\subfigure[Binary Classification]{
\begin{minipage}[t]{0.5\linewidth}
\centering
\includegraphics[width=0.95\textwidth,height=60mm]{figures/Two_ROC.pdf}
\end{minipage}%
}%
\subfigure[Four-class Classification]{
\begin{minipage}[t]{0.5\linewidth}
\centering
\includegraphics[width=0.95\textwidth,height=60mm]{figures/Four_ROC.pdf}
\end{minipage}%
}%
\centering
\caption{The ROC curve of our method for binary classification of cavitation/ non-cavitation (a), and four-class classification of choked-flow-cavitation/ contant-cavitation/ incipient-cavitation/ non-cavitation (b).}
\label{fig:Two-Four ROC}
\end{figure*}
\subsubsection{Parameters analysis}
From a detailed evaluation, the optimal top $k$ value and sliding-window size for the binary classification and four-class classification separately pinned down. The value of top $k$ is chosen to be integers from 5 to 10. The sliding-window size is set to be 2334720, 1556480, 1167360, 933888, 778240, 667062, 583680, 518825 and 466944, respectively. The corresponding results are shown in Table \ref{tab:DifferentKwithDataAugmentation}.
\input{tables/DifferentKWithDataAugmentation}
\begin{figure*}
\centering
\subfigure[Binary Classification]{
\begin{minipage}[t]{0.5\textwidth}
\includegraphics[width=0.9\textwidth,height=50mm]{figures/Two-OptimalK.jpg}
\end{minipage}%
}%
\subfigure[Four-class Classification]{
\begin{minipage}[t]{0.5\textwidth}
\includegraphics[width=0.9\textwidth,height=50mm]{figures/Four-OptimalK.jpg}
\end{minipage}%
}%
\caption{Effect of window size on the best accuracy under traditional XGBoost + NOSW and our method.} 
\label{fig:Two-Four With the optiamal k}
\end{figure*}

From Table \ref{tab:DifferentKwithDataAugmentation}, the performance of our method achieves the best for binary classification when $k$ is 9 and window size is 2334720. The value of mean accuracy of our method is \textbf{92.68$\%$}, 92.36$\%$, 91.84$\%$, 91.02$\%$, 92.13$\%$, 91.04$\%$, 91.23$\%$, 91.46$\%$ and 90.79$\%$ with different window size values. Therefore, our method is more stable, when the window size is 2334720. For four-class classification, the performance of our method achieves the best when $k$ is 8 or 9 and the window size is 1556480. The value of mean accuracy of our method is 89.00$\%$, \textbf{90.59$\%$}, 80.85$\%$, 87.46$\%$, 84.22$\%$, 86.31$\%$, 84.94$\%$, 86.29$\%$ and 86.14$\%$ with different window size values. Therefore, our method is more stable, when the window size is 1556480.

And we show the best accuracy of traditional XGBoost + NOSW and our method (XGBoost + NOSW + ASFE) under different window sizes. The results are shown in Figure \ref{fig:Two-Four With the optiamal k}. As shown in Figure \ref{fig:Two-Four With the optiamal k} (a), in almost all window sizes, our method achieves better performances than XGBoost + NOSW. The best result of our method is 93.56$\%$ at the window size is 2334720. The best result of XGBoost + NOSW is 92.59$\%$ at the window size is 778240. The accuracy value  of  our  method  are  increased  by 0.97$\%$, compared with the XGBoost + NOSW. As shown in Figure \ref{fig:Two-Four With the optiamal k} (b), in all window sizes, our method achieves better performances than XGBoost + NOSW. The best result of our method is 91.67$\%$ at the window size is 1556480. The best result of XGBoost + NOSW is 86.11$\%$ at the window size is 778240. The accuracy value of our method are increased  by 5.56$\%$, compared with XGBoost + NOSW. 

\subsubsection{Correlation analysis}
In order to quantitatively analyze correlation between features, we analyze the correlation of different sub-sequences of the same sample. If the correlation of different subsequences is weak, the correlation of statistical features extracted from different sub-sequences is also weak. Conversely, the correction is strong.

Figure \ref{fig:correlation} shows the results of Pearson correlation coefficients between different sub-sequences of the same sample for different cavitation states. From Figure \ref{fig:correlation}, it can be seen that the correlations between different sub-sequences are distributed between 0.40 and 0.57. This indicates that the correlation between different sub-sequences presents a moderate degree of correlation for any of the cavitation states. Furthermore, the features extracted from different sub-sequences of the same sample will not be highly correlated, i.e., the features will not be redundant. Since these different sub-sequences represent the same cavitation state, there is no weak or extremely weak correlation between different sub-sequences of the same sample for any cavitation state.

\begin{figure*}
\centering
\subfigure[Cavitation Choked Flow]{
\begin{minipage}[t]{0.4\linewidth}
\centering
\includegraphics[width=\textwidth,height=50mm]{figures/choked.pdf}
\end{minipage}%
}%
\subfigure[Constant Cavitation]{
\begin{minipage}[t]{0.4\linewidth}
\centering
\includegraphics[width=\textwidth,height=50mm]{figures/constant.pdf}
\end{minipage}%
}%

\subfigure[Incipient Cavitation]{
\begin{minipage}[t]{0.4\linewidth}
\centering
\includegraphics[width=\textwidth,height=50mm]{figures/incipient.pdf}
\end{minipage}%
}%
\subfigure[Non cavitation]{
\begin{minipage}[t]{0.4\linewidth}
\centering
\includegraphics[width=\textwidth,height=50mm]{figures/non.pdf}
\end{minipage}%
}%
\centering
\caption{The correlation of different sub-sequences of the same sample for different cavitation states.}
\label{fig:correlation}
\end{figure*}

\subsubsection{Comparison results between XGBoost and XGBoost+NOSW}
\begin{figure}[htbp]
    \centering
    \includegraphics[width=0.45\textwidth,height=40mm]{figures/comparsion_results.pdf}
    \caption{The results of XGBoost and XGBoost+NOSW accuracy difference in binary classification (cavitation detection) and four-class classification (cavitation intensity recognition).}
    \label{fig:results_nosw}
\end{figure}

Figure \ref{fig:results_nosw} shows the accuracy difference between XGBoost on the augmented data and the original data in binary classification (cavitation detection) and four-class classification (cavitation intensity recognition). From figure \ref{fig:results_nosw}, it can be seen that the accuracy difference is above zero at each particular window size, which indicates that the results of XGBoost in the augmented data are better compared to the original data. And the larger the accuracy difference indicates the more significant effect from data augmentation. Furthermore, this implicates that data augmentation using sliding windows is effective. And it also means that the extraction of statistical features after data augmentation does not lead to feature redundancy.

\subsubsection{Comparison results with other methods}
To demonstrate the feasibility of our method, Our approach is compared to traditional machine learning with SVM \cite{yang2005cavitation} and decision trees (DT) \cite{sakthivel2010vibration} and 1D convolutional neural networks (1D CNN) \cite{shervani2018cavitation} for deep learning (see Table \ref{tab:conpared with other methods}). 

For four-class classification (cavitation intensity recognition), XGBoost + NOSW+ASFE gets the best cavitation intensity recognition accuracy of 91.67$\%$ when window size is 1556480. The best cavitation intensity recognition results from SVM, Decision Tree and 1D CNN are 68.71$\%$, 53.86$\%$ and 87.14$\%$, respectively, which are surpassed by 22.96$\%$, 37.81$\%$ and 4.53$\%$ compared to our method. This shows that our method achieves better performance in cavitation intensity recognition than deep learning and traditional machine learning. For binary classification (cavitation detection), XGBoost + NOSW+ASFE achieves the best accuracy 93.56$\%$ when Wsize is 2334720. And the accuracy of the XGBoost + NOSW+ASFE is above 91$\%$ at each value of window size. The best cavitation detection results from SVM and Decision Tree are 83.29$\%$ and 77.00$\%$, respectively, which are surpassed by 10.27$\%$ and 16.56$\%$ compared to our method. Although the results of our method do not outperform the 1D CNN, we only decrease by 0.9$\%$. In addition, the time complexity of our method is much smaller than that of 1D CNN, and our method does not require higher computational hardware.
\input{tables/compared_other_results}








\subsection{Data Augmentation}
In general, machine learning is driven by big data \cite{l2017machine}. However, our data set has only 356 measured acoustic signals, so data augmentation is essential for handling this kind of small-sample problem. Data augmentation can improve the accuracy of the model and prevent over-fitting, and it can also teach the model the desired invariance and robustness properties \cite{shorten2019survey, Pang:2016vdc}. 

Considering the purposed maintaining for steady flow status (i.e. for each measurement it is the same fluid status class within the 3-second record duration) in each recorded data sample and the fine resolution for the sensor, one can actually split every sample into several pieces with each still holding enough essential information to decipher the flow status, but also with independent characteristics per piece due to the intrinsic randomness of the noise emission - given the piece is not so short. Therefore, we propose here a data augmentation method based on Non - Overlapping Sliding Window (NOSW), see Figure \ref{fig:sliding-window}. The method is divided into two steps:
\begin{figure}[htbp]
    \centering
     \includegraphics[width=0.5\textwidth,height=55mm]{figures/SlidingWindow.pdf}
    \caption{Data augmentation based on non-overlapping sliding window.}
    \label{fig:sliding-window}
\end{figure}
\begin{enumerate}[Step 1:]
\item The raw acoustic signal is split into pieces by a non-overlapping sliding window.
\item The time-domain signal data is then transformed into frequency-domain data by Fast Fourier Transform (FFT).
\end{enumerate}

Table \ref{tab:window-size} shows the number of samples in training and testing sets under different sized sliding windows.
\input{tables/window-size}

\subsection{Feature Extraction}
\label{sec:featureExtracion}
Although the frequency-domain signal contains abundant information, it also contains non-essential information e.g. the background noise. A statistical feature extraction thus can further bring in effectiveness for cavitation detection under statistical learning paradigm. In general, the statistical features are the average attribute of the data those can be captured. Many of these statistical features are based on moments and therefore the method of estimating these statistical features are reflecting their relationship with the distribution of the random variable. Normally, the probability density function can be decomposed into the components of moments. 

In this paper, three types of statistical features are extracted: central trend statistics, dispersion degree statistics and distribution shape statistics, given a frequency-domain signal ${x_i}(i = 1, \ldots ,n)$ with $n$ the number of data points.

\textit{\textbf{Central Trend Statistics}} The central trend statistics represent the central tendency of the overall distribution and is thus a crucial feature.  In this paper, the central trend statistics include mean value $\mu $, median $M$, low quartile ${Q_1}$ and upper quartile ${Q_3}$, which are defined as follows respectively:
\input{equations/CentralTrendStatistics}
\textit{\textbf{Dispersion Degree Statistics}} The dispersion degree statistics is the distribution of all observations off-center. In this paper, the dispersion degree statistics include minimum $Min$, maximum $Max$, inter quartile range $IQR$, standard deviation $\sigma$, root mean square $RMS$ and square root amplitude $SRA$, which are defined as follows respectively:
\input{equations/DispersionDegreeStatistics}

All data is in $[Min,Max]$, the number of data in $[Min,{Q_1}]$, $[{Q_1},M],\,[M,{Q_3}],\,[{Q_3},Max]$ accounts for about a quarter. When the interval is short, it means that the points in the interval are concentrated, and vice versa.

If the $M$ is located in the middle of ${Q_1}$ and ${Q_2}$, the data distribution is more symmetrical. If the distance between $Min$ and $M$ is greater than the distance between $Max$ and $M$, the data distribution is inclined to the left, otherwise the data distribution is inclined to the right. And we can see the length of the tail of the distribution. 

\textit{\textbf{Distribution Shape Statistics}} The distribution shape statistics is a crucial feature to measure the distribution shape of observation data. In this paper, the distribution shape statistics include kurtosis $\kappa$, skewness $\chi$, shape factor ${s_f}$, clearance factor ${c_{lf}}$ and crest factor ${c_f}$, which are defined as follows:
\input{equations/DistributionShapeStatistics}

The kurtosis $\kappa$ \cite{mardia1970measures} is a statistic that describes the steepness of the distribution of all observations in the overall sample. And it is defined as the standardized 4rd central moment of the sample. If $\kappa  = 0$, it means that the overall data distribution is the same as the normal distribution. If $\kappa  > 0$, it means that the overall data distribution is steeper compared with the normal distribution. If $\kappa  < 0$, it means that the overall data distribution is flatter compared with the normal distribution. 

The skewness $\chi$ \cite{mardia1970measures} is similar to the kurtosis $\kappa$. The skewness $\chi$ is a statistic that describes the symmetry of the distribution of all observations in the overall sample. And it is defined as the standardized 3rd central moment. If $\chi = 0$, it means that the overall data distribution is the same as the normal distribution. If $\chi \ge 0$, it means that the overall data distribution is positively skewed compared with the normal distribution. If $\chi \le 0$, it means that the overall data distribution is negatively skewed compared with the normal distribution.

Shape factor ${s_f}$ is the ratio of $RMS$ to average rectified value. Clearance factor ${c_{lf}}$ is the ratio of signal peak to $SRA$. Crest factor ${c_f}$ is the ratio of signal peak to $RMS$. These features have similar physical meanings and are often used for fault detection.

\subsection{Adaptive Selection Feature Engineering (ASFE)}
Feature engineering is an important part of data mining. In general, the upper limit of machine learning is determined by data and features. And the upper limit is approached by algorithms and models \cite{turner1999conceptual,nargesian2017learning}. In addition, deep learning has achieved good results in many fields. The main reason is that it has a good ability of feature extraction and feature processing. How to increase the nonlinear relationship between features is very necessary. Therefore, the ASFE is proposed to enhance the expression ability of original features, i.e., to enhance the quality of the features. The method includes two parts: Adaptive Feature Aggregation and Adaptive Feature Crosses.

\subsubsection{Adaptive Feature Aggregation} 
\label{sec:AdaptiveFeatureAggregation} 
The adaptive feature aggregation is to aggregate numerical features according to classification features. The adaptive feature aggregation includes two modules: Adaptive Selection Module and Feature Aggregation Module, see Figure \ref{fig:featureaggregation}. 
\begin{figure*}
    \centering
    \includegraphics[width=0.85\textwidth,height=60mm]{figures/FeatureAggregation.pdf}
    \caption{The diagram of the adaptive feature aggregation. The adaptive feature aggregation include the adaptive selection module and the feature aggregation module.}
    \label{fig:featureaggregation}
\end{figure*}

\textbf{Adaptive Selection Module} Firstly, the numerical features are input to XGBoost. Where, the numerical features include the central trend statistics, the dispersion degree statistics and the distribution shape statistics. Then, the score of each feature is obtained. Finally, the $top - k(k = 5, \cdots ,10)$ features $F_i^k(i = 1, \cdots ,k)$ are selected for feature aggregation according to the feature scores. 

\textbf{Feature Aggregation Module} The module includes three operations: \textit{Group}, \textit{Apply} and \textit{Combine}.

\textbf{\textit{-Group}} The upstream pressure and valve opening are classification features. Therefore, the numerical features are grouped according to the upstream pressure and valve opening. Then, the group of upstream pressure $Group-1$ and the group of valve opening $Group-2$ are obtained. Where, the statistical features of $Group-1$ and $Group-2$ are respectively denoted as $F_i^k(p)$, $F_i^k(o)$. And $F_i^k(p) \cup F_i^k(o) = F_i^k$.

\textbf{\textit{-Apply}} The four operations are respectively applied to $Group-1$ and $Group-2$: Median, Mean, Maximum, and Minimum.

\textbf{\textit{-Combine}} The $Group-1$ and the $Group-2$ are combined according to the same type of application.

The aggregation features $F(A)$ are generated through the adaptive feature aggregation. The number of aggregation features $F(A)$ are $Num(F(A)) = Num(apply) \times k$. Table \ref{tab:featureaggregation} shows the number of aggregation features under different $k$ through adaptive feature aggregation.
\input{tables/featureaggregation}
\subsubsection{Adaptive Feature Crosses}
\label{sec:AdaptiveFeatureCrosses} 
The adaptive feature crosses is a cross feature formed by crossing two or more features. Similar to the adaptive feature aggregation, the adaptive feature crosses is also composed of two modules: Adaptive Selection Module (see \ref{sec:AdaptiveFeatureAggregation}) and Feature Crosses Module, see Figure \ref{fig:featurecross}. 

Since the aggregation features $F(A)$ are used in the adaptive feature crosses, the adaptive feature crosses is after the adaptive feature aggregation. Firstly, the $top-k$ features $F_i^k$ and the aggregation features $F(A)$ are obtained through the Adaptive Selection Module and the Adaptive Feature Aggregation, respectively. Then, the $top-k$ features $F_i^k$ and the aggregation features $F(A)$ are merged and denoted as $\mathcal{F}_i(i = 1, \cdots ,Num(F(A) )+ k)$. Thirdly, $\mathcal{F}_i$ are copied and denoted as $\mathcal{F}_j(j = 1, \cdots ,Num(F(A) )+ k)$. Finally, ${{{\mathcal{F}_i}} \mathord{\left/{\vphantom {{{\mathcal{F}_i}} {{\mathcal{F}_j}}}} \right.\kern-\nulldelimiterspace} {{\mathcal{F}_j}}}$ and ${\mathcal{F}_i} - {\mathcal{F}_j}$, where $i \ne j$.
\begin{figure}[htbp]
    \centering
    \includegraphics[width=0.50\textwidth,height=45mm]{figures/FeatureCrosses.pdf}
    \caption{The diagram of the adaptive feature crosses.}
    \label{fig:featurecross}
\end{figure}

The number of cross features $F(C)$ are $Num(F(C)) = 2 \times (k + Num(F(A))) \times (k + Num(F(A)) - 1)$. Table \ref{tab:featurecross} shows the number of cross features under different $k$ through adaptive feature crosses.
\input{tables/featurecross}

\subsection{eXtreme Gradient Boosting with adaptive selection feature engineering}
eXtreme Gradient Boosting (XGBoost) \cite{chen2015xgboost} is a supervised ensemble learning algorithm. It uses the second order Taylor \cite{levenberg1944method} expansion loss function with further a regularization term added based on the Gradient Boosting Decision Tree (GBDT) \cite{friedman2001greedy} algorithm. It has been widely applied in different kinds of data mining fields. 

For a given data set with $n$ samples and $m$ features, ${\cal D} = \{ ({x_i},{y_i})\} (\left| {\cal D} \right| = n,{x_i} \in {\mathbb{R}^m},{y_i} \in\mathbb{R})$, a tree ensemble model uses $K$ additive functions is used to predict the output,
\begin{equation}
{{\hat y}_i} = \sum\limits_{k = 1}^K {{f_k}({x_i}),{f_k} \in {\cal F},} 
\end{equation}
where ${\cal F} = \{ f(x) = {w_{q(x)}}\} (q:{\mathbb{R}^m} \to T,w \in {\mathbb{R}^T})$ is the space of Classification and Regression Trees (CART)\cite{loh2011classification}. $q$ is an independent tree structure. $T$ is the number of leaves in each tree. $w$ is the weight of the leaf node. Therefore, the regularized objective function of XGBoost is defined as follows:
\begin{equation}
\label{eq:objectivefunction}
O{b_j} = \sum\limits_i {l({{\hat y}_i},{y_i}) + \sum\limits_k {\Omega ({f_k})} } ,
\end{equation}
\begin{equation}
\label{eq:regularizeditem}
\Omega ({f_k}) = \gamma T + \frac{1}{2}\lambda \sum\limits_{j = 1}^T {w_j^2} ,
\end{equation}
where $\sum\limits_i {l({{\hat y}_i},{y_i})}$ is the normal loss function with ${{y_i}}$ the actual value and ${{{\hat y}_i}}$ the predicted value. ${\sum\limits_k {\Omega ({f_k})} }$ is the regularization term that is composed of the sum of the complexity of all classification and regression trees. $T$ is the number of leaves in each tree. $\gamma$ is the penalty coefficient, which is used to control the number of leaf nodes. ${w_j}$ is the weight of the leaf node. The regularization term (\ref{eq:regularizeditem}) helps to smooth the final learned weights and to avoid over-fitting.

When a new CART are generated, XGBoost needs to fit the error of the previous prediction. When $t$ CART are generated, that is, after $t$-th iterations, the objective function becomes
\begin{equation}
Ob_j^{(t)} = \sum\limits_{i = 1}^n {l[{y_i},\hat y_i^{(t - 1)} + {f_t}({x_i})] + } \Omega ({f_t}),
\end{equation}
where $l[{y_i},\hat y_i^{(t - 1)} + {f_t}({x_i})]$ represents the error function and $\Omega ({f_t})$ represents the regularization term.

In order to optimize the objective function quickly, we approximate the objective function using the second-order Taylor expansion,
\begin{equation}
Ob_j^{(t)} \cong \sum\limits_{i = 1}^n {l[{y_i},{{\hat y}^{(t - 1)}} + {g_i}{f_t}({x_i}) + \frac{1}{2}{h_i}f_t^2({x_i})] + } \Omega ({f_t}),
\end{equation}
where ${g_i} = {\partial _{{{\hat y}^{(t - 1)}}}}l({y_i},{{\hat y}^{(t - 1)}})$ is the first order gradient statistics on the objective function, ${h_i} = \partial _{{{\hat y}^{(t - 1)}}}^2l({y_i},{{\hat y}^{(t - 1)}})$ is the second order gradient statistics on the objective function. ${{\hat y}^{(t - 1)}}$ represents the prediction result of the previous $t-1$ tree, and ${{f_t}({x_i})}$ represents the model of the $i$-th tree.

Since the objective function only depends on the first and second order stochastic gradients of the loss function over each data point, the constant term in the objective function can be removed, thus the following simplified objective function is used at each training step $t$,
\begin{equation}
\label{eq:SimplifiedObjFunc}
Ob_j^{(t)} \cong \sum\limits_{i = 1}^n {[{g_i}{f_t}({x_i}) + \frac{1}{2}{h_i}f_t^2({x_i})] + \Omega } ({f_t}).
\end{equation}

We substitute Equation (\ref{eq:regularizeditem}) into Equation (\ref{eq:SimplifiedObjFunc}), and arrived at the final objective function as follows :
\begin{equation}
Ob_j^{(t)} = \sum\limits_{j = 1}^T {[{G_j}{w_j} + \frac{1}{2}({H_j} + \lambda )w_j^2]}  + \gamma T,
\end{equation}
where ${G_j} = \sum\limits_{i \in {I_j}} {{g_i}}$ and ${H_j} = \sum\limits_{i \in {I_j}} {{h_i}}$. ${I_j} = \{ i|q({x_i}) = j\}$ is defined as the instance set of leaf $j$.

Based on the above XGBoost together with the adaptive selection feature engineering, we developed the cavitation detection framework on the valve acoustic signal, see Figure \ref{fig:framework} for the flow chart. Our method mainly includes five modules: data acquisition, data augmentation, feature extraction, adaptive selection feature engineering and XGBoost classification. And our source code is released at
\url{https://github.com/CavitationDetection/XGBoost_ASFE}.

First, in the data acquisition module we collected the acoustic data of the valve (see \ref{sec:2-Data Acquisition}) at different flow status. A total of 356 measurements are performed.

Second, we use a NOSW method to augment the obtained data. Before data augmentation, the total data is split into training set and testing set (with a ratio of $80\%$ : $20\%$). Perform the train/test splitting in advance can ensure that a piece of signal data after any data augmentation would only exit in the training set or the testing set. 

Third, the statistical features are extracted, which include the central trend statistics, the dispersion degree statistics and the distribution shape statistics (see \ref{sec:featureExtracion}). Thus the training feature set and the testing feature set are obtained separately.

Fourth, the adaptive selection feature engineering is applied on both the training feature set and the testing feature set to obtain the optimal feature set. The adaptive selection feature engineering in our methods includes feature aggregation (see \ref{sec:AdaptiveFeatureAggregation}) and feature crosses (see \ref{sec:AdaptiveFeatureCrosses}), both of them contain the adaptive selection modules. The adaptive selection module uses the method of solving the Gini index \cite{gastwirth1972estimation} in the XGBoost algorithm to calculate the importance of the features in the feature data. We can also adaptively select $top-k$ features to form the optimal feature subset according to different data.

Finally, the optimal training feature set and testing feature set are used for the training and prediction of the XGBoost algorithm, respectively. Since the amount of no-flow status is very small and cavitation phenomenon is more relevant for practice, we regard both turbulent-flow and no-flow as ``no cavitation''. Therefore, we study binary classification (``cavitation'' / ``no-cavitation'') and four class cavitation-staging detection.
\begin{figure}[htbp]
    \centering
    \includegraphics[width=0.45\textwidth,height=205mm]{figures/Framework.pdf}
    \caption{Flow chart of our cavitation detection framework on acoustic signal, based on XGBoost with the adaptive feature engineering.}
    \label{fig:framework}
\end{figure}








\subsection*{Binary classification}
The precision, recall and F1-score of XGBoost + NOSW are shown in Table \ref{tab:appendix_xgboost_nosw_two}.
\input{tables/Appendix_XGBoost_NOSW_Two}

The precision, recall and F1-score of our method (XGBoost + NOSW + ASFE) are shown in Tables \ref{tab:appendix_xgboost_nosw_asfe_precision}, \ref{tab:appendix_xgboost_nosw_asfe_recall} and \ref{tab:appendix_xgboost_nosw_asfe_f1score}.
\input{tables/Appendix_XGBoost_ASFE_precision}
\input{tables/Appendix_XGBoost_ASFE_recall}
\input{tables/Appendix_XGBoost_ASFE_f1score}

\subsection*{Four-class classification}
The precision, recall and F1-score of XGBoost + NOSW are shown in Table \ref{tab:appendix_xgboost_nosw_four}.
\input{tables/Appendix_XGBoost_NOSW_Four}


The precision, recall and F1-score of our method (XGBoost + NOSW + ASFE) are shown in Tables \ref{tab:appendix_xgboost_nosw_asfe_Precision_four}, \ref{tab:appendix_xgboost_nosw_asfe_Recall_four} and \ref{tab:appendix_xgboost_nosw_asfe_F1score_four}.
\input{tables/Appendix_XGBoost_ASFE_Presision_Four}
\input{tables/Appendix_XGBoost_ASFE_Recall_Four}
\input{tables/Appendix_XGBoost_ASFE_F1score_Four}



















\section{Introduction}
\input{sections/1_Introduction}

\section{Experimental setup and data acquisition}
\label{sec:2-Data Acquisition}
\input{sections/2_Experimental_setup_and_Data_Acqusition}

\section{Methods}
\label{sec:3-methods}
\input{sections/3_Methods}

\section{Results and Discussions}
\label{sec:4-Results and Discussion}
\input{sections/4_Results_and_Discussion}

\section{Conclusion and Future Research}
\label{sec:5-Conclusion}
\input{sections/5_Conclusion}

\section*{Acknowledgements}
\label{sec:acknowledgements}
\input{sections/Acknowledgements}

\bibliographystyle{unsrt}

\section{Introduction}
\lipsum[2]
\lipsum[3]


\section{Headings: first level}
\label{sec:headings}

\lipsum[4] See Section \ref{sec:headings}.

\subsection{Headings: second level}
\lipsum[5]
\begin{equation}
\xi _{ij}(t)=P(x_{t}=i,x_{t+1}=j|y,v,w;\theta)= {\frac {\alpha _{i}(t)a^{w_t}_{ij}\beta _{j}(t+1)b^{v_{t+1}}_{j}(y_{t+1})}{\sum _{i=1}^{N} \sum _{j=1}^{N} \alpha _{i}(t)a^{w_t}_{ij}\beta _{j}(t+1)b^{v_{t+1}}_{j}(y_{t+1})}}
\end{equation}

\subsubsection{Headings: third level}
\lipsum[6]

\paragraph{Paragraph}
\lipsum[7]

\section{Examples of citations, figures, tables, references}
\label{sec:others}
\lipsum[8] \cite{kour2014real,kour2014fast} and see \cite{hadash2018estimate}.

The documentation for \verb+natbib+ may be found at
\begin{center}
  \url{http://mirrors.ctan.org/macros/latex/contrib/natbib/natnotes.pdf}
\end{center}
Of note is the command \verb+\citet+, which produces citations
appropriate for use in inline text.  For example,
\begin{verbatim}
   \citet{hasselmo} investigated\dots
\end{verbatim}
produces
\begin{quote}
  Hasselmo, et al.\ (1995) investigated\dots
\end{quote}

\begin{center}
  \url{https://www.ctan.org/pkg/booktabs}
\end{center}


\subsection{Figures}
\lipsum[10] 
See Figure \ref{fig:fig1}. Here is how you add footnotes. \footnote{Sample of the first footnote.}
\lipsum[11] 

\begin{figure}
  \centering
  \fbox{\rule[-.5cm]{4cm}{4cm} \rule[-.5cm]{4cm}{0cm}}
  \caption{Sample figure caption.}
  \label{fig:fig1}
\end{figure}

\subsection{Tables}
\lipsum[12]
See awesome Table~\ref{tab:table}.

\begin{table}
 \caption{Sample table title}
  \centering
  \begin{tabular}{lll}
    \toprule
    \multicolumn{2}{c}{Part}                   \\
    \cmidrule(r){1-2}
    Name     & Description     & Size ($\mu$m) \\
    \midrule
    Dendrite & Input terminal  & $\sim$100     \\
    Axon     & Output terminal & $\sim$10      \\
    Soma     & Cell body       & up to $10^6$  \\
    \bottomrule
  \end{tabular}
  \label{tab:table}
\end{table}

\subsection{Lists}
\begin{itemize}
\item Lorem ipsum dolor sit amet
\item consectetur adipiscing elit. 
\item Aliquam dignissim blandit est, in dictum tortor gravida eget. In ac rutrum magna.
\end{itemize}


\section{Conclusion}
Your conclusion here

\section*{Acknowledgments}
This was was supported in part by......

\bibliographystyle{unsrt}  
