A Tale of Two Countries: A Longitudinal Cross-Country Study of Mobile Users' Reactions to the COVID-19 Pandemic Through the Lens of App Popularity
Liu Wang
Haoyu Wang
Yi Wang
Gareth Tyson
Fei Lyu
The ongoing COVID-19 pandemic has profoundly impacted people's life around the world, including how they interact with mobile technologies. In this paper, we seek to develop an understanding of how the dynamic trajectory of a pandemic shapes mobile phone users' experiences. Through the lens of app popularity, we approach this goal from a cross-country perspective. We compile a dataset consisting of six-month daily snapshots of the most popular apps in the iOS App Store in China and the US, where the pandemic has exhibited distinct trajectories. Using this longitudinal dataset, our analysis provides detailed patterns of app ranking during the pandemic at both category and individual app levels. We reveal that app categories' rankings are correlated with the pandemic, contingent upon country-specific development trajectories. Our work offers rich insights into how the COVID-19, a typical global public health crisis, has influence people's day-to-day interaction with the Internet and mobile technologies.
00-main.tex
\documentclass[acmsmall,screen]{acmart}

\AtBeginDocument{%
  \providecommand\BibTeX{{%
    \normalfont B\kern-0.5em{\scshape i\kern-0.25em b}\kern-0.8em\TeX}}}


\settopmatter{printacmref=false, printccs=false, printfolios=true} % We want page numbers on submissions
\setcopyright{acmcopyright}
\copyrightyear{2020}
\acmYear{2020}

\renewcommand\footnotetextcopyrightpermission[1]{} %

\usepackage{multirow}
\usepackage{booktabs}
\usepackage{ifthen}
\usepackage{color}
\usepackage{url}
% \usepackage{amssymb}
\usepackage{xspace}
\usepackage[T1]{fontenc}
\usepackage{enumerate}
\usepackage[linesnumbered,ruled,vlined]{algorithm2e}
\usepackage{array,multirow,graphicx}
\usepackage{float}
\usepackage{balance}
\usepackage{tikz}
\usepackage{calc}
\usepackage{subfigure}
\usepackage{listings,amsfonts}
\usepackage{amsmath,xcolor,pifont}
\usepackage{url}
\usepackage{xspace}
\usepackage{hyperref,endnotes}
\usepackage{xcolor}
\usepackage{array}
\usepackage{multirow,makecell}
\usepackage[misc]{ifsym}
\usepackage{enumitem}


\lstset{numbers=left, numberstyle=\tiny,
xleftmargin=2em,xrightmargin=2em, aboveskip=1em}


\newcommand{\gareth}[1] {\textcolor{red}{GT: \it{#1}}}

\definecolor{yellow}{RGB}{255,255,153}
\definecolor{grey}{RGB}{224,224,224}
\newcommand{\highlight}{\cellcolor{grey}}
\newcommand{\haoyu}[1]{{\color{red} HW: #1}}


\begin{document}

\title{A Tale of Two Countries: A Longitudinal Cross-Country Study of Mobile Users' Reactions to the COVID-19 Pandemic Through the Lens of App Popularity}
\renewcommand{\shorttitle}{A Tale of Two Countries}


\author{Liu Wang}
\affiliation{%
  \institution{Beijing University of Posts and Telecommunications}
  \city{Beijing}
  \country{China}
}

\author{Haoyu Wang}
\authornote{Corresponding Author: Haoyu Wang (haoyuwang@bupt.edu.cn).}
\affiliation{%
  \institution{Beijing University of Posts and Telecommunications}
  \city{Beijing}
  \country{China}
}
\email{haoyuwang@bupt.edu.cn}

\author{Yi Wang}
\affiliation{%
  \institution{Beijing University of Posts and Telecommunications}
  \city{Beijing}
  \country{China}
}

\author{Gareth Tyson}
\affiliation{%
  \institution{Queen Mary University of London}
  \city{London}
  \country{United Kingdom}
}

\author{Fei Lyu}
\affiliation{%
  \institution{Beijing University of Posts and Telecommunications}
  \city{Beijing}
  \country{China}
}

\input{01-abstract}


\maketitle

\input{intro}

\input{background}

\input{studydesign}

\input{PopularityOfCategory}

\input{PerCategory}

\input{PerApp}

\input{side-effect}

\input{discussion}


\balance

\bibliographystyle{plain}
\bibliography{cite}


\end{document}

01-abstract.tex
\begin{abstract}
The ongoing COVID-19 pandemic has profoundly impacted people's life around the world, including how they interact with mobile technologies. In this paper, we seek to develop an understanding of how the dynamic trajectory of a pandemic shapes mobile phone users' experiences. Through the lens of app popularity, we approach this goal from a cross-country perspective. We compile a dataset consisting of six-month daily snapshots of the most popular apps in the iOS App Store in China and the US, where the pandemic has exhibited distinct trajectories. Using this longitudinal dataset, our analysis provides detailed patterns of app ranking during the pandemic at both category and individual app levels. We reveal that app categories' rankings are correlated with the pandemic, contingent upon country-specific development trajectories. Our work offers rich insights into how the COVID-19, a typical global public health crisis, has influence people's day-to-day interaction with the Internet and mobile technologies.   

\end{abstract}
PerApp.tex
\section{Temporal Patterns in App Popularity}
\label{sec:perapp}

We next introduce the findings related to the $\mathbf{RQ_2}$.
Up to now, we have identified that the case rates of the outbreak have an immediate impact on app rankings, with a number of apps significantly correlated with the outbreak.
We then delve into the different patterns that each app ranking conforms to, i.e., the different correlations with the pandemic trends. We make efforts to create a taxonomy and classify the apps based on their calculated correlation coefficients, where each category corresponds to a pattern of change in rankings. 
Considering the different trends in the evolution of the pandemic situation in China and the United States over the first six months, i.e., China has experienced a progression from deterioration to improvement, while the US has tended to be more severe. We therefore examine the situation in China in two stages for a more fine-grained comparison.

The first stage was when the pandemic was worsening, and the second stage was when it was improving. The cut-off date is February 17, when the number of new confirmed cases reached its highest.
% \gareth{How was this date decided?}
Besides, since a larger correlation coefficient shows a more significant correlation, we consider introducing a threshold \emph{n}, and regard an app as strongly positively or negatively correlated with the outbreak (overall or in a stage) whose absolute correlation coefficient is greater than \emph{n}. As such, the key challenge is to select an appropriate \emph{n}. Nevertheless, the correlation coefficient represents an effect size and so we can verbally describe the strength of the correlation using the guide that Evans \cite{evans1996straightforward} suggests for the absolute value of \emph{r} in Table~\ref{tab:r-strength}. Thus, we set the threshold to $0.6$. To be specific, we take $r>0.6$ as strong positive correlation (SPC) and $r<-0.6$ as strong negative correlation (SNC).
Under all these scenarios, we create a taxonomy including eight categories for China and two categories for the US.

\begin{table}[htb]
    \centering
    \vspace{-0.15in}
    \caption{A commonly used interpretations of the \emph{r} values}
    \vspace{-0.1in}
    \begin{tabular}{c|c}
    \toprule
     Magnitude of Correlation & Description of Strength \\
     \midrule
     0.01 - 0.19 & Very Weak \\
     0.20 - 0.39 & Weak \\
     0.40 - 0.59 & Moderate \\
     0.60 - 0.79 & Strong \\
     0.80 - 1.00 & Very Strong \\
     \bottomrule
    \end{tabular}
    \label{tab:r-strength}
\end{table}

\subsection{Taxonomy}
\subsubsection{China}
In China, we consider the situation from both an overall and a phased perspective.

(1) Overall-level.
We start with checking the correlation coefficient \emph{r} between the app ranking and the overall period of the pandemic. If \emph{r}>0.6 we classify the app as ``SPC with the overall.'' Similarly, we categorize the app as ``SNC with the overall'' if its \emph{r}<-0.6.

(2) Stage-level.
For the apps without a strong correlation between its ranking and the overall pandemic period, we then examine their correlations with the two stages respectively, i.e., the growth stage of the pandemic and the decline stage of the pandemic. If an app's rank is strongly correlated to only one of the stages, we label it as a one-stage correlation. Concretely, there are four categories including ``SPC with growth stage'', ``SNC with growth stage'', ``SPC with decline stage'' and ``SNC with decline stage.'' 

If an app's ranking is strongly correlated with both stages but in opposite directions, we deem it as two-stages correlation. Specifically, there are two categories including ``SPC with growth stage, SNC with decline stage'' and ``SNC with growth stage, SPC with decline stage.''

\subsubsection{US}
In the US, the situation is relatively straightforward due to the single upward trend of the pandemic, and we can simply focus on two categories, i.e., ``SPC with the overall'' and ``SNC with the overall.''


\subsubsection{Results}
Based on the above, we classify the apps in the two countries separately and the results are shown in Table~\ref{tab:taxonomy}. For China, we have eight categories due to the shifts in the pandemic trend, with the number of apps ranging from 2 to 44. In the US, there are two categories with app quantities of 72 and 116, respectively.

\begin{table}[htb]
    \centering
    \caption{Eight categories in China and two categories in the US}
    \vspace{-0.1in}
    \resizebox{0.99\textwidth}{!}{
    \begin{tabular}{|l|l|l|c|}
    \toprule
    \multicolumn{1}{|l|}{} & \textbf{Category}  & \textbf{Ranking/Popularity Change} & \textbf{\# Apps} \\ 
    \midrule
    \multirow{2}{*}{Overall}  & SPC with the overall & The app ranking/popularity decreases significantly during the growth stage & 41\\ & & of the pandemic and increases significantly during the decline stage. &  \\ \cline{2-4}
    & SNC with the overall & The app ranking/popularity increases significantly during the growth stage & 44\\ & & of the pandemic and decreases significantly during the decline stage. &  \\ \hline
    \multicolumn{1}{|l|}{\multirow{4}{*}{One-stage}} 
    & SPC with the growth stage  & The app ranking/popularity decreases significantly during the growth stage & 20\\&&of the pandemic but no significant increase or decrease in the decline stage. & \\ \cline{2-4}
    & SNC with the growth stage & The app ranking/popularity increases significantly during the growth stage & 50\\ &&of the pandemic but no significant increase or decrease in the decline stage. &  \\ \cline{2-4}
    & SPC with the decline stage & The app ranking/popularity increases significantly during the decline stage & 22\\ &&of the pandemic but no significant increase or decrease in the growth stage. & \\ \cline{2-4}
    & SNC with the decline stage & The app ranking/popularity decreases significantly during the decline stage & 19\\&&of the pandemic but no significant increase or decrease in the growth stage. & \\ \hline
    \multicolumn{1}{|l|}{\multirow{2}{*}{two-stages}} 
    & SPC with growth stage,&The app ranking/popularity decreases significantly during the growth stage & 7\\ & and SNC with decline stage &of the pandemic and decreases significantly during the decline stage. & \\ \cline{2-4}
    & SNC with growth stage,& The app ranking/popularity increases significantly during the growth stage & 2\\ & and SPC with decline stage &of the pandemic and increases significantly during the decline stage. & \\
    \bottomrule
    \hline
    \multicolumn{1}{|l|}{
    \multirow{2}{*}{US}} &  SPC with the overall & The app ranking/popularity decreases significantly during the period. & 72 \\ \cline{2-4}
                         &  SNC with the overall & The app ranking/popularity increases significantly during the period. & 116 \\
    \bottomrule
    \end{tabular}}
    \label{tab:taxonomy}
\end{table}


\subsection{Per Group Analysis}
As aforementioned, each of the four groups symbolize a pattern of how an app's ranking changes in response to the outbreak. We next deep dive into each group.

\subsubsection{SPC with the overall in China.}
There are 41 apps in China whose app rankings are strongly positively related to the pandemic situation, i.e., they suffered a regression in ranking when the pandemic got worse (measured by case rate). Subsequently, their rankings rebounded as the situation recovered. 
The app with the strongest correlation (0.911) is \textit{Dianping}, a leading local lifestyle information and trading platform in China, providing users with information services such as merchant information, consumer reviews and offers, as well as transaction services including group buying, restaurant reservations, takeaway and e-membership card. It is followed by \textit{TikTak Travel} (0.885), \textit{Ctrip Travel} (0.882) and \textit{Tencent Maps} (0.876), all of which cater to services like travel and navigation. Unsurprisingly, most of the apps in this category are closely tied to outdoor activities. Given the strict quarantine policy during the outbreak in China, it is no wonder that the rankings of this kind of apps undergo such a shift. The trend comparisons for two representative apps are visualized in Figure~\ref{fig:c1-example}. The red line shows the ranking of the app each day and the green line presents the number of active confirmed cases per day. We can observe that their trends are almost identical.

\begin{figure}[htb]
    \centering
    \subfigure[Dianping]{
    \includegraphics[width=0.4\textwidth]{Figures/dianping.pdf}}
    \quad
    \subfigure[TikTak Travel]{
    \includegraphics[width=0.4\textwidth]{Figures/dida.pdf}}
    \vspace{-0.15in}
    \caption{Two examples for SPC with the overall in China.}
    \label{fig:c1-example}
\end{figure}


\subsubsection{SNC with the overall in China.}
The situation in this category is opposite to the previous one. There is a strong negative correlation between the app ranking and the pandemic case rate trends for the 44 apps in this category. This means that the outbreak has contributed to the growth of their rankings. The subsequent recovery has therefore resulted in a drop in the rankings. 
For context, Figure~\ref{fig:c2-example} presents two example apps with these properties. We see opposite trends in the app ranking vs.\ case rate. Note that since the Apple Store's app ranking list offers up to 1,500 slots, for rankings that fall out of the top 1,500, we replace them with 1,500 in the chart.

We highly four prominent examples that exhibit these behaviors:  
1) \textit{Happy Disappear}, a triple-elimination casual game with the strongest Pearson correlation coefficient of -0.916, which may be a good choice for people to pass time during the lockdown. 
2) \textit{Diandu} (-0.88), an educational app for primary and secondary school language, mathematics, and English learning, which is helpful for tutoring children during the extended period of home study. 
3) \textit{Yong Hui Buy Food} (-0.818), an online service platform that provides consumers with fresh food and other grocery products through home delivery services, which alleviates the inconvenience of going out to buy food during quarantine. 
4) \textit{Tinker Medicine} (-0.776), a pharmaceutical and health product that helps pharmacies provide convenient services to the public and provides free delivery services for users who place orders through the app.
Overall, we observe that most apps in this category have a positive effect on people's lives in some way during the pandemic.




\begin{figure}[htb]
    \centering
    \subfigure[Diandu]{
    \includegraphics[width=0.4\textwidth]{Figures/diandu.pdf}}
    \quad
    \subfigure[Yong Hui Buy Food]{
    \includegraphics[width=0.4\textwidth]{Figures/yonghui.pdf}}
    \vspace{-0.15in}
    \caption{Two examples for SNC with the overall in China.}
    \label{fig:c2-example}
\end{figure}

\subsubsection{SPC with the growth stage in China.} 
In this category, the app ranking is strongly positively correlated only with the growth stage of the pandemic. In other words, as the situation gets worse, the app ranking drops; yet it does not show a definitive upward or downward trend as the situation recovers. For example, there are two typical apps shown in Figure~\ref{fig:c3-example}.
One is called \textit{Eastern Airlines} that aims to provide safe and convenient ticketing and travel experiences. The other is \textit{Damek}, a comprehensive live entertainment ticket marketing platform in China, covering concerts, dramas, musicals, sporting events, etc. Both of them have a high correlation coefficient between app ranking and active cases in the pandemic growth stage (0.933 and 0.772, respectively). However, such correlations are not that strong during the decline stage. This meas there are some apps that are adversely affected by the pandemic but do not recover in tandem with the pandemic.

\begin{figure}[htb]
    \centering
    \subfigure[Eastern Airlines]{
    \includegraphics[width=0.4\textwidth]{Figures/esternair.pdf}}
    \quad
    \subfigure[Damek]{
    \includegraphics[width=0.4\textwidth]{Figures/damai.pdf}}
    \vspace{-0.15in}
    \caption{Two examples for SPC with the growth stage in China.}
    \label{fig:c3-example}
\end{figure}

\subsubsection{SNC with the growth stage in China.}
Similarly, we investigate the situation where app rankings are only strongly negatively correlated with the \emph{growth} stage of the pandemic, i.e., as the pandemic worsened, the app ranking improved, but there was no clear trend of a growing or declining thereafter. 
We take a look at the following examples that have high correlation coefficients:
1) \textit{Tencent Classroom}  (-0.943) is a professional online education platform that provides teachers with online teaching and students with interactive learning, which has been a great support for digital learning due to the school closures.
2)	\textit{Anhui Things} (-0.93) is a mobile app for Anhui Province Government Services Network designed to provide government services, high-frequency public services and citizen-friendly service matters, which allows citizens to handle some business without leaving home.
3) \textit{Tencent Meetings}  (-0.801) is a video conferencing product that features online meetings, useful for the needs of users working from home. 
This indicates that there are a number of apps that played an important role in the war against the pandemic, and continue to be used. This may indicate longer-term implications in how people live and work after the pandemic. 

\begin{figure}[htb]
    \centering
    \subfigure[Tencent Classroom]{
    \includegraphics[width=0.4\textwidth]{Figures/tencent-classroom.pdf}}
    \quad
    \subfigure[Tencent Meetings]{
    \includegraphics[width=0.4\textwidth]{Figures/tencent-meeting.pdf}}
    \vspace{-0.15in}
    \caption{Two examples for SNC with the growth stage in China.}
    \label{fig:c4-example}
\end{figure}

\subsubsection{SPC with decline stage in China.}
In this group, the correlation between app ranking and the pandemic case rate is not significant in the first stage of the pandemic, but as the situation turns better, the app ranking rises. 
For example, the app with the highest correlation coefficient (0.924) in the decline stage is \textit{Shell Finder}, a home searching platform featuring comprehensive and real property information as well as industry innovations such as VR viewings, home valuation, and intelligent recommendations.
In addition, \textit{Traffic Management 12123} (0.881) is the official client of the Internet traffic safety management platform, providing a full range of traffic services and reservation of motor vehicle, driver's license and illegal processing business, etc. As shown in Figure~\ref{fig:c5-example}, we can observe that the rankings of these apps were only slightly affected in the early days of the pandemic.
This reflects the fact that as the situation improves and people's lives gradually return to normal, some apps are coming back to the forefront as soon as possible.

\begin{figure}[htb]
    \centering
    \subfigure[Shell Finder]{
    \includegraphics[width=0.4\textwidth]{Figures/shell.pdf}}
    \quad
    \subfigure[Traffic Management 12123]{
    \includegraphics[width=0.4\textwidth]{Figures/12123.pdf}}
    \vspace{-0.15in}
    \caption{Two examples for SPC with the decline stage in China.}
    \label{fig:c5-example}
\end{figure}

\subsubsection{SNC with decline stage in China.}
As expected, this category contains apps whose rankings have a strong negative correlation with the active cases only in the decline stage, i.e., as the pandemic improves, the ranking declines. For this case, some apps may seem less important as people gradually return to their normal lives. For example, the app with the highest correlation coefficient (-0.852) is \textit{Fast View}, an informational app that aggregates and provides users different forms of content including graphics, videos, etc., as well as current affairs content and the latest real-time information.
Furthermore, there is an online gaming app called \textit{Doudizhu} (-0.762), which is a very popular card game in China. This rose to the top of the rankings at the beginning of the outbreak, but quickly declined for a long time as shown in Figure~\ref{fig:doudizhu}.
These apps seem to become popular for a short period of the outbreak as a means for people to keep abreast of events or to pass the time. However, they cannot last very long, especially when people return to their normal life routines.

\begin{figure}[htb]
    \centering
    \subfigure[Fast View]{
    \label{fig:fastview}
    \includegraphics[width=0.4\textwidth]{Figures/fastview.pdf}}
    \quad
    \subfigure[Doudizhu]{
    \label{fig:doudizhu}
    \includegraphics[width=0.4\textwidth]{Figures/doudizhu.pdf}}
    \vspace{-0.15in}
    \caption{Two examples for SNC with the decline stage in China.}
    \label{fig:c6-example}
\end{figure}


\subsubsection{SPC with growth stage, SNC with decline stage \& SNC with growth stage, SPC with decline stage.}
In addition to the above apps, there are also a few apps that have strong correlations with both stages but in different directions. There are 7 apps that show strongly positive correlations during the growth stage of the pandemic while exhibiting negative correlations with the decline stage. Their rankings can be roughly treated as constantly declining over the outbreak. For example, there is a puzzle elimination game called \textit{Cube Battle} that has fallen steadily out of the top 1,500 since the outbreak began, as shown in Figure~\ref{fig:cubebattle}. It has a high correlation coefficient of 0.803 within the growth stage and -0.878 within the decline stage. In contrast, the 2 apps that are strongly negatively correlated with the growth stage and positively related to the decline stage exhibit the opposite trend, i.e., the ranking is almost continuously rising for most of the time. For example, there is a beauty and cosmetic app called \textit{SoYong} that has a generally fluctuating upward trend in the ranking from February to May, as displayed in Figure~\ref{fig:soyoung}. It has the correlation coefficient of -0.86 within the growth stage and 0.717 within the decline stage. Note that both types are few in number, indicating that the two evolving patterns may be very uncommon.

\begin{figure}[htb]
    \centering
    \subfigure[Cube Battle]{
    \label{fig:cubebattle}
    \includegraphics[width=0.4\textwidth]{Figures/cubebattle.pdf}}
    \quad
    \subfigure[SoYong]{
    \label{fig:soyoung}
    \includegraphics[width=0.4\textwidth]{Figures/soyoung.pdf}}
    \vspace{-0.15in}
    \caption{Two examples for the two-stages correlations respectively.}
    \label{fig:c7-example}
\end{figure}

\subsubsection{SPC and SNC in the US.}
Due to the single trend in the number of the active confirmed cases in the US, i.e., consistently increasing since mid-March, we construct two categories: SPC and SNC with the overall, with 72 and 116 apps, respectively. Since the pandemic situation in the US has been increasingly serious, the apps with an SPC to the pandemic case rate should show a largely declining ranking. For example, the app with the highest correlation coefficient (0.965) is a gaming app called \textit{Trivia Crack}, which has its ranking change in parallel with the number of active cases, as shown in Figure~\ref{fig:triviacrack}. In contrast, the apps with an SNC with the pandemic display a generally rising trend. An example is displayed in Figure~\ref{fig:videoleap}, which is an app called \textit{Videoleap Video Editor \& Maker} (-0.88), which is an app for producing videos. Its ranking has kept advancing during the six months.  

\begin{figure}[htb]
    \centering
    \subfigure[Trivia Crack]{
    \label{fig:triviacrack}
    \includegraphics[width=0.4\textwidth]{Figures/triviacrack.pdf}}
    \quad
    \subfigure[Videoleap Video Editor \& Maker]{
    \label{fig:videoleap}
    \includegraphics[width=0.4\textwidth]{Figures/videoleap.pdf}}
    \vspace{-0.15in}
    \caption{Two examples for the US.}
    \label{fig:us-example}
\end{figure}

%\subsection{Summary}
%In terms of the classification observations, apps' rankings exhibits different evolution processes with a great relation to the pandemic.

\subsection{Summary}

In summary, the pandemic has diverse patterns in terms of its impact on app ranking. In China, 85 out of 586 apps are strongly correlated with the half-year pandemic, either positively or negatively. Moreover, 111 apps have strong correlations (positive or negative) with only one stage of the pandemic (growth or decline). 9 apps even show strong correlations with the two stages in different directions. These correlation categories reflect different characteristics of the COVID-19 effect on app rankings. Most of the apps in the same category share some similarities in terms of their adaptability to the pandemic situation.
While for the US, since the vast majority of apps are game apps, this somewhat limits our study of the adaptation of app functions and features to the pandemic.
PerCategory.tex
\subsection{Correlation Analysis}
As mentioned in the above, we employ Pearson Correlation Coefficient to explore the correlations between app popularity and the pandemic situation.
In this section, we provide correlation analysis results at both the overall and category levels, respectively.




\subsubsection{Overall distribution}
Figure~\ref{fig:corr-overall} presents the Cumulative Distribution Function (CDF) of the correlation coefficients between app ranking and the number of active confirmed cases in the two countries. 
The trends are very similar in the US and China, with a roughly 50:50 number of positively and negatively correlated apps. Besides, there are indeed a number of apps whose ranking changes have a clear correlation with the pandemic evolution in both countries. Specifically, 27\% of the apps in China have a correlation coefficient less than -0.4 and nearly 40\% in the US. In addition, over 20\% of the apps show a correlation coefficient greater than 0.4 both in China and the US, which manifests many positive correlations. More notably, 23\% of the apps in the US and 10\% in China have correlation coefficients less than -0.6, and nearly 10\% in both countries are greater than 0.6, which can be considered as strong or very strong correlations (see Table~\ref{tab:r-strength}). This observation suggests that many apps' popularity does haves significant correlation with the development of pandemic, positively or negatively.

\begin{figure}[htb]
    \centering
    
    \includegraphics[width=0.5\textwidth]{Figures/active-correlation-cdf-2.pdf}
    \caption{The overall distribution of correlation coefficients}
    \label{fig:corr-overall}
\end{figure}

\subsubsection{Category-level distribution}
We further study the distribution of correlations at the category level.
Figure~\ref{fig:boxplot-cn} shows the boxplot of correlation coefficients for each category in \emph{China}. The red line in the box indicates the median, while the blue line represents the mean. First, we observe that the two most prominent categories are \texttt{Travel} and \texttt{Navigation}, with all correlation coefficients being positive and a median greater than 0.75. This indicates that there is a significant positive correlation between pandemic situation and the ranking of apps in these two categories, i.e., the ranking decreases as the pandemic situation worsens and increases when the situation improves (as measured by case rate).


Moreover, most apps in the \texttt{Education} category have negative correlation coefficients with a median of about -0.6, suggesting that \texttt{Education} apps are in general negatively correlated with the pandemic situation.
The findings are consistent with the evolution of the popularity of these three categories, as previously discussed. 
However, for the \texttt{Business} category, there seems to be no significant negative correlation discernible. A possible explanation is the long term popularity for most \texttt{Business} apps after the outbreak, as we can observe in Figure~\ref{fig:business}. There are a number of \texttt{Business} apps that jumped up the rankings during the outbreak and have been leading ever since, even though the number of active cases has dwindled to almost nothing. As a result, there is no significant positive or negative correlation in the later phases, thereby undermining the overall correlation coefficient. We posit that we can examine it in more detail by splitting it into two stages, which we will discuss later.
In general, although most of the categories do not show a significant positive or negative correlation, there are certain categories that do, such as \texttt{Travel} and \texttt{Navigation} showing a strong positive correlation and \texttt{Education} presenting a strong negative correlation.

\begin{figure}
    \centering
    \includegraphics[width=0.8\textwidth]{Figures/category-boxplot-cn.pdf}
    \caption{The distribution of correlation coefficients in each category in China}
    \vspace{-0.2in}
    \label{fig:boxplot-cn}
\end{figure}


The situation in the \emph{US} is somewhat different. Figure~\ref{fig:boxplot-us} presents the boxplot of correlation coefficients for each category in the US. Surprisingly, there seem to be no conspicuous categories showing significant positive or negative correlations with the pandemic. Again focusing on the Travel category, we can see the correlation coefficients range between -0.5 and 0.5, with a mean close to 0, failing to indicate a strong correlation. 

It is not hard to understand when we review Figure~\ref{fig:travel}. Although the \texttt{Travel} apps have a significant drop in ranking at the beginning, it only lasted for about two months and then gradually rebounded back, during which time the number of diagnoses in general kept increasing, thus there is no noticeable correlation between ranking and pandemic during the entire half year. In the same vein, there is hardly any category in which app rankings are generally growing or declining for a extended period, which weakens the correlations. Therefore, in terms of category level, the correlation between app popularity and the pandemic seems to be less significant.

\begin{figure}
    \centering
    \includegraphics[width=0.8\textwidth]{Figures/category-boxplot-us.pdf}
    \caption{The distribution of correlation coefficients in each category in US}
    \vspace{-0.2in}
    \label{fig:boxplot-us}
\end{figure}

\subsubsection{Summary}
From the overall-level analysis, we can conclude that there are indeed a number of apps that are strongly correlated with the outbreak situation over the six months of the pandemic, positively or negatively, in both countries. In terms of the category level, the vast majority of categories fail to demonstrate strong correlations.
Besides, we can discover some distinctions in the correlations between app rankings and pandemic conditions in China and the US. In China, there are certain categories (e.g., Travel and Navigation) showing significant correlations, while in the US, the correlations are not very strong in any category. This confirms that the impact of COVID-19 on the mobile app ecosystem may differ from country to country.
PopularityOfCategory.tex
\section{Popularity of App Categories During the Pandemic}
\label{sec:category}

\begin{figure*}[h]
    \centering
    \subfigcapskip=-5pt
    \begin{center}
        \subfigure{
        \includegraphics[width=1\textwidth]{Figures/legend.pdf}
        } 
        \vskip -0.05in
        \setcounter{subfigure}{0}
        \subfigure[China]{
        \includegraphics[width=1\textwidth]{Figures/category-cn-2.pdf}
        \label{fig:category-cn}}
        \vskip -0.03in
        \subfigure[US]{
        \includegraphics[width=1\textwidth]{Figures/category-us-2.pdf}
        \label{fig:category-us}}
        \vspace{-0.15in}
        \caption{The distribution of number of apps in each category among top-100 apps per day}
        \label{fig:category}
    \end{center}
\end{figure*}


\subsection{The dynamics of the popularity}
\subsubsection{Coarse-grained inspection.}
Figure~\ref{fig:category} visualizes the dynamics of the popularity of each app category. We use stacked bar charts to show the number of apps in each category among each day's top 100 apps in the app stores of China and the Unite States, respectively. As aforementioned, there are 22 categories (see the legend in Figure~\ref{fig:category}). 
In a specific day, the change in  the numbers of apps in each category over time is represented by the fluctuations of the vertical lengths of the corresponding color.

In general, most category's popularity does not exhibit significant fluctuations in either China or the US. Some categories has been constantly popular over the long run, such as \texttt{Games}, \texttt{Social Networking}, \texttt{Photo \& Video} and \texttt{Entertainment}, as well as some being consistently less popular, such as \texttt{Reference}, \texttt{Medical} and \texttt{Weather}. However, there are four categories that are worth noting in both countries, i.e., \texttt{Business}, \texttt{Education}, \texttt{Travel}, and \texttt{Navigation}, whose popularity exhibits 
remarkable dynamics along with the outbreak in the respective countries. 
We further examine the details of the dynamics of their popularity.

\subsubsection{Fine-grained inspection.}
For each of the four categories, we plot the overlay of scatter and line charts for both countries in Figure~\ref{fig:four-category}. The scatter represents the ranking of each app in that category and the lines show the number of newly confirmed and active COVID-19 cases per day.
First, we observe a number of \texttt{Business} apps (e.g., Tencent Meetings and Zoom Cloud Meetings) and \texttt{Education} apps (e.g., Tencent Classroom and Google Classroom) jumped to the top of the list.
This started in early February in China and mid-March in the US, just as the COVID-19 began to spread widely in the corresponding countries. This observation is quite straightforward since the outbreak had quickly led to a large-scale shift to remote work for employees and online learning for students. 
Besides, the \texttt{Travel} apps (e.g.,Ctrip and Airbnb) and \texttt{Navigation} apps (e.g., Baidu Maps and Google Maps) show the opposite trend during the same time, with almost all of them falling out of the top 100. This is also reasonable because the mandatory social distancing and quarantines are required for containing virus transmission, which left such apps less desired.
It is important to note that the ranking changes in these four categories starts almost simultaneously with the outbreak, which demonstrates the pandemic's immediate impact. 
In sum, the findings indicate that mobile app ecosystems have had immediate reactions to the COVID-19 outbreak. Regarding app categories, COVID-19 has an effect of enhancing or decreasing the popularity of certain categories of apps.

Comparing the two countries, it is also interesting to note that \texttt{Navigation} and \texttt{Travel} apps in the US rebounded in mid-April and mid-May, respectively, while the US was still in the midst of an increasingly severe pandemic situation. In contrast, the rankings for these two categories in China improved only after the conditions began to improve. To some extent, this may reflect the different attitudes of the two countries in dealing with COVID-19.

\iffalse
\begin{figure*}[htb]
    \centering
    \begin{center}
        \subfigure{
        \includegraphics[width=0.98\textwidth]{Figures/legend.pdf}
        } 
        \vskip -0.15in
        \subfigure[China]{
        \includegraphics[width=0.48\textwidth]{Figures/category-cn.pdf}
        \label{fig:category-cn}}
        \subfigure[US]{
        \includegraphics[width=0.48\textwidth]{Figures/category-us.pdf}
        \label{fig:category-us}}
        \vspace{-0.1in}
        \caption{The distribution of number of apps in each category per day}
        \label{fig:category}
    \end{center}
\end{figure*}
\fi




\begin{figure*}
    \centering
    \subfigure[Business]{
    \label{fig:business}
    \includegraphics[width=0.48\textwidth]{Figures/business.pdf}
    }
    \subfigure[Education]{
    \includegraphics[width=0.48\textwidth]{Figures/education.pdf}}
    
    \subfigure[Travel]{
    \label{fig:travel}
    \includegraphics[width=0.48\textwidth]{Figures/travel.pdf}}
    \subfigure[Navigation]{
    \includegraphics[width=0.48\textwidth]{Figures/navigation.pdf}}
    \caption{Evolution of the daily ranking of apps and the number of diagnoses in four categories (Business, Education, Travel and Navigation) in two countries.}
    \label{fig:four-category}
\end{figure*}
background.tex
\section{Background and Related Work}
\label{sec:background}

\subsection{COVID-19 Pandemic in China and the United States}

We start by describing the development of COVID-19 pandemic and contrasting the responses of China and the US. 

\subsubsection{COVID-19 in China}

The first case of COVID-19 was identified in Wuhan, China, in December 2019.
In the face of a previously unknown virus, China rolled out perhaps the most ambitious, agile, and aggressive disease containment effort in history~\cite{whoreport}. The central and local governments launched the national emergency response and various prevention and control measures have been implemented rapidly. Investigations began quickly and traced the outbreak to a seafood market, which was immediately closed by Chinese authorities as an initial method to terminate all meat trades. 

In response to the rapid spread of SARS-Cov-2 within Hubei province, the Chinese government expanded its precautionary measures, announcing a complete lockdown of Wuhan and Hubei province cities by closing airports, railway stations, highway entrances, and suspending all local public transportation to prevent anyone from entering or leaving~\cite{10.1093/jtm/taaa036}. The government also implemented aggressive quarantine and social distancing in the entire country including cancelling activities with large crowds, wearing a mask when going out, postponing the reopening of schools, enabling home office, etc. After all the measures taken and people’s full commitment, a decline in the number of new cases and deaths was clearly observed by the end of February. Later on,  with all the situation improvements happening, China announced the lifting of the embargo and travel restrictions on Wuhan and restarting of the economy in April~\cite{CNBC}.

\subsubsection{COVID-19 in US}

The first case of COVID-19 in the US was reported on January 20, 2020~\cite{doi:10.1056/NEJMoa2001191} and the first known American deaths occurred in February~\cite{CNN}. President Donald Trump declared the US outbreak a public health emergency on January 31. Subsequently, restrictions were placed on flights arriving from China~\cite{public-health-emergency,travel-claim}. However, the initial US response to the outbreak was otherwise slow in terms of preparing the medical system, halting other travel, and testing~\cite{apnews,politico,theguardian}. Meanwhile, Trump downplayed the threat posed by the virus and claimed the outbreak was under control~\cite{trump-timeline}. In addition, each state that had imposed a stay-at-home order or shelter in place had begun lifting the restrictions of businesses and public spaces in May~\cite{reopening}. 
Despite its considerable advantages --- abundant resources, biomedical infrastructure, and scientific expertise --- the US has been severely hit by COVID-19. More than 21 million confirmed cases have been reported by the time of writing, resulting in more than 368K deaths, which is the most of any country and the 14th highest on a per capita basis~\cite{jhu-csse,jhu}.


\iffalse
Figure~\ref{fig:trends} shows the trends in the number of new confirmed cases per day in the two countries, from which we can see a shape contrast.

\begin{figure}[htbp]
    \centering
    \subfigure[China]{
    \includegraphics[width=0.48\textwidth]{Figures/cn-trend.pdf}}
    \subfigure[US]{
    \includegraphics[width=0.48\textwidth]{Figures/us-trend.pdf}}
    \vspace{-0.15in}
    \caption{The trends in the number of daily new cases in China and the US.}
    \label{fig:trends}
\end{figure}
\fi

\subsection{App Ranking in iOS App Store}
App ranking refers to the position of an app in a store or marketplace. In general, app rankings can reference not only how an app ranks in a search, but also the overall app rankings from the store. These rankings are country specific, and can be viewed at an overall or category level. The Apple App Store, one of the most common app marketplaces, provides the real-time app ranking lists both overall and by category. In the world of apps, ranking is of utmost importance. A minor change in rank can make a significant difference in traffic and revenue. Apptentive's recent mobile consumer survey~\cite{survey} reveals that nearly half of all mobile app users identified browsing the app store charts and search results (the placement on either of which depends on rankings) as a preferred method for finding new apps. Simply put, higher rankings mean more downloads, and to some degree, more money for app developers.


The Apple iOS App Store has a complex and highly guarded algorithm for determining rankings for both keyword-based app store searches and composite top charts. Although the exact ranking algorithms are not publicly available, there are several known factors that influence the rankings, including downloads (how many people have downloaded the app), rating (how many stars people give the app), rating count (how many ratings the app has) and trends (how quickly the app is growing), etc~\cite{factors}.
\textit{Since it is the combination of several other factors (average rating, number of ratings, installs, etc.), this ranking speaks to the overall awareness, usefulness, and satisfaction of an app. Consequently, app ranking is a particularly useful metric for mobile app analysis, particularly, its popularity.}



\subsection{Related Studies on COVID-19}
Since its outbreak, COVID-19 has attracted much attention from various research communities. A large number of studies are focused on the medical domain. Many medical researchers have made tremendous contributions to  pathology study, epidemiology study, treatment study and so on~\cite{bai2020presumed,zu2020coronavirus,onder2020case,shen2020treatment}.
Also, a number of computer scientists have adopted computing techniques like machine learning to help medical practitioners cope with the disease~\cite{latif2020leveraging}. 
For example, Zhang et al.~\cite{zhang2020covid} proposed the confidence-aware anomaly detection (CAAD) model to screen viral pneumonia on chest X-ray images. Wang et al.~\cite{wang2020covid} proposed COVID-Net, a deep convolutional neural network design tailored for the detection of COVID-19 cases from chest X-ray (CXR) images.

There are a growing number of mobile app studies relevant to COVID-19. Ahmed et al.~\cite{9144194} provided the first comprehensive review of contact tracing apps and discussed the concerns users have reported regarding their usage.
Oliver et al.~\cite{oliver2020mobile} described how mobile phone data can guide government and public health authorities in determining the best course of action to control the COVID-19 pandemic and in assessing the effectiveness of control measures.
He et al.~\cite{he2020beyond} presented a systematic analysis of coronavirus-themed mobile malware and found these apps aim to steal users’ private information or to make a profit by phishing and extortion.
There is also research into security and privacy issues. For example, Sun et al.~\cite{sun2021empirical} proposed an automated assessment tool to determine security and privacy weaknesses for apps, and undertook a user study to investigate concerns regarding contact tracing apps. Sharma et al.~\cite{sharma2020advocating} assessed privacy controls offered in COVID-19 apps and users' preferences if they were to adopt a COVID-19 app. Wang et al.~\cite{wang2020market} reveals various issues related to contact tracing apps from the users' perspective by analyzing a large number of user reviews.
Although a few previous studies have looked at COVID-19 through the lens of mobile apps, to the best of our knowledge, our work is the first comprehensive study to investigate mobile users' reactions to the COVID-19 pandemic through the lens of app ranking.


\subsection{Related Work on the Mobile App Ecosystem}
A large number of research studies have characterized the mobile app ecosystem from different perspectives. Wang et al.~\cite{wang2019understanding} characterized the evolution of Google Play based on 5.3 million app records collected from three snapshots of Google Play over three years. They observed that although the overall ecosystem shows promising
progress, there still exists a considerable
number of unsolved security issues. Besides, some studies were focused on app developers~\cite{wang2017explorative,wang2019characterizing,wang20203}, app market maintenance behaviors~\cite{RMVDroid,wang2018android,wang2018beyond}, app clone detection~\cite{li2019rebooting,li2019identifying,wang2015using}, third-party libraries~\cite{zhan2020automated,libradar,dong2018mobile,wang2017understanding,xu2020dissecting}, security~\cite{zhou2012dissecting,maddroid,liu2019dapanda,dong2018frauddroid,hu2020mobile,zhou2020demystifying,tang2020all,wang2019signing,hu2018dating} and privacy issues~\cite{razaghpanah2018apps,reardon201950,xi2019deepintent,wang2017understanding,liu2016identifying,wang2015reevaluating} of the ecosystem, based on app binary analysis, UI analysis, app traffic analysis, and app metadata analysis, etc.
This paper, investigates the mobile app ecosystem from a novel perspective, to understanding how the dynamic trajectory of the COVID-19 pandemic shapes mobile smartphone users' experiences.
discussion.tex
\section{Discussion}


\subsection{Implications}
The most significant result in our paper is that COVID-19 has had a significant impact on the app rankings in app store. This impact is reflected in both the popularity of several categories and the rankings of individual apps. Moreover, the evolution of app rankings are diverse and have a lot to do with whether the features and functionalities of the apps are compatible with the current environment. Besides, we notice that some apps that rapidly rise in popularity may also experience the side effect of rating drops. This reflects the challenges that rapid changes in popularity can cause app developers. We also note that findings are different between China and the United States.

The relevant stakeholders should take note of these findings, as they play different roles at different levels of the app market. Our findings have further implications for understanding the behavior of the mobile app ecosystem during public health crises, and helps developers to make better decisions on app developmennt and management. As our first finding indicates, the popularity of apps may experience disturbances resulting from public events. Such disturbances demonstrate the importance of having a contingency plan. When some sudden public events such as the COVID-19 pandemic happens, a contingency plan could help app developers, operators, as well as managers better cope with them. For example, one may develop a quick server deployment plan when expecting high demand. App developers may diversify their portfolios to avoid potential loss caused by popularity decreases. The second finding suggests that the impact may be complicated, depending on how the event is going on. Thus, app developers may need to develop the ability to develop the short-term and long-term insights, and plan accordingly. For example, for an app that gains popularity at the early phase of the pandemic but gradually return to normal later, app developers need to decide if it is necessary to invest resources for the short-term spikes. We also notice that increases of app popularity often coincide with decreases in ratings, and growing number of reviews containing users' insights. App developers thus need to be vigilant to users' feedback, and take this opportunities to improve the app quality. Moreover, our results and findings are not restrict to a specific event; it could be helpful in other public emergencies at the global scale.

.

\subsection{Limitations}
We recognize that our study carries several limitations and threats to validity.
First, since the main impact of the pandemic was reflected on only a small number of apps, we selected the top 100 apps each day, yet there was no accepted standard for selecting this number. 
Second, in determining the strength of the correlation, we set a threshold value of 0.6. However, there is no standard definition for the selection of this threshold. 
Third, only a small number of apps have been used for our review analysis. The major reason is that many apps have insignificant side effects. Hence, we are concerned that reviews of such apps are not representative and reliable enough.


\section{Conclusion}
This paper has presented the first longitudinal empirical study of the evolution of app rankings during COVID-19. Our analysis covers 586 apps in China and 590 apps in the US with records of being ranked in the top 100 from January 1 to June 30, 2020, across 22 categories. We performed analysis from the perspectives of category popularity. To further understand the evolution patterns of app rankings, we proposed a correlation-based method to classify them and then perform a per category analysis. Besides, we characterized the side effect of declining ratings accompanied by the rising popularity of some apps. Our observations reveal mobile users' reactions in the mobile ecosystem in the face of unexpected public crises, and provide insights for app developers to make better decisions on developing apps.


intro.tex
\section{Introduction}


The ongoing COVID-19 pandemic has impacted almost every aspect of human life. The pandemic introduced changes in people's perceptions and attitudes, as well as how they work, pursue leisure, interact with others, and so on~\cite{bussing2020tumor,dwivedi2020impact,kramer2020potential}. Among these changes, interacting with internet and information technologies could be very significant. Compared with prior years, technologies have played a critical role in the pandemic, as an essential infrastructure for supporting the response to the pandemic. For instance, to help curb its spread, governments and public health authorities around the world have launched contact-tracing apps~\cite{cho2020contact,ahmed2020survey}. Given that mobile phones have been the most popular and pervasive interfaces for people to interact with the Internet, most technological experience has occurred via mobile apps. We therefore posit that \textit{the popularity of mobile apps, as well as their dynamics, may provide a lens for understanding people's experiences with technologies during the pandemic.} 

Due to the different strategies employed by countries in fighting COVID-19, as well as the degree of public compliance, the regional development of the pandemic has exhibited distinct patterns. For example, China introduced a strict nation-wide lockdown during the initial outbreak and achieved strong public cooperation, making the pandemic largely under control in most of areas from April~\cite{altakarli2020china}. In contrast, the United States, although implementing multiple measures at different levels, failed to contain the pandemic, resulting in chaos spanning the entire year~\cite{CasesinUS}. We conjecture that such differences may be reflected in the app popularity in different countries. \textit{Thus, understanding people's experiences with mobile apps must consider the country-specific patterns of the pandemic.} 


\textbf{This Work.}
In this paper, we seek to understand, characterize, and compare the dynamics of app popularity during the pandemic, under the circumstance of different country-specific pandemic development patterns.
We are particularly keen to explore the relationship between app utilization and the infection trajectories experienced across countries. 
We first create the daily snapshots of the ranking of popular apps in the iOS App Store in both China and the United States (US), from the January 2020 to the end of June 2020 (\textbf{See Section~\ref{sec:design}}). 
This longitudinal dataset enables us to study app usage behavior evolution across billions of mobile users in both China and US.
We next characterize the dynamics of app popularity from both a Macro- (i.e., app category) and Micro- (i.e., individual app) lens. 
We investigate which app categories are positively or negatively correlated with the coronavirus pandemic and try to formulate general laws in their ranking evolution (\textbf{See Section~\ref{sec:category}}), especially when comparing the differing situations and cultures in China and the US. 
We further perform fine-grained per-app analysis (\textbf{See Section~\ref{sec:perapp}}), to summarize the dynamics of app popularity.
Finally, we investigate whether the COVID-19 pandemic has introduced side-effects on the app maintenance behaviors (e.g., performance) by analyzing millions of app reviews (\textbf{See Section~\ref{sec:sideeffect}}).
We highlight a number of key findings:

\begin{itemize}
    \item \textit{COVID-19 has played a key role in changing the popularity of some categories of app}. During the outbreak, several app categories experienced significant ups and downs in popularity in both China and the US, particularly \texttt{Business}, \texttt{Education}, \texttt{Navigation}, and \texttt{Travel}. This change reflects the reactions and efforts people made in response to the outbreak, and highlights differences between China and the US.
    
    \item \textit{The impact of COVID-19 on app rankings are diverse and can vary from country to country}. There are many apps in our dataset whose rankings are strongly correlated, either positively or negatively, with the pandemic case rates. Their ranking variations can be classified into several groups. Most apps within the same group share some similarities in their adaptation to the pandemic situation.
    
    \item \textit{The rapid popularity of some apps may be accompanied by the side effect of declining ratings, especially for Chinese apps}. We explore the reasons for this from the app reviews and found a massive increase in the number of negative reviews compared to the pre-popularity. We reveal some potential challenges for apps.
\end{itemize}

To the best of our knowledge, this is the first longitudinal study to characterize the behavior dynamics of mobile users through the lens of mobile app popularity. We have released our dataset and the experimental results to the research community at Github:

\begin{center}
    \url{https://app-popularity-covid19.github.io/}
\end{center}


side-effect.tex
\section{Characterizing Side Effects}
\label{sec:sideeffect}


We conclude by briefly inspecting side effects that may be caused by these rapid fluctuations in app popularity. Particularly, an interesting phenomenon we observe is that the ratings of some apps are likely to decrease in varying degrees as their rankings go up.
For example, Figure~\ref{fig:rank-rate-example} shows the trends in ranking and rating of the two app examples over this period. 
While the ranking and popularity of the app is rising rapidly, its rating is experiencing a decline.
This prompts us to investigate whether the rise in app ranking and popularity may have some side effects on app maintenance behavior.

\begin{figure}[htbp]
    \centering
    \subfigure[ViLin]{
    \includegraphics[width=0.4\textwidth]{Figures/rank-rate-vilin.pdf}}
    \quad
    \subfigure[Tencent Meeting]{
    \includegraphics[width=0.4\textwidth]{Figures/rank-rate-tm.pdf}}
    \vspace{-0.15in}
    \caption{Two app examples where the rating goes down as the ranking goes up.}
    \label{fig:rank-rate-example}
\end{figure}

\subsection{Ratings Analysis}
We briefly focus on those apps that gained popularity for a short or long time as COVID-19 hit, exploring the relationship between their ratings and rankings.
Considering that there is a wide diversity in the evolution of many app rankings throughout the time span, we shorten the time frame to the period when COVID-19 was first spreading, and target only those apps whose rankings rose rapidly during this period.

Specifically, for \emph{China}, we choose the period when the ranking exhibits strong negative correlation with the number of active confirmed cases during the worsening of the COVID-19, i.e., from January to mid-February. 
For the \emph{US}, we pay attention to the two months just after the outbreak of COVID-19, i.e., from March to May.

We then calculate the correlation coefficients between the ratings and rankings of the focused apps. 
The results are presented in Table~\ref{tab:rank-rate}. We see that there were 95 apps in China and 88 apps in the US that improved their rankings significantly at the beginning of the COVID-19 outbreak. Of these, 63 apps in China and 24 apps in the US have a correlation coefficient \emph{r}>0 between rating and ranking.
This means that their ratings decreased as their ranking rises, i.e., introducing side effects. 
Curiously, the results show that this particle side effect is more prevalent in China than the US. 

\begin{table}[htbp]
    \centering
    \caption{Distribution of correlation coefficient between rankings and ratings for focused apps.}
    \begin{tabular}{c|c|c|c}
    \toprule
      Country  & \# Apps & \# r<0 (p<0.05) & \# r>0 (p<0.05)\\
      \hline
      China & 95 & 29 & 63 \\
      US & 88 & 60 & 24\\
      \bottomrule
    \end{tabular}
    \label{tab:rank-rate}
\end{table}

\subsection{Reviews Analysis}
We have shown that some apps that gains popularity while experiencing a decline in their ratings. 
We next dive into the reasons behind this phenomenon and investigate the challenges for app developers.  
Previous studies~\cite{carreno2013analysis,pagano2013user} showed that reviews represent a rich source of information for app developers, such as user requirements, bug reports, feature requests, and documentation of user experiences. Thus, we look at user reviews to analyze the possible reasons for the drop in app ratings. 
It is important to note that while the ratings of all of these apps have dropped, the magnitude of the drop varies greatly, with many showing only slight drops.
For the sake of representativeness, we target apps where the correlation coefficient between app rankings and ratings is greater than 0.6 and the absolute value of the range of rating decline is greater than 0.1. This leaves 37 apps for review analysis.

We use AppBot to extract the app reviews. This tool provides a large number of data-mining and sentiment analysis features, and is used in many studies, e.g.,~\cite{garousi2020mining}.
For each app, we use AppBot for automated text mining and sentiment analysis of reviews.
To uncover new traits, we use reviews from the second half of 2019 as the benchmark for comparision.  

Overall, we find a significant increase in the number of reviews for these apps in the first half of 2020 compared to the second half of 2019. This is intuitive considering their increased demand. A total of 1,355,460 reviews occur in the first half of 2020, which is 4.3 times more than the 313,209 reviews in the previous six months.
More importantly, sentiment analysis reveals that the number and percentage of negative reviews among them has also increased considerably, with a total of 716,129 negative reviews in the first half of 2020 (53\%) compared to 72,345 in the second half of 2019 (23\%), a visible number about 10 times higher. 
This is also the case for individual apps. For example, \textit{Tencent Meeting} had only 33 user reviews during the second half of 2019, of which only 5 (15\%) were negative, while the number of reviews during the first half of 2020 was astonishingly high at 10,507, of which 4,583 (44\%) were negative.
This may mean that such apps have struggled to fulfil user expectations as their user base has grown. 

Further, we focus on the negative reviews and seek to get a sense of the topics of user complaints. 
We thereofre rely on AppBot's topic grouping feature and identify six core topics raised within these reviews:


(1) \emph{Bugs.} There are lots of negative reviews describing problems with the app which should be corrected, such as a crash, an erroneous behavior, or a performance issue. This is likely a product of more users experimenting with the features of the apps.

(2) \emph{Device.} There are many negative comments regarding the device, including device incompatibility, interface mismatch, lack of matching function and device features, unfriendly to ipad devices, triggering device heat and lag, etc. Again, such issues become more prevalent as your user base diversifies. 

(3) \emph{Connectivity.} This mainly refers to network connection problems, such as difficult WiFi connections, slow loading, long waiting times, etc.

(4) \emph{Design \& UX.} There are some negative reviews about the user interface, specifically that the user interface is not attractive, and the UI design and interaction are not friendly enough, making the user experience poor.

(5) \emph{Video/Audio.} For some apps, there are some problems with opening video or voice, such as delay, not smooth, etc. Usually such complaints arise in apps featuring multi-person online real-time sessions, such as online meetings and online lecture apps.

(6) \emph{Privacy.} There are also some concerns about privacy and security of some apps, such as invasion of privacy, sharing information with third-parties, stealing edits and tracking location, etc.

\begin{figure}
    \centering
    \includegraphics[width=0.9\textwidth]{Figures/app-review.pdf}
    \caption{Distribution of the six negative review topics in each app.}
    \label{fig:app-review}
\end{figure}

\iffalse
\begin{table}[htbp]
    \centering
    \small
    \caption{Topics of negative reviews for each app.}
    \resizebox{0.98\textwidth}{!}{
    \begin{tabular}{|c|c|c|ccccc|}
    \hline\noalign{\smallskip}
      App Name & Country & Category & \multicolumn{5}{|c|}{Negative Review Topics} \\ 
      &&& Bugs & Devices & Connectivity & Design\&UX & Video/Audio \\
      \noalign{\smallskip}\hline\noalign{\smallskip}
      ClassIn & China & Education & $\surd$ & $\surd$ & $\surd$ &  & \\
      WeCom & China & Business & $\surd$ & $\surd$ & $\surd$ & $\surd$ & $\surd$ \\
      Tencent Meeting & China & Business & $\surd$ & $\surd$ & $\surd$ & $\surd$ & $\surd$ \\
      DingTalk & China & Business & $\surd$ & $\surd$ & $\surd$ & $\surd$ & $\surd$ \\
      WanShiTong & China & Productivity & $\surd$ & & $\surd$ & $\surd$ & \\
      MinZhengTong & China & Lifestyle & $\surd$ & & $\surd$ & $\surd$ &  \\
      Tencent Class & China & Education & $\surd$ & $\surd$ & $\surd$ & $\surd$ & $\surd$ \\
      ZhangShangHuaYi & China & Education & $\surd$ & $\surd$ & $\surd$ & & \\
      ViLin & China & Business & $\surd$ & & $\surd$ & & \\
      Douyu Live & China & Entertainment & $\surd$ & $\surd$ & $\surd$ & $\surd$ & \\
      YiQiXue & China & Education & $\surd$ & $\surd$ & & $\surd$ & \\
      JiLiGuaLa & China & Education & $\surd$ & $\surd$ & $\surd$ & $\surd$ & \\
      YongHui Life & China & Shopping & $\surd$ & & & & \\
      King's Glory Assistant & China & Game & $\surd$ & $\surd$ & $\surd$ & $\surd$ & \\
      Health ShenZhen & China & Medical & $\surd$ & & & & \\
      Onion OMALL & China & Shopping & $\surd$ & $\surd$ & $\surd$ & & \\
      ChangAnShuYuan & China & Education & $\surd$ & $\surd$ & & & \\
      XiaoMiYouPin & China & Shopping & $\surd$ & $\surd$ & $\surd$ & & \\
      ZhiDao & China & Education & $\surd$ & $\surd$ & $\surd$ & $\surd$ & \\
      XiaoHe Student & China & Education & $\surd$ & $\surd$ & $\surd$ & & \\
      Feishu & China & Business & $\surd$ & $\surd$ & $\surd$ & & \\
      My ChangZhou & China & Lifestyle & $\surd$ & $\surd$ & $\surd$ & $\surd$ & \\
      JinLingWangZheng & China & Utility & $\surd$ & $\surd$ & $\surd$ & $\surd$ & \\
      DaXiang News & China & News & $\surd$ & $\surd$ & $\surd$ & & \\
      LianGongBao & China & Education & $\surd$ & & & $\surd$ & \\
      WPS Office & China & Productivity & $\surd$ & $\surd$ & & $\surd$ & \\
      Tencent WeCloud & China & Productivity & & & $\surd$ & & \\
      Zoom Cloud Meetings & China,US & Business & $\surd$ & $\surd$ & $\surd$ & $\surd$ & $\surd$ \\
      Fitness Coach & US & Health\&Fitness & $\surd$ & $\surd$ & $\surd$ & $\surd$ & $\surd$ \\
      Google Meet & US & Business & $\surd$ & $\surd$ & $\surd$ & $\surd$ &  $\surd$ \\
      HBO Max & US & Entertainment & $\surd$ & $\surd$ & $\surd$ & $\surd$ & $\surd$ \\
      Vinkle & US & Photo\&Video & $\surd$ & $\surd$ & $\surd$ & $\surd$ & $\surd$ \\
      Google Classroom & US & Education & $\surd$ & $\surd$ & $\surd$ & $\surd$ & $\surd$ \\
      Life360 & US & Social & $\surd$ & $\surd$ & $\surd$ & $\surd$ & $\surd$ \\
      Flipgrid & US & Education & $\surd$ & $\surd$ & $\surd$ & $\surd$ & $\surd$ \\
      \noalign{\smallskip}\hline
    \end{tabular}
    }
    \label{tab:review-topics}
\end{table}
\fi
We further measure the distribution of different kinds of topics for each app, as shown in Figure~\ref{fig:app-review}.
As can be seen, the distribution of user concerns varies greatly across apps, where bugs and devices issues are prevalent.
We also notice that some of the issues are related to the increase in the number of users. For example, \textit{Play It}, a multiplayer interactive trivia game app in China, suffered several crashes due to a surge in the number of users during the initial outbreak of COVID-19 in China as people were quarantined at home.
Generally speaking, as more people use the app, the number of complaints grows as well. Some issues that used to be obscure seem to be magnified, such as the aesthetics of the user interface.
This sudden popularity presents a great challenge to the app in many aspects including functionality, compatibility, stability, fault tolerance and UI design, etc.
In a way, it serves as a wake-up call for developers to learn from the experience in dealing with unexpected situations in the future. Meanwhile, it offers unprecedented opportunities for app developers to resolve the hidden problems.


studydesign.tex
\section{Study Design}
\label{sec:design}


\subsection{Research Questions}
Our study is driven by the following research questions:


\begin{itemize}
\item[RQ1] \textbf{General Trends.}
\emph{Across different regions, what are the characteristics of app popularity during COVID-19, and how are they impacted by regional measures?}
%     We seek to study the popularity (i.e., the number of top-ranked apps) of different app categories, understand which app categories are more competitive over the outbreak and explore their general laws in ranking evolution.
    
%     \item[RQ2] 
%     \textit{What is the correlation between app ranking trends and the evolution of the pandemic situation?}
%     Considering that the outbreak may affect people's usage of mobile apps, it is thus interesting to investigate what kinds of apps are positively or negatively correlated with the coronavirus pandemic.
    
\item[RQ2] \textbf{Behavior Patterns.}
\textit{What are the different patterns in the change of app popularity over the course of the pandemic?} 
We attempt to classify the app ranking variations according to relevance to the pandemic, then examine the distribution of the number of apps in each class and explore their characteristics.



\item[RQ3] \textbf{Side-effect.}
%The COVID-19 pandemic has drastically changed working and social habits for billions of people, which could be reflected on the mobile app usage.
\textit{Considering that app popularity may change greatly during the COVID-19 pandemic, does it introduce any side-effect on app maintenance behaviors?} 

\end{itemize}


\subsection{Data Collection}

We first harvest a dataset of app popularity. Given that the most straightforward impact of COVID-19 on the mobile app ecosystem will be mirrored in a number of popular apps, we leverage the App Store's ranking list and focus on the top-ranked apps. However, it is non-trivial to obtain such a dataset as it requires effort to continuously monitor apps in the app market to check their rankings. To this end, we implement a crawler to retrieve the daily app ranking list from iOS app store. Considering that the pandemic in different countries can evolve in different ways, we take two representative countries (China and the US), as case studies for comparison. %to make our characterization more robust.

To be specific, we crawl the ranking list with top 1500 apps each day from January 1 to June 30, 2020, to keep track of the apps and their daily rankings, containing over 543K records in total for the two countries. We then take the top 100 apps per day as the target of this study. 
Overall, there are 586 unique apps in China and 590 apps in the US covering 22 categories, each of which ranked in the top 100 for at least one day during a given half year.
Besides, we collect ratings for each app on a daily basis, as well as the reviews for some of these apps. 

Table~\ref{tab:dataset} summarizes our dataset.

\begin{table}[htbp]
    \centering
    \caption{Overview of our dataset (from Jan 1 to June 30, 2020).}
    \begin{tabular}{c|ccccc}
    \toprule
     Country & \# Categories & \# Top100 Unique Apps & \# Ranking Records & \# Ratings & \# Reviews \\
     \hline
     China & 22 & 587 & 271,765 & 106,653 & 980,573 \\
     US & 22 & 590 & 271,522 & 107,381 & 756,617 \\
     \bottomrule
    \end{tabular}
    \label{tab:dataset}
\end{table}

\subsection{Analysis Preliminaries}

We briefly present the methodologies used in the subsequent sections.

\subsubsection{Popularity of app categories.}
To understand which app categories are more competitive and explore general laws in ranking evolution, we begin our analysis by investigating the most intuitive metric of app-category popularity, i.e., the number of apps ranked in the top 100 for each category. 
Specifically, we measure the popularity of category by app volume, i.e., the number of apps belonging to that app category in the top-100. The larger the number, the more popular the category is. We examine the number of apps in each category on a daily basis to gain insight into how their popularity trends are changing. In more detail, we also zoom in on each category to check the ranking variations of all apps in that category.

\subsubsection{Correlation analysis.}
To understand if any apps have been significantly affected by the pandemic, and whether the impact has been positive or negative, we also study the correlation between the pandemic evolution and the app ranking changes, which could reflect the popularity changes of apps, for each app.
In our case, we measure the development of the pandemic situation over the half year through two metrics, i.e., the number of daily new confirmed cases and daily active cases, respectively. Moreover, we present a correlation comparison strategy based on Pearson Correlation Coefficient~\cite{pearson}. The Pearson Correlation Coefficient is a number that summarizes the direction and closeness of linear relations between two variables, taking a value between -1 and 1. The closer the correlation coefficient is to 1 (-1), the stronger the positive (negative) correlation is.


For each app, the daily ranking is viewed as a vector in Pearson Correlation Coefficient formula, then the correlation degree is obtained separately from the daily number of newly diagnosed cases and active cases in the corresponding countries. 
Note that we also take P-value into account because it tells us whether the result of an experiment is statistically significant. Typically if the P-value is lower than the conventional 5\% (P<0.05), the correlation coefficient is called statistically significant. Thus, results with P-value less than 0.05 are of concern to us, otherwise are ignored as insignificantly correlated.

Finally, we calculate the correlation coefficient for each app and obtained 358(61.1\%) apps in China and 497(84.2\%) apps in the US with statistically significant correlation with the number of daily new cases in respective countries, as well as 425(72.5\%) apps in China and 502(85.1\%) apps in the US with statistically significant correlation with daily active cases.
The results illustrate that the majority of apps have a statistically significant correlation between their rankings and the COVID-19 case rates.
This encourages and motivates the following analysis, i.e., the distributions of these correlation coefficients at the overall and category level. Due to the greater number of apps with statistically significant correlation between rankings and active cases compared to newly confirmed cases, we take the active cases as the pandemic marker for the subsequent study. And note that all the correlation coefficients provided below are statistically significant.


\iffalse
\begin{table}[htb]
    \centering
    \caption{The number of apps whose ranking has a significant correlation coefficient with the daily new cases or daily active cases}
    \vspace{-0.1in}
    \resizebox{0.7\textwidth}{!}{
    \begin{tabular}{l|c|c}
    \toprule
         & \# with Daily New cases & \# with Daily Active cases \\
         \midrule
    China & 358 (61.1\%) & 425 (72.5\%) \\
    \hline
    US & 497 (84.2\%) & 502 (85.1\%) \\
    \bottomrule
    \end{tabular}}
    \label{tab:number}
\end{table}
\fi
00-main.bbl
\begin{thebibliography}{10}

\bibitem{CNBC}
{China to lift lockdown on Wuhan, the epicenter of its coronavirus outbreak.
  CNBC}.
\newblock
  https://www.cnbc.com/2020/03/24/china-to-lift-lockdown-on-wuhan-city-epicenter-of-coronavirus-outbreak.html.

\bibitem{jhu-csse}
{Coronavirus COVID-19 (2019-nCoV). Johns Hopkins CSSE.}
\newblock https://www.arcgis.com/apps/opsdashboard/index.html\#/bda
  7594740fd40299423467b48e9ecf6.

\bibitem{jhu}
{Mortality Analyses. Johns Hopkins University.}
\newblock https://coronavirus.jhu.edu/data/mortality.

\bibitem{factors}
{Ranking Factors}.
\newblock
  https://moz.com/blog/app-store-rankings-formula-deconstructed-in-5-mad-science-experiments?\_ga=2.18395834.2057695595.1604471809-802527328.1604471809.

\bibitem{whoreport}
{Report of the WHO-China Joint Mission on Coronavirus Disease 2019 (COVID-19).}
\newblock
  https://www.who.int/docs/default-source/coronaviruse/who-china-joint-mission-on-covid-19-final-report.pdf.

\bibitem{apnews}
{Signs missed and steps slowed in Trump’s response}.
\newblock https://apnews.com/article/6a8f85aad99607f313cca6ab1398e04d.

\bibitem{CNN}
{The first known US coronavirus-related fatality. CNN}.
\newblock
  https://www.cnn.com/2020/04/23/us/california-woman-first-coronavirus-death/index.html.

\bibitem{theguardian}
{The missing six weeks: how Trump failed the biggest test of his life}.
\newblock
  https://www.theguardian.com/us-news/2020/mar/28/trump-coronavirus-politics-us-health-disaster.

\bibitem{reopening}
{This is where each state is during its phased reopening. CNN}.
\newblock
  https://www.cnn.com/interactive/2020/us/states-reopen-coronavirus-trnd/.

\bibitem{trump-timeline}
{Timeline: The more than 60 times Trump has downplayed the coronavirus threat}.
\newblock https://web.archive.org/web/20200811160119if\_/https://www.washing
  tonpost.com/politics/2020/03/12/trump-coronavirus-timeline/.

\bibitem{CasesinUS}
{Tracking Covid-19 cases in the US}.
\newblock
  https://www.cnn.com/interactive/2020/health/coronavirus-us-maps-and-cases/.

\bibitem{public-health-emergency}
{Trump Declares Coronavirus A Public Health Emergency And Restricts Travel From
  China. NPR.}
\newblock
  https://www.npr.org/sections/health-shots/2020/01/31/801686524/trump-declares-coronavirus-a-public-health-emergency-and-restricts-travel-from-c.

\bibitem{politico}
{Trump halts funding to World Health Organization}.
\newblock
  https://www.politico.com/news/2020/04/14/trump-world-health-organization-funding-186786.

\bibitem{travel-claim}
{Trump’s Snowballing China Travel Claim}.
\newblock
  https://www.factcheck.org/2020/04/trumps-snowballing-china-travel-claim/.

\bibitem{survey}
Apptentive.
\newblock
  https://www.apptentive.com/blog/2020/02/04/mobile-app-ratings-and-reviews,
  2020.

\bibitem{pearson}
{Pearson correlation coefficient}.
\newblock https://en.wikipedia.org/wiki/Pearson\_correlation\_coefficient,
  2020.

\bibitem{9144194}
N.~{Ahmed}, R.~A. {Michelin}, W.~{Xue}, S.~{Ruj}, R.~{Malaney}, S.~S.
  {Kanhere}, A.~{Seneviratne}, W.~{Hu}, H.~{Janicke}, and S.~K. {Jha}.
\newblock A survey of covid-19 contact tracing apps.
\newblock {\em IEEE Access}, 8:134577--134601, 2020.

\bibitem{ahmed2020survey}
Nadeem Ahmed, Regio~A Michelin, Wanli Xue, Sushmita Ruj, Robert Malaney,
  Salil~S Kanhere, Aruna Seneviratne, Wen Hu, Helge Janicke, and Sanjay~K Jha.
\newblock A survey of covid-19 contact tracing apps.
\newblock {\em IEEE Access}, 8:134577--134601, 2020.

\bibitem{altakarli2020china}
Nourah~S AlTakarli.
\newblock China’s response to the covid-19 outbreak: a model for epidemic
  preparedness and management.
\newblock {\em Dubai Medical Journal}, 3(2):44--49, 2020.

\bibitem{bai2020presumed}
Yan Bai, Lingsheng Yao, Tao Wei, Fei Tian, Dong-Yan Jin, Lijuan Chen, and
  Meiyun Wang.
\newblock Presumed asymptomatic carrier transmission of covid-19.
\newblock {\em Jama}, 323(14):1406--1407, 2020.

\bibitem{bussing2020tumor}
Arndt B{\"u}ssing, Jutta H{\"u}bner, Stefanie Walter, Wolfgang Gie{\ss}ler, and
  Jens B{\"u}ntzel.
\newblock Tumor patients{\'{}} perceived changes of specific attitudes,
  perceptions, and behaviors due to the covid-19 pandemic and its relation to
  reduced wellbeing.
\newblock {\em Frontiers in psychiatry}, 11, 2020.

\bibitem{carreno2013analysis}
Laura V~Galvis Carreno and Kristina Winbladh.
\newblock Analysis of user comments: an approach for software requirements
  evolution.
\newblock In {\em 2013 35th international conference on software engineering
  (ICSE)}, pages 582--591. IEEE, 2013.

\bibitem{cho2020contact}
Hyunghoon Cho, Daphne Ippolito, and Yun~William Yu.
\newblock Contact tracing mobile apps for covid-19: Privacy considerations and
  related trade-offs.
\newblock {\em arXiv preprint arXiv:2003.11511}, 2020.

\bibitem{dong2018frauddroid}
Feng Dong, Haoyu Wang, Li~Li, Yao Guo, Tegawend{\'e}~F Bissyand{\'e}, Tianming
  Liu, Guoai Xu, and Jacques Klein.
\newblock Frauddroid: Automated ad fraud detection for android apps.
\newblock In {\em Proceedings of the 2018 26th ACM Joint Meeting on European
  Software Engineering Conference and Symposium on the Foundations of Software
  Engineering}, pages 257--268. ACM, 2018.

\bibitem{dong2018mobile}
Feng Dong, Haoyu Wang, Li~Li, Yao Guo, Guoai Xu, and Shaodong Zhang.
\newblock How do mobile apps violate the behavioral policy of advertisement
  libraries?
\newblock In {\em Proceedings of the 19th International Workshop on Mobile
  Computing Systems \& Applications}, pages 75--80, 2018.

\bibitem{dwivedi2020impact}
Yogesh~K Dwivedi, D~Laurie Hughes, Crispin Coombs, Ioanna Constantiou, Yanqing
  Duan, John~S Edwards, Babita Gupta, Banita Lal, Santosh Misra, Prakhar
  Prashant, et~al.
\newblock Impact of covid-19 pandemic on information management research and
  practice: Transforming education, work and life.
\newblock {\em International Journal of Information Management}, 55:102211,
  2020.

\bibitem{evans1996straightforward}
James~D Evans.
\newblock {\em Straightforward statistics for the behavioral sciences.}
\newblock Thomson Brooks/Cole Publishing Co, 1996.

\bibitem{garousi2020mining}
Vahid Garousi, David Cutting, and Michael Felderer.
\newblock Mining user reviews of covid contact-tracing apps: An exploratory
  analysis of nine european apps.
\newblock {\em arXiv preprint arXiv:2012.13589}, 2020.

\bibitem{he2020beyond}
Ren He, Haoyu Wang, Pengcheng Xia, Liu Wang, Yuanchun Li, Lei Wu, Yajin Zhou,
  Xiapu Luo, Yao Guo, and Guoai Xu.
\newblock Beyond the virus: A first look at coronavirus-themed mobile malware.
\newblock {\em arXiv preprint arXiv:2005.14619}, 2020.

\bibitem{doi:10.1056/NEJMoa2001191}
Michelle~L. Holshue, Chas DeBolt, Scott Lindquist, Kathy~H. Lofy, John Wiesman,
  Hollianne Bruce, Christopher Spitters, Keith Ericson, Sara Wilkerson, Ahmet
  Tural, George Diaz, Amanda Cohn, LeAnne Fox, Anita Patel, Susan~I. Gerber,
  Lindsay Kim, Suxiang Tong, Xiaoyan Lu, Steve Lindstrom, Mark~A. Pallansch,
  William~C. Weldon, Holly~M. Biggs, Timothy~M. Uyeki, and Satish~K. Pillai.
\newblock First case of 2019 novel coronavirus in the united states.
\newblock {\em New England Journal of Medicine}, 382(10):929--936, 2020.
\newblock PMID: 32004427.

\bibitem{hu2020mobile}
Yangyu Hu, Haoyu Wang, Ren He, Li~Li, Gareth Tyson, Ignacio Castro, Yao Guo,
  Lei Wu, and Guoai Xu.
\newblock Mobile app squatting.
\newblock In {\em Proceedings of The Web Conference 2020}, pages 1727--1738,
  2020.

\bibitem{hu2018dating}
Yangyu Hu, Haoyu Wang, Yajin Zhou, Yao Guo, Li~Li, Bingxuan Luo, and Fangren
  Xu.
\newblock Dating with scambots: Understanding the ecosystem of fraudulent
  dating applications.
\newblock {\em arXiv preprint arXiv:1807.04901}, 2018.

\bibitem{kramer2020potential}
Amit Kramer and Karen~Z Kramer.
\newblock The potential impact of the covid-19 pandemic on occupational status,
  work from home, and occupational mobility, 2020.

\bibitem{latif2020leveraging}
Siddique Latif, Muhammad Usman, Sanaullah Manzoor, Waleed Iqbal, Junaid Qadir,
  Gareth Tyson, Ignacio Castro, Adeel Razi, Maged N~Kamel Boulos, Adrian
  Weller, et~al.
\newblock Leveraging data science to combat covid-19: A comprehensive review.
\newblock {\em IEEE Transactions on Artificial Intelligence}, 2020.

\bibitem{li2019rebooting}
Li~Li, Tegawend{\'e}~F Bissyand{\'e}, and Jacques Klein.
\newblock Rebooting research on detecting repackaged android apps: Literature
  review and benchmark.
\newblock {\em IEEE Transactions on Software Engineering}, 2019.

\bibitem{li2019identifying}
Li~Li, Tegawend{\'e}~F Bissyand{\'e}, Hao-Yu Wang, and Jacques Klein.
\newblock On identifying and explaining similarities in android apps.
\newblock {\em Journal of Computer Science and Technology}, 34(2):437--455,
  2019.

\bibitem{liu2016identifying}
Minxing Liu, Haoyu Wang, Yao Guo, and Jason Hong.
\newblock Identifying and analyzing the privacy of apps for kids.
\newblock In {\em Proceedings of the 17th International Workshop on Mobile
  Computing Systems and Applications}, pages 105--110, 2016.

\bibitem{liu2019dapanda}
Tianming Liu, Haoyu Wang, Li~Li, Guangdong Bai, Yao Guo, and Guoai Xu.
\newblock Dapanda: Detecting aggressive push notifications in android apps.
\newblock In {\em 2019 34th IEEE/ACM International Conference on Automated
  Software Engineering (ASE)}, pages 66--78. IEEE, 2019.

\bibitem{maddroid}
Tianming Liu, Haoyu Wang, Li~Li, Xiapu Luo, Feng Dong, Yao Guo, Liu Wang,
  Tegawende~F. Bissyande, and Jacques Klein.
\newblock Maddroid: Characterising and detecting devious ad content for android
  apps.
\newblock In {\em Proceedings of the Web Conference 2020 (WWW'20)}, 2020.

\bibitem{libradar}
Ziang Ma, Haoyu Wang, Yao Guo, and Xiangqun Chen.
\newblock Libradar: fast and accurate detection of third-party libraries in
  android apps.
\newblock In {\em Proceedings of the 38th International Conference on Software
  Engineering Companion}, pages 653--656, 2016.

\bibitem{oliver2020mobile}
Nuria Oliver, Emmanuel Letouzé, Harald Sterly, Sébastien Delataille, Marco~De
  Nadai, Bruno Lepri, Renaud Lambiotte, Richard Benjamins, Ciro Cattuto,
  Vittoria Colizza, Nicolas de~Cordes, Samuel~P. Fraiberger, Till Koebe, Sune
  Lehmann, Juan Murillo, Alex Pentland, Phuong~N Pham, Frédéric Pivetta,
  Albert~Ali Salah, Jari Saramäki, Samuel~V. Scarpino, Michele Tizzoni,
  Stefaan Verhulst, and Patrick Vinck.
\newblock Mobile phone data and covid-19: Missing an opportunity?, 2020.

\bibitem{onder2020case}
Graziano Onder, Giovanni Rezza, and Silvio Brusaferro.
\newblock Case-fatality rate and characteristics of patients dying in relation
  to covid-19 in italy.
\newblock {\em Jama}, 323(18):1775--1776, 2020.

\bibitem{pagano2013user}
Dennis Pagano and Walid Maalej.
\newblock User feedback in the appstore: An empirical study.
\newblock In {\em 2013 21st IEEE international requirements engineering
  conference (RE)}, pages 125--134. IEEE, 2013.

\bibitem{razaghpanah2018apps}
Abbas Razaghpanah, Rishab Nithyanand, Narseo Vallina-Rodriguez, Srikanth
  Sundaresan, Mark Allman, Christian Kreibich, and Phillipa Gill.
\newblock Apps, trackers, privacy, and regulators: A global study of the mobile
  tracking ecosystem.
\newblock 2018.

\bibitem{reardon201950}
Joel Reardon, {\'A}lvaro Feal, Primal Wijesekera, Amit Elazari~Bar On, Narseo
  Vallina-Rodriguez, and Serge Egelman.
\newblock 50 ways to leak your data: An exploration of apps' circumvention of
  the android permissions system.
\newblock In {\em 28th $\{$USENIX$\}$ Security Symposium ($\{$USENIX$\}$
  Security 19)}, pages 603--620, 2019.

\bibitem{sharma2020advocating}
Tanusree Sharma, Tian Wang, and Masooda Bashir.
\newblock Advocating for users’ privacy protections: A case study of covid-19
  apps.
\newblock 2020.

\bibitem{shen2020treatment}
Chenguang Shen, Zhaoqin Wang, Fang Zhao, Yang Yang, Jinxiu Li, Jing Yuan,
  Fuxiang Wang, Delin Li, Minghui Yang, Li~Xing, et~al.
\newblock Treatment of 5 critically ill patients with covid-19 with
  convalescent plasma.
\newblock {\em Jama}, 323(16):1582--1589, 2020.

\bibitem{sun2021empirical}
R~Sun, W~Wang, M~Xue, G~Tyson, S~Camtepe, and DC~Ranasinghe.
\newblock An empirical assessment of global covid-19 contact tracing
  applications.
\newblock 2021.

\bibitem{tang2020all}
Yutian Tang, Yulei Sui, Haoyu Wang, Xiapu Luo, Hao Zhou, and Zhou Xu.
\newblock All your app links are belong to us: understanding the threats of
  instant apps based attacks.
\newblock In {\em Proceedings of the 28th ACM Joint Meeting on European
  Software Engineering Conference and Symposium on the Foundations of Software
  Engineering}, pages 914--926, 2020.

\bibitem{wang2017understanding}
Haoyu Wang and Yao Guo.
\newblock Understanding third-party libraries in mobile app analysis.
\newblock In {\em 2017 IEEE/ACM 39th International Conference on Software
  Engineering Companion (ICSE-C)}, pages 515--516. IEEE, 2017.

\bibitem{wang2015reevaluating}
Haoyu Wang, Yao Guo, Zihao Tang, Guangdong Bai, and Xiangqun Chen.
\newblock Reevaluating android permission gaps with static and dynamic
  analysis.
\newblock In {\em 2015 IEEE Global Communications Conference (GLOBECOM)}, pages
  1--6. IEEE, 2015.

\bibitem{wang2015using}
Haoyu Wang, Jason Hong, and Yao Guo.
\newblock Using text mining to infer the purpose of permission use in mobile
  apps.
\newblock In {\em Proceedings of the 2015 ACM International Joint Conference on
  Pervasive and Ubiquitous Computing}, pages 1107--1118, 2015.

\bibitem{wang2019understanding}
Haoyu Wang, Hao Li, and Yao Guo.
\newblock Understanding the evolution of mobile app ecosystems: A longitudinal
  measurement study of google play.
\newblock In {\em The World Wide Web Conference}, pages 1988--1999. ACM, 2019.

\bibitem{wang2018android}
Haoyu Wang, Hao Li, Li~Li, Yao Guo, and Guoai Xu.
\newblock Why are android apps removed from google play? a large-scale
  empirical study.
\newblock In {\em The 15th International Conference on Mining Software
  Repositories (MSR 2018)}, 2018.

\bibitem{wang2019signing}
Haoyu Wang, Hongxuan Liu, Xusheng Xiao, Guozhu Meng, and Yao Guo.
\newblock Characterizing android app signing issues.
\newblock In {\em 2019 34th IEEE/ACM International Conference on Automated
  Software Engineering (ASE)}, pages 280--292. IEEE, 2019.

\bibitem{wang2017explorative}
Haoyu Wang, Zhe Liu, Yao Guo, Xiangqun Chen, Miao Zhang, Guoai Xu, and Jason
  Hong.
\newblock An explorative study of the mobile app ecosystem from app developers'
  perspective.
\newblock In {\em Proceedings of the 26th International Conference on World
  Wide Web}, pages 163--172, 2017.

\bibitem{wang2018beyond}
Haoyu Wang, Zhe Liu, Jingyue Liang, Narseo Vallina-Rodriguez, Yao Guo, Li~Li,
  Juan Tapiador, Jingcun Cao, and Guoai Xu.
\newblock Beyond google play: A large-scale comparative study of chinese
  android app markets.
\newblock In {\em Proceedings of the Internet Measurement Conference 2018},
  pages 293--307, 2018.

\bibitem{RMVDroid}
Haoyu Wang, Junjun Si, Hao Li, and Yao Guo.
\newblock Rmvdroid: Towards a reliable android malware dataset with app
  metadata.
\newblock In {\em Proceedings of the 16th International Conference on Mining
  Software Repositories}, page 404–408, 2019.

\bibitem{wang2019characterizing}
Haoyu Wang, Xupu Wang, and Yao Guo.
\newblock Characterizing the global mobile app developers: a large-scale
  empirical study.
\newblock In {\em 2019 IEEE/ACM 6th International Conference on Mobile Software
  Engineering and Systems (MOBILESoft)}, pages 150--161. IEEE, 2019.

\bibitem{wang2020market}
Huiyi Wang, Liu Wang, and Haoyu Wang.
\newblock Market-level analysis of government-backed covid-19 contact tracing
  apps.
\newblock {\em arXiv preprint arXiv:2012.10866}, 2020.

\bibitem{wang2020covid}
Linda Wang and Alexander Wong.
\newblock Covid-net: A tailored deep convolutional neural network design for
  detection of covid-19 cases from chest x-ray images.
\newblock {\em arXiv preprint arXiv:2003.09871}, 2020.

\bibitem{wang20203}
Wei Wang, Guozhu Meng, Haoyu Wang, Kai Chen, Weimin Ge, and Xiaohong Li.
\newblock A3ident: A two-phased approach to identify the leading authors of
  android apps.
\newblock In {\em 2020 IEEE International Conference on Software Maintenance
  and Evolution (ICSME)}, pages 617--628. IEEE, 2020.

\bibitem{xi2019deepintent}
Shengqu Xi, Shao Yang, Xusheng Xiao, Yuan Yao, Yayuan Xiong, Fengyuan Xu, Haoyu
  Wang, Peng Gao, Zhuotao Liu, Feng Xu, et~al.
\newblock Deepintent: Deep icon-behavior learning for detecting
  intention-behavior discrepancy in mobile apps.
\newblock In {\em Proceedings of the 2019 ACM SIGSAC Conference on Computer and
  Communications Security}, pages 2421--2436, 2019.

\bibitem{xu2020dissecting}
Xu~Xu, Yangyu Hu, Qian Guo, Ren He, Li~Li, Guoai Xu, Zhihui Han, and Haoyu
  Wang.
\newblock Dissecting mobile offerwall advertisements: An explorative study.
\newblock In {\em 2020 IEEE 20th International Conference on Software Quality,
  Reliability and Security (QRS)}, pages 518--526. IEEE, 2020.

\bibitem{zhan2020automated}
Xian Zhan, Lingling Fan, Tianming Liu, Sen Chen, Li~Li, Haoyu Wang, Yifei Xu,
  Xiapu Luo, and Yang Liu.
\newblock Automated third-party library detection for android applications: Are
  we there yet?
\newblock In {\em 2020 35th IEEE/ACM International Conference on Automated
  Software Engineering (ASE)}, pages 919--930. IEEE, 2020.

\bibitem{zhang2020covid}
Jianpeng Zhang, Yutong Xie, Yi~Li, Chunhua Shen, and Yong Xia.
\newblock Covid-19 screening on chest x-ray images using deep learning based
  anomaly detection.
\newblock {\em arXiv preprint arXiv:2003.12338}, 2020.

\bibitem{10.1093/jtm/taaa036}
Ping Zhong, Songxue Guo, and Ting Chen.
\newblock {Correlation between travellers departing from Wuhan before the
  Spring Festival and subsequent spread of COVID-19 to all provinces in China}.
\newblock {\em Journal of Travel Medicine}, 27(3), 03 2020.
\newblock taaa036.

\bibitem{zhou2020demystifying}
Hao Zhou, Haoyu Wang, Yajin Zhou, Xiapu Luo, Yutian Tang, Lei Xue, and Ting
  Wang.
\newblock Demystifying diehard android apps.
\newblock In {\em 2020 35th IEEE/ACM International Conference on Automated
  Software Engineering (ASE)}, pages 187--198. IEEE, 2020.

\bibitem{zhou2012dissecting}
Yajin Zhou and Xuxian Jiang.
\newblock Dissecting android malware: Characterization and evolution.
\newblock In {\em 2012 IEEE symposium on security and privacy}, pages 95--109.
  IEEE, 2012.

\bibitem{zu2020coronavirus}
Zi~Yue Zu, Meng~Di Jiang, Peng~Peng Xu, Wen Chen, Qian~Qian Ni, Guang~Ming Lu,
  and Long~Jiang Zhang.
\newblock Coronavirus disease 2019 (covid-19): a perspective from china.
\newblock {\em Radiology}, page 200490, 2020.

\end{thebibliography}

cite.bib
@inproceedings{libradar,
  title={LibRadar: fast and accurate detection of third-party libraries in Android apps},
  author={Ma, Ziang and Wang, Haoyu and Guo, Yao and Chen, Xiangqun},
  booktitle={Proceedings of the 38th International Conference on Software Engineering Companion},
  pages={653--656},
  year={2016},
}

@inproceedings{RMVDroid,
author = {Wang, Haoyu and Si, Junjun and Li, Hao and Guo, Yao},
title = {RmvDroid: Towards a Reliable Android Malware Dataset with App Metadata},
year = {2019},
booktitle = {Proceedings of the 16th International Conference on Mining Software Repositories},
pages = {404–408},
numpages = {5},
}


@article{shen2020treatment,
  title={Treatment of 5 critically ill patients with COVID-19 with convalescent plasma},
  author={Shen, Chenguang and Wang, Zhaoqin and Zhao, Fang and Yang, Yang and Li, Jinxiu and Yuan, Jing and Wang, Fuxiang and Li, Delin and Yang, Minghui and Xing, Li and others},
  journal={Jama},
  volume={323},
  number={16},
  pages={1582--1589},
  year={2020},
  publisher={American Medical Association}
}

@article{zhang2020covid,
  title={Covid-19 screening on chest x-ray images using deep learning based anomaly detection},
  author={Zhang, Jianpeng and Xie, Yutong and Li, Yi and Shen, Chunhua and Xia, Yong},
  journal={arXiv preprint arXiv:2003.12338},
  year={2020}
}

@article{wang2020covid,
  title={COVID-Net: A Tailored Deep Convolutional Neural Network Design for Detection of COVID-19 Cases from Chest X-Ray Images},
  author={Wang, Linda and Wong, Alexander},
  journal={arXiv preprint arXiv:2003.09871},
  year={2020}
}

@ARTICLE{9144194,
  author={N. {Ahmed} and R. A. {Michelin} and W. {Xue} and S. {Ruj} and R. {Malaney} and S. S. {Kanhere} and A. {Seneviratne} and W. {Hu} and H. {Janicke} and S. K. {Jha}},
  journal={IEEE Access}, 
  title={A Survey of COVID-19 Contact Tracing Apps}, 
  year={2020},
  volume={8},
  number={},
  pages={134577-134601},
  doi={10.1109/ACCESS.2020.3010226}}
  
@inproceedings{wang2019understanding,
  title={Understanding the evolution of mobile app ecosystems: A longitudinal measurement study of google play},
  author={Wang, Haoyu and Li, Hao and Guo, Yao},
  booktitle={The World Wide Web Conference},
  pages={1988--1999},
  year={2019},
  organization={ACM}
}

@inproceedings{reardon201950,
  title={50 ways to leak your data: An exploration of apps' circumvention of the android permissions system},
  author={Reardon, Joel and Feal, {\'A}lvaro and Wijesekera, Primal and On, Amit Elazari Bar and Vallina-Rodriguez, Narseo and Egelman, Serge},
  booktitle={28th $\{$USENIX$\}$ Security Symposium ($\{$USENIX$\}$ Security 19)},
  pages={603--620},
  year={2019}
}

@article{razaghpanah2018apps,
  title={Apps, trackers, privacy, and regulators: A global study of the mobile tracking ecosystem},
  author={Razaghpanah, Abbas and Nithyanand, Rishab and Vallina-Rodriguez, Narseo and Sundaresan, Srikanth and Allman, Mark and Kreibich, Christian and Gill, Phillipa},
  year={2018}
}

@inproceedings{dong2018mobile,
  title={How do mobile apps violate the behavioral policy of advertisement libraries?},
  author={Dong, Feng and Wang, Haoyu and Li, Li and Guo, Yao and Xu, Guoai and Zhang, Shaodong},
  booktitle={Proceedings of the 19th International Workshop on Mobile Computing Systems \& Applications},
  pages={75--80},
  year={2018}
}

@inproceedings{liu2019dapanda,
  title={DaPanda: Detecting Aggressive Push Notifications in Android Apps},
  author={Liu, Tianming and Wang, Haoyu and Li, Li and Bai, Guangdong and Guo, Yao and Xu, Guoai},
  booktitle={2019 34th IEEE/ACM International Conference on Automated Software Engineering (ASE)},
  pages={66--78},
  year={2019},
  organization={IEEE}
}


@inproceedings{maddroid,
  title={MadDroid: Characterising and Detecting Devious Ad Content for Android Apps},
  author={Liu, Tianming and Wang, Haoyu and Li, Li and Luo, Xiapu and Dong, Feng and Guo, Yao and Wang, Liu and Bissyande, Tegawende F. and Klein, Jacques
},
  booktitle={Proceedings of the Web Conference 2020 (WWW'20)},
  year={2020},
}

@inproceedings{wang2018android, 
  title={Why are Android Apps Removed From Google Play? A Large-scale Empirical Study}, 
  author={Wang, Haoyu and Li, Hao and Li, Li and Guo, Yao and Xu, Guoai}, 
  booktitle={The 15th International Conference on Mining Software Repositories (MSR 2018)}, 
  year={2018} 
}

@inproceedings{wang2017explorative,
  title={An explorative study of the mobile app ecosystem from app developers' perspective},
  author={Wang, Haoyu and Liu, Zhe and Guo, Yao and Chen, Xiangqun and Zhang, Miao and Xu, Guoai and Hong, Jason},
  booktitle={Proceedings of the 26th International Conference on World Wide Web},
  pages={163--172},
  year={2017}
}

@inproceedings{wang2019characterizing,
  title={Characterizing the global mobile app developers: a large-scale empirical study},
  author={Wang, Haoyu and Wang, Xupu and Guo, Yao},
  booktitle={2019 IEEE/ACM 6th International Conference on Mobile Software Engineering and Systems (MOBILESoft)},
  pages={150--161},
  year={2019},
  organization={IEEE}
}

@inproceedings{wang2017understanding,
  title={Understanding third-party libraries in mobile app analysis},
  author={Wang, Haoyu and Guo, Yao},
  booktitle={2017 IEEE/ACM 39th International Conference on Software Engineering Companion (ICSE-C)},
  pages={515--516},
  year={2017},
  organization={IEEE}
}

@Misc{SmartphoneMarket,
  author =       "Rayna Hollander",
  year =         "2017",
  title =        "Two-thirds of the world's population are now connected by mobile devices",
  howpublished =        "https://www.businessinsider.com/world-population-mobile-devices-2017-9/?r=AU\&IR=T",
}

@inproceedings{wang2018beyond,
  title={Beyond google play: A large-scale comparative study of chinese android app markets},
  author={Wang, Haoyu and Liu, Zhe and Liang, Jingyue and Vallina-Rodriguez, Narseo and Guo, Yao and Li, Li and Tapiador, Juan and Cao, Jingcun and Xu, Guoai},
  booktitle={Proceedings of the Internet Measurement Conference 2018},
  pages={293--307},
  year={2018}
}

@inproceedings{hu2020mobile,
  title={Mobile app squatting},
  author={Hu, Yangyu and Wang, Haoyu and He, Ren and Li, Li and Tyson, Gareth and Castro, Ignacio and Guo, Yao and Wu, Lei and Xu, Guoai},
  booktitle={Proceedings of The Web Conference 2020},
  pages={1727--1738},
  year={2020}
}

@inproceedings{zhou2020demystifying,
  title={Demystifying Diehard Android Apps},
  author={Zhou, Hao and Wang, Haoyu and Zhou, Yajin and Luo, Xiapu and Tang, Yutian and Xue, Lei and Wang, Ting},
  booktitle={2020 35th IEEE/ACM International Conference on Automated Software Engineering (ASE)},
  pages={187--198},
  year={2020},
  organization={IEEE}
}

@inproceedings{zhan2020automated,
  title={Automated Third-Party Library Detection for Android Applications: Are We There Yet?},
  author={Zhan, Xian and Fan, Lingling and Liu, Tianming and Chen, Sen and Li, Li and Wang, Haoyu and Xu, Yifei and Luo, Xiapu and Liu, Yang},
  booktitle={2020 35th IEEE/ACM International Conference on Automated Software Engineering (ASE)},
  pages={919--930},
  year={2020},
  organization={IEEE}
}


@inproceedings{zhou2012dissecting,
  title={Dissecting android malware: Characterization and evolution},
  author={Zhou, Yajin and Jiang, Xuxian},
  booktitle={2012 IEEE symposium on security and privacy},
  pages={95--109},
  year={2012},
  organization={IEEE}
}

@article{li2019rebooting,
  title={Rebooting research on detecting repackaged android apps: Literature review and benchmark},
  author={Li, Li and Bissyand{\'e}, Tegawend{\'e} F and Klein, Jacques},
  journal={IEEE Transactions on Software Engineering},
  year={2019},
  publisher={IEEE}
}


@inproceedings{tang2020all,
  title={All your app links are belong to us: understanding the threats of instant apps based attacks},
  author={Tang, Yutian and Sui, Yulei and Wang, Haoyu and Luo, Xiapu and Zhou, Hao and Xu, Zhou},
  booktitle={Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering},
  pages={914--926},
  year={2020}
}

@article{wang2020market,
  title={Market-level Analysis of Government-backed COVID-19 Contact Tracing Apps},
  author={Wang, Huiyi and Wang, Liu and Wang, Haoyu},
  journal={arXiv preprint arXiv:2012.10866},
  year={2020}
}

@inproceedings{wang20203,
  title={A3Ident: A Two-phased Approach to Identify the Leading Authors of Android Apps},
  author={Wang, Wei and Meng, Guozhu and Wang, Haoyu and Chen, Kai and Ge, Weimin and Li, Xiaohong},
  booktitle={2020 IEEE International Conference on Software Maintenance and Evolution (ICSME)},
  pages={617--628},
  year={2020},
  organization={IEEE}
}

@inproceedings{xu2020dissecting,
  title={Dissecting Mobile Offerwall Advertisements: An Explorative Study},
  author={Xu, Xu and Hu, Yangyu and Guo, Qian and He, Ren and Li, Li and Xu, Guoai and Han, Zhihui and Wang, Haoyu},
  booktitle={2020 IEEE 20th International Conference on Software Quality, Reliability and Security (QRS)},
  pages={518--526},
  year={2020},
  organization={IEEE}
}


@inproceedings{xi2019deepintent,
  title={DeepIntent: Deep icon-behavior learning for detecting intention-behavior discrepancy in mobile apps},
  author={Xi, Shengqu and Yang, Shao and Xiao, Xusheng and Yao, Yuan and Xiong, Yayuan and Xu, Fengyuan and Wang, Haoyu and Gao, Peng and Liu, Zhuotao and Xu, Feng and others},
  booktitle={Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security},
  pages={2421--2436},
  year={2019}
}

@inproceedings{wang2019signing,
  title={Characterizing Android app signing issues},
  author={Wang, Haoyu and Liu, Hongxuan and Xiao, Xusheng and Meng, Guozhu and Guo, Yao},
  booktitle={2019 34th IEEE/ACM International Conference on Automated Software Engineering (ASE)},
  pages={280--292},
  year={2019},
  organization={IEEE}
}

@article{hu2018dating,
  title={Dating with scambots: Understanding the ecosystem of fraudulent dating applications},
  author={Hu, Yangyu and Wang, Haoyu and Zhou, Yajin and Guo, Yao and Li, Li and Luo, Bingxuan and Xu, Fangren},
  journal={arXiv preprint arXiv:1807.04901},
  year={2018}
}



@article{li2019identifying,
  title={On identifying and explaining similarities in android apps},
  author={Li, Li and Bissyand{\'e}, Tegawend{\'e} F and Wang, Hao-Yu and Klein, Jacques},
  journal={Journal of Computer Science and Technology},
  volume={34},
  number={2},
  pages={437--455},
  year={2019},
  publisher={Springer}
}

@inproceedings{liu2016identifying,
  title={Identifying and analyzing the privacy of apps for kids},
  author={Liu, Minxing and Wang, Haoyu and Guo, Yao and Hong, Jason},
  booktitle={Proceedings of the 17th International Workshop on Mobile Computing Systems and Applications},
  pages={105--110},
  year={2016}
}

@inproceedings{wang2015using,
  title={Using text mining to infer the purpose of permission use in mobile apps},
  author={Wang, Haoyu and Hong, Jason and Guo, Yao},
  booktitle={Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing},
  pages={1107--1118},
  year={2015}
}

@inproceedings{dong2018frauddroid,
  title={Frauddroid: Automated ad fraud detection for android apps},
  author={Dong, Feng and Wang, Haoyu and Li, Li and Guo, Yao and Bissyand{\'e}, Tegawend{\'e} F and Liu, Tianming and Xu, Guoai and Klein, Jacques},
  booktitle={Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering},
  pages={257--268},
  year={2018},
  organization={ACM}
}

@inproceedings{wang2015reevaluating,
  title={Reevaluating android permission gaps with static and dynamic analysis},
  author={Wang, Haoyu and Guo, Yao and Tang, Zihao and Bai, Guangdong and Chen, Xiangqun},
  booktitle={2015 IEEE Global Communications Conference (GLOBECOM)},
  pages={1--6},
  year={2015},
  organization={IEEE}
}



@book{evans1996straightforward,
  title={Straightforward statistics for the behavioral sciences.},
  author={Evans, James D},
  year={1996},
  publisher={Thomson Brooks/Cole Publishing Co}
}

@misc{pearson,
  year =         "2020",
  title =        "{Pearson correlation coefficient}",
  howpublished =          "https://en.wikipedia.org/wiki/Pearson\_correlation\_coefficient",
}

@article{bai2020presumed,
  title={Presumed asymptomatic carrier transmission of COVID-19},
  author={Bai, Yan and Yao, Lingsheng and Wei, Tao and Tian, Fei and Jin, Dong-Yan and Chen, Lijuan and Wang, Meiyun},
  journal={Jama},
  volume={323},
  number={14},
  pages={1406--1407},
  year={2020},
  publisher={American Medical Association}
}

@article{zu2020coronavirus,
  title={Coronavirus disease 2019 (COVID-19): a perspective from China},
  author={Zu, Zi Yue and Jiang, Meng Di and Xu, Peng Peng and Chen, Wen and Ni, Qian Qian and Lu, Guang Ming and Zhang, Long Jiang},
  journal={Radiology},
  pages={200490},
  year={2020},
  publisher={Radiological Society of North America}
}

@article{onder2020case,
  title={Case-fatality rate and characteristics of patients dying in relation to COVID-19 in Italy},
  author={Onder, Graziano and Rezza, Giovanni and Brusaferro, Silvio},
  journal={Jama},
  volume={323},
  number={18},
  pages={1775--1776},
  year={2020},
  publisher={American Medical Association}
}


  
@misc{oliver2020mobile,
      title={Mobile phone data and COVID-19: Missing an opportunity?}, 
      author={Nuria Oliver and Emmanuel Letouzé and Harald Sterly and Sébastien Delataille and Marco De Nadai and Bruno Lepri and Renaud Lambiotte and Richard Benjamins and Ciro Cattuto and Vittoria Colizza and Nicolas de Cordes and Samuel P. Fraiberger and Till Koebe and Sune Lehmann and Juan Murillo and Alex Pentland and Phuong N Pham and Frédéric Pivetta and Albert Ali Salah and Jari Saramäki and Samuel V. Scarpino and Michele Tizzoni and Stefaan Verhulst and Patrick Vinck},
      year={2020},
      eprint={2003.12347},
      archivePrefix={arXiv},
      primaryClass={cs.CY}
}

@article{he2020beyond,
  title={Beyond the Virus: A First Look at Coronavirus-themed Mobile Malware},
  author={He, Ren and Wang, Haoyu and Xia, Pengcheng and Wang, Liu and Li, Yuanchun and Wu, Lei and Zhou, Yajin and Luo, Xiapu and Guo, Yao and Xu, Guoai},
  journal={arXiv preprint arXiv:2005.14619},
  year={2020}
}

@misc{survey,
  year =         "2020",
  title =        "Apptentive",
  howpublished =          "https://www.apptentive.com/blog/2020/02/04/mobile-app-ratings-and-reviews",
}

@misc{factors,
  title = "{Ranking Factors}",
  howpublished = "https://moz.com/blog/app-store-rankings-formula-deconstructed-in-5-mad-science-experiments?\_ga=2.18395834.2057695595.1604471809-802527328.1604471809",
}

@misc{whoreport,
     title = "{Report of the WHO-China Joint Mission on Coronavirus Disease 2019 (COVID-19).}",
     howpublished =          "https://www.who.int/docs/default-source/coronaviruse/who-china-joint-mission-on-covid-19-final-report.pdf",
}

@article{10.1093/jtm/taaa036,
    author = {Zhong, Ping and Guo, Songxue and Chen, Ting},
    title = "{Correlation between travellers departing from Wuhan before the Spring Festival and subsequent spread of COVID-19 to all provinces in China}",
    journal = {Journal of Travel Medicine},
    volume = {27},
    number = {3},
    year = {2020},
    month = {03},
    issn = {1708-8305},
    doi = {10.1093/jtm/taaa036},
    url = {https://doi.org/10.1093/jtm/taaa036},
    note = {taaa036},
    eprint = {https://academic.oup.com/jtm/article-pdf/27/3/taaa036/33226371/taaa036.pdf},
}

@misc{CNBC,
  title = "{China to lift lockdown on Wuhan, the epicenter of its coronavirus outbreak. CNBC}",
  howpublished =          "https://www.cnbc.com/2020/03/24/china-to-lift-lockdown-on-wuhan-city-epicenter-of-coronavirus-outbreak.html",
}

@article{doi:10.1056/NEJMoa2001191,
author = {Holshue, Michelle L. and DeBolt, Chas and Lindquist, Scott and Lofy, Kathy H. and Wiesman, John and Bruce, Hollianne and Spitters, Christopher and Ericson, Keith and Wilkerson, Sara and Tural, Ahmet and Diaz, George and Cohn, Amanda and Fox, LeAnne and Patel, Anita and Gerber, Susan I. and Kim, Lindsay and Tong, Suxiang and Lu, Xiaoyan and Lindstrom, Steve and Pallansch, Mark A. and Weldon, William C. and Biggs, Holly M. and Uyeki, Timothy M. and Pillai, Satish K.},
title = {First Case of 2019 Novel Coronavirus in the United States},
journal = {New England Journal of Medicine},
volume = {382},
number = {10},
pages = {929-936},
year = {2020},
doi = {10.1056/NEJMoa2001191},
    note ={PMID: 32004427},

URL = { 
        https://doi.org/10.1056/NEJMoa2001191
    
},
eprint = { 
        https://doi.org/10.1056/NEJMoa2001191
    
}
}

@misc{CNN,
  title = "{The first known US coronavirus-related fatality. CNN}",
  howpublished = "https://www.cnn.com/2020/04/23/us/california-woman-first-coronavirus-death/index.html",
}

@misc{public-health-emergency,
  title = "{Trump Declares Coronavirus A Public Health Emergency And Restricts Travel From China. NPR.}",
  howpublished = "https://www.npr.org/sections/health-shots/2020/01/31/801686524/trump-declares-coronavirus-a-public-health-emergency-and-restricts-travel-from-c",
}

@misc{travel-claim,
  title = "{Trump’s Snowballing China Travel Claim}",
  howpublished = "https://www.factcheck.org/2020/04/trumps-snowballing-china-travel-claim/",
}

@misc{apnews,
  title = "{Signs missed and steps slowed in Trump’s response}",
  howpublished = "https://apnews.com/article/6a8f85aad99607f313cca6ab1398e04d",
}

@misc{politico,
  title = "{Trump halts funding to World Health Organization}",
  howpublished = "https://www.politico.com/news/2020/04/14/trump-world-health-organization-funding-186786",
}

@misc{theguardian,
  title = "{The missing six weeks: how Trump failed the biggest test of his life}",
  howpublished = "https://www.theguardian.com/us-news/2020/mar/28/trump-coronavirus-politics-us-health-disaster",
}

@misc{trump-timeline,
  title = "{Timeline: The more than 60 times Trump has downplayed the coronavirus threat}",
  howpublished = "https://web.archive.org/web/20200811160119if\_/https://www.washing tonpost.com/politics/2020/03/12/trump-coronavirus-timeline/",
}

@misc{reopening,
  title = "{This is where each state is during its phased reopening. CNN}",
  howpublished = "https://www.cnn.com/interactive/2020/us/states-reopen-coronavirus-trnd/",
}

@misc{jhu-csse,
  title = "{Coronavirus COVID-19 (2019-nCoV). Johns Hopkins CSSE.}",
  howpublished = "https://www.arcgis.com/apps/opsdashboard/index.html\#/bda 7594740fd40299423467b48e9ecf6",
}

@misc{jhu,
  title = "{Mortality Analyses. Johns Hopkins University.}",
  howpublished = "https://coronavirus.jhu.edu/data/mortality",
}

@article{bussing2020tumor,
  title={Tumor Patients{\'{}} Perceived Changes of Specific Attitudes, Perceptions, and Behaviors Due to the COVID-19 Pandemic and Its Relation to Reduced Wellbeing},
  author={B{\"u}ssing, Arndt and H{\"u}bner, Jutta and Walter, Stefanie and Gie{\ss}ler, Wolfgang and B{\"u}ntzel, Jens},
  journal={Frontiers in psychiatry},
  volume={11},
  year={2020},
  publisher={Frontiers Media SA}
}

@article{dwivedi2020impact,
  title={Impact of COVID-19 pandemic on information management research and practice: Transforming education, work and life},
  author={Dwivedi, Yogesh K and Hughes, D Laurie and Coombs, Crispin and Constantiou, Ioanna and Duan, Yanqing and Edwards, John S and Gupta, Babita and Lal, Banita and Misra, Santosh and Prashant, Prakhar and others},
  journal={International Journal of Information Management},
  volume={55},
  pages={102211},
  year={2020},
  publisher={Elsevier}
}

@article{latif2020leveraging,
  title={Leveraging data science to combat covid-19: A comprehensive review},
  author={Latif, Siddique and Usman, Muhammad and Manzoor, Sanaullah and Iqbal, Waleed and Qadir, Junaid and Tyson, Gareth and Castro, Ignacio and Razi, Adeel and Boulos, Maged N Kamel and Weller, Adrian and others},
  journal={IEEE Transactions on Artificial Intelligence},
  year={2020},
  publisher={IEEE}
}

@misc{kramer2020potential,
  title={The potential impact of the Covid-19 pandemic on occupational status, work from home, and occupational mobility},
  author={Kramer, Amit and Kramer, Karen Z},
  year={2020},
  publisher={Elsevier}
}

@article{cho2020contact,
  title={Contact tracing mobile apps for COVID-19: Privacy considerations and related trade-offs},
  author={Cho, Hyunghoon and Ippolito, Daphne and Yu, Yun William},
  journal={arXiv preprint arXiv:2003.11511},
  year={2020}
}

@article{ahmed2020survey,
  title={A survey of covid-19 contact tracing apps},
  author={Ahmed, Nadeem and Michelin, Regio A and Xue, Wanli and Ruj, Sushmita and Malaney, Robert and Kanhere, Salil S and Seneviratne, Aruna and Hu, Wen and Janicke, Helge and Jha, Sanjay K},
  journal={IEEE Access},
  volume={8},
  pages={134577--134601},
  year={2020},
  publisher={IEEE}
}

@article{altakarli2020china,
  title={China’s response to the COVID-19 outbreak: a model for epidemic preparedness and management},
  author={AlTakarli, Nourah S},
  journal={Dubai Medical Journal},
  volume={3},
  number={2},
  pages={44--49},
  year={2020},
  publisher={Karger Publishers}
}

@misc{CasesinUS,
  title = "{Tracking Covid-19 cases in the US}",
  howpublished = "https://www.cnn.com/interactive/2020/health/coronavirus-us-maps-and-cases/",
}

@inproceedings{carreno2013analysis,
  title={Analysis of user comments: an approach for software requirements evolution},
  author={Carreno, Laura V Galvis and Winbladh, Kristina},
  booktitle={2013 35th international conference on software engineering (ICSE)},
  pages={582--591},
  year={2013},
  organization={IEEE}
}

@inproceedings{pagano2013user,
  title={User feedback in the appstore: An empirical study},
  author={Pagano, Dennis and Maalej, Walid},
  booktitle={2013 21st IEEE international requirements engineering conference (RE)},
  pages={125--134},
  year={2013},
  organization={IEEE}
}

@article{garousi2020mining,
  title={Mining user reviews of COVID contact-tracing apps: An exploratory analysis of nine European apps},
  author={Garousi, Vahid and Cutting, David and Felderer, Michael},
  journal={arXiv preprint arXiv:2012.13589},
  year={2020}
}

@article{sharma2020advocating,
  title={Advocating for Users’ Privacy Protections: A Case study of COVID-19 apps},
  author={Sharma, Tanusree and Wang, Tian and Bashir, Masooda},
  year={2020},
  publisher={ACM}
}

@article{sun2021empirical,
  title={An Empirical Assessment of Global COVID-19 Contact Tracing Applications},
  author={Sun, R and Wang, W and Xue, M and Tyson, G and Camtepe, S and Ranasinghe, DC},
  year={2021}
}