  Web search is an integral part of our daily lives. Recently, there has been a
trend of personalization in Web search, where different users receive different
results for the same search query. The increasing level of personalization is
leading to concerns about Filter Bubble effects, where certain users are simply
unable to access information that the search engines' algorithm decides is
irrelevant. Despite these concerns, there has been little quantification of the
extent of personalization in Web search today, or the user attributes that
cause it.
  In light of this situation, we make three contributions. First, we develop a
methodology for measuring personalization in Web search results. While
conceptually simple, there are numerous details that our methodology must
handle in order to accurately attribute differences in search results to
personalization. Second, we apply our methodology to 200 users on Google Web
Search and 100 users on Bing. We find that, on average, 11.7% of results show
differences due to personalization on Google, while 15.8% of results are
personalized on Bing, but that this varies widely by search query and by result
ranking. Third, we investigate the user features used to personalize on Google
Web Search and Bing. Surprisingly, we only find measurable personalization as a
result of searching with a logged in account and the IP address of the
searching user. Our results are a first step towards understanding the extent
and effects of personalization on Web search engines today.
