  Online advertisements have become one of today's most widely used tools for
enhancing businesses partly because of their compatibility with A/B testing.
A/B testing allows sellers to find effective advertisement strategies such as
ad creatives or segmentations. Even though several studies propose a technique
to maximize the effect of an advertisement, there is insufficient comprehension
of the customers' offline shopping behavior invited by the online
advertisements. Herein, we study the difference in offline behavior between
customers who received online advertisements and regular customers (i.e., the
customers visits the target shop voluntary), and the duration of this
difference. We analyzed approximately three thousand users' offline behavior
with their 23.5 million location records through 31 A/B testings. We first
demonstrate the externality that customers with advertisements traverse larger
areas than those without advertisements, and this spatial difference lasts
several days after their shopping day. We then find a long-run effect of this
externality of advertising that a certain portion of the customers invited to
the offline shops revisit these shops. Finally, based on this revisit effect
findings, we utilize a causal machine learning model to propose a marketing
strategy to maximize the revisit ratio. Our results suggest that advertisements
draw customers who have different behavior traits from regular customers. This
study's findings demonstrate that a simple analysis may underrate the effects
of advertisements on businesses, and an analysis considering externality can
attract potentially valuable customers.
