Q: Bootstrap hypothesis testing comparing proportions over time I'm new to bootstrap hypothesis testing and have a few questions that I can't seem to find the answer to. To make the questions easier to interpret I have added example data below. The test-statistic is the proportion of clicks / impressions. So for the hypothesis test I have H0 stating proportionA = proportionB.




date
userId
click
impression
version




01-01-2022
123
5
10
A


01-01-2022
456
1
2
A


01-02-2022
123
10
20
A


01-02-2022
456
2
4
B


01-01-2022
321
10
15
B


01-01-2022
654
1
4
B


01-02-2022
321
15
20
B


01-02-2022
654
0
2
B




I have the following questions:

*

*Since I have data over time, is 1 sample equal to 1 userId, or should 1 sample be equal to 1 userId per date?

*Should each userId be treated equally when resampling, even if their clicks are different in magnitude? For example userId 123 has 15 clicks, while userId 654 only has 1 click. Is there a way to "weight" these, or is that not necessary?

