  The Neutron Star Interior Composition Explorer (NICER) records data of
exceptional quality on the energy-dependent X-ray pulse profile of pulsars.
However, in searching for evidence of pulsations, Guillot et al. (2019)
introduce a procedure to select an ordered subset of the data that maximizes a
detection statistic (the H-test). I show that this procedure can degrade
subsequent analyses using an idealized model with a stationary expected count
rates from both noise and signal. Specifically, the data-selection procedure
biases the inferred mean count rate to be too low, biases the inferred
pulsation amplitude to be too high, and that the size of these biases scales
strongly with the amount of data that is rejected and the true signal
amplitude. The procedure also alters the null-distribution of the H-test
rendering nominal detection significance estimates overly optimistic. While the
idealized model does not capture all the complexities of real NICER data, it
suggests that these biases could be important for NICER's observations of
J0740+6620 and other faint pulsars (observations of J0030+0451 are likely less
affected). I estimate that these selection effects may introduce a bias of
$\mathcal{O}(10\%)$ on average in the inferred modulation depth of lightcurves
like J0740+6620's, and may be as large as $\mathcal{O}(50\%)$ for fainter
pulsars. However, the change for a single dataset like J0740+6620 is expected
to be a shift between $-5\%$ and $+20\%$. This could imply that the lower limit
on J0740+6620's radius is slightly larger than it should be, although
preliminary investigations suggest the change in the radius constraints are
$\mathcal{O}(1\%)$ with real J0740+6620 data.
