  Hierarchical Bayesian inference can simultaneously account for both
measurement uncertainty and selection effects within astronomical catalogs. In
particular, the hierarchy imposed encodes beliefs about the interdependence of
the physical processes that generate the observed data. We show that several
proposed approximations within the literature actually correspond to inferences
that are incompatible with any physical detection process, which can be
described by a directed acyclic graph (DAG). This generically leads to biases
and is associated with the assumption that detectability is independent of the
observed data given the true source parameters. We show several examples of how
this error can affect astrophysical inferences based on catalogs of coalescing
binaries observed through gravitational waves, including misestimating the
redshift evolution of the merger rate as well as incorrectly inferring that
General Relativity is the correct theory of gravity when it is not. In general,
one cannot directly fit for the ``detected distribution'' and ``divide out''
the selection effects in post-processing. Similarly, when comparing theoretical
predictions to observations, it is better to simulate detected data (including
both measurement noise and selection effects) rather than comparing estimates
of the detected distributions of event parameters (which include only selection
effects). While the biases introduced by model misspecification from incorrect
assumptions may be smaller than statistical uncertainty for moderate catalog
sizes (O(100) events), they will nevertheless pose a significant barrier to
precision measurements of astrophysical populations.
