  Modern spacecraft are increasingly relying on machine learning (ML). However,
physical equipment in space is subject to various natural hazards, such as
radiation, which may inhibit the correct operation of computing devices.
Despite plenty of evidence showing the damage that naturally-induced faults can
cause to ML-related hardware, we observe that the effects of radiation on ML
models for space applications are not well-studied. This is a problem: without
understanding how ML models are affected by these natural phenomena, it is
uncertain "where to start from" to develop radiation-tolerant ML software. As
ML researchers, we attempt to tackle this dilemma. By partnering up with
space-industry practitioners specialized in ML, we perform a reflective
analysis of the state of the art. We provide factual evidence that prior work
did not thoroughly examine the impact of natural hazards on ML models meant for
spacecraft. Then, through a "negative result", we show that some existing
open-source technologies can hardly be used by researchers to study the effects
of radiation for some applications of ML in satellites. As a constructive step
forward, we perform simple experiments showcasing how to leverage current
frameworks to assess the robustness of practical ML models for cloud detection
against radiation-induced faults. Our evaluation reveals that not all faults
are as devastating as claimed by some prior work. By publicly releasing our
resources, we provide a foothold -- usable by researchers without access to
spacecraft -- for spearheading development of space-tolerant ML models.
