  New capabilities in foundation models are owed in large part to massive,
widely-sourced, and under-documented training data collections. Existing
practices in data collection have led to challenges in documenting data
transparency, tracing authenticity, verifying consent, privacy, representation,
bias, copyright infringement, and the overall development of ethical and
trustworthy foundation models. In response, regulation is emphasizing the need
for training data transparency to understand foundation models' limitations.
Based on a large-scale analysis of the foundation model training data landscape
and existing solutions, we identify the missing infrastructure to facilitate
responsible foundation model development practices. We examine the current
shortcomings of common tools for tracing data authenticity, consent, and
documentation, and outline how policymakers, developers, and data creators can
facilitate responsible foundation model development by adopting universal data
provenance standards.
