  Software developers typically rely upon a large network of dependencies to
build their applications. For instance, the NPM package repository contains
over 3 million packages and serves tens of billions of downloads weekly.
Understanding the structure and nature of packages, dependencies, and published
code requires datasets that provide researchers with easy access to metadata
and code of packages. However, prior work on NPM dataset construction typically
has two limitations: 1) only metadata is scraped, and 2) packages or versions
that are deleted from NPM can not be scraped. Over 330,000 versions of packages
were deleted from NPM between July 2022 and May 2023. This data is critical for
researchers as it often pertains to important questions of security and
malware. We present npm-follower, a dataset and crawling architecture which
archives metadata and code of all packages and versions as they are published,
and is thus able to retain data which is later deleted. The dataset currently
includes over 35 million versions of packages, and grows at a rate of about 1
million versions per month. The dataset is designed to be easily used by
researchers answering questions involving either metadata or program analysis.
Both the code and dataset are available at https://dependencies.science.
