  Non-parametric dimensionality reduction techniques, such as t-SNE and UMAP,
are proficient in providing visualizations for datasets of fixed sizes.
However, they cannot incrementally map and insert new data points into an
already provided data visualization. We present Self-Organizing Nebulous
Growths (SONG), a parametric nonlinear dimensionality reduction technique that
supports incremental data visualization, i.e., incremental addition of new data
while preserving the structure of the existing visualization. In addition, SONG
is capable of handling new data increments, no matter whether they are similar
or heterogeneous to the already observed data distribution. We test SONG on a
variety of real and simulated datasets. The results show that SONG is superior
to Parametric t-SNE, t-SNE and UMAP in incremental data visualization.
Specifically, for heterogeneous increments, SONG improves over Parametric t-SNE
by 14.98 % on the Fashion MNIST dataset and 49.73% on the MNIST dataset
regarding the cluster quality measured by the Adjusted Mutual Information
scores. On similar or homogeneous increments, the improvements are 8.36% and
42.26% respectively. Furthermore, even when the above datasets are presented
all at once, SONG performs better or comparable to UMAP, and superior to t-SNE.
We also demonstrate that the algorithmic foundations of SONG render it more
tolerant to noise compared to UMAP and t-SNE, thus providing greater utility
for data with high variance, high mixing of clusters, or noise.
