  In this paper, our main contributions are that embeddings from relatively
smaller corpora can outperform ones from larger corpora and we make the new
Swedish analogy test set publicly available. To achieve a good network
performance in natural language processing (NLP) downstream tasks, several
factors play important roles: dataset size, the right hyper-parameters, and
well-trained embeddings. We show that, with the right set of hyper-parameters,
good network performance can be reached even on smaller datasets. We evaluate
the embeddings at both the intrinsic and extrinsic levels. The embeddings are
deployed with the Transformer in named entity recognition (NER) task and
significance tests conducted. This is done for both Swedish and English. We
obtain better performance in both languages on the downstream task with smaller
training data, compared to recently released, Common Crawl versions; and
character n-grams appear useful for Swedish, a morphologically rich language.
