  This paper describes, develops, and validates SciLens, a method to evaluate
the quality of scientific news articles. The starting point for our work are
structured methodologies that define a series of quality aspects for manually
evaluating news. Based on these aspects, we describe a series of indicators of
news quality. According to our experiments, these indicators help non-experts
evaluate more accurately the quality of a scientific news article, compared to
non-experts that do not have access to these indicators. Furthermore, SciLens
can also be used to produce a completely automated quality score for an
article, which agrees more with expert evaluators than manual evaluations done
by non-experts. One of the main elements of SciLens is the focus on both
content and context of articles, where context is provided by (1) explicit and
implicit references on the article to scientific literature, and (2) reactions
in social media referencing the article. We show that both contextual elements
can be valuable sources of information for determining article quality. The
validation of SciLens, done through a combination of expert and non-expert
annotation, demonstrates its effectiveness for both semi-automatic and
automatic quality evaluation of scientific news.
