  The development of rigorous quality assessment model relies on the collection
of reliable subjective data, where the perceived quality of visual multimedia
is rated by the human observers. Different subjective assessment protocols can
be used according to the objectives, which determine the discriminability and
accuracy of the subjective data.
  Single stimulus methodology, e.g., the Absolute Category Rating (ACR) has
been widely adopted due to its simplicity and efficiency. However, Pair
Comparison (PC) is of significant advantage over ACR in terms of
discriminability. In addition, PC avoids the influence of observers' bias
regarding their understanding of the quality scale. Nevertheless, full pair
comparison is much more time-consuming. In this study, we therefore 1) employ a
generic model to bridge the pair comparison data and ACR data, where the
variance term could be recovered and the obtained information is more complete;
2) propose a fusion strategy to boost pair comparisons by utilizing the ACR
results as initialization information; 3) develop a novel active batch sampling
strategy based on Minimum Spanning Tree (MST) for PC. In such a way, the
proposed methodology could achieve the same accuracy of pair comparison but
with the compelxity as low as ACR. Extensive experimental results demonstrate
the efficiency and accuracy of the proposed approach, which outperforms the
state of the art approaches.
