  The robust rank-order test (Fligner and Policello, 1981) was designed as an
improvement of the non-parametric Wilcoxon-Mann-Whitney U-test to be more
appropriate when the samples being compared have unequal variance. However, it
tends to be excessively liberal when the samples are asymmetric. This is likely
because the test statistic is assumed to have a standard normal distribution
for sample sizes > 12. This work proposes an on-the-fly method to obtain the
distribution of the test statistic from which the critical/p-value may be
computed directly. The method of likelihood maximization is used to estimate
the parameters of the parent distributions of the samples being compared. Using
these estimated populations, the null distribution of the test statistic is
obtained by the Monte-Carlo method. Simulations are performed to compare the
proposed method with that of standard normal approximation of the test
statistic. For small sample sizes (<= 20), the Monte-Carlo method outperforms
the normal approximation method. This is especially true for low values of
significance levels (< 5%). Additionally, when the smaller sample has the
larger standard deviation, the Monte-Carlo method outperforms the normal
approximation method even for large sample sizes (= 40/60). The two methods do
not differ in power. Finally, a Monte-Carlo sample size of 10^4 is found to be
sufficient to obtain the aforementioned relative improvements in performance.
Thus, the results of this study pave the way for development of a toolbox to
perform the robust rank-order test in a distribution-free manner.
