  Understanding when and why neural ranking models fail for an IR task via
error analysis is an important part of the research cycle. Here we focus on the
challenges of (i) identifying categories of difficult instances (a pair of
question and response candidates) for which a neural ranker is ineffective and
(ii) improving neural ranking for such instances. To address both challenges we
resort to slice-based learning for which the goal is to improve effectiveness
of neural models for slices (subsets) of data. We address challenge (i) by
proposing different slicing functions (SFs) that select slices of the
dataset---based on prior work we heuristically capture different failures of
neural rankers. Then, for challenge (ii) we adapt a neural ranking model to
learn slice-aware representations, i.e. the adapted model learns to represent
the question and responses differently based on the model's prediction of which
slices they belong to. Our experimental results (the source code and data are
available at https://github.com/Guzpenha/slice_based_learning) across three
different ranking tasks and four corpora show that slice-based learning
improves the effectiveness by an average of 2% over a neural ranker that is not
slice-aware.
