  Large language models (LLMs) are being increasingly tuned to power complex
generation tasks such as writing, fact-seeking, querying and reasoning.
Traditionally, human or model feedback for evaluating and further tuning LLM
performance has been provided at the response level, enabling faster and more
cost-effective assessments. However, recent works (Amplayo et al. [2022], Wu et
al. [2023]) indicate that sentence-level labels may provide more accurate and
interpretable feedback for LLM optimization. In this work, we introduce methods
to disaggregate response-level labels into sentence-level (pseudo-)labels. Our
approach leverages multiple instance learning (MIL) and learning from label
proportions (LLP) techniques in conjunction with prior information (e.g.,
document-sentence cosine similarity) to train a specialized model for
sentence-level scoring. We also employ techniques which use model predictions
to pseudo-label the train-set at the sentence-level for model training to
further improve performance.
  We conduct extensive evaluations of our methods across six datasets and four
tasks: retrieval, question answering, summarization, and math reasoning. Our
results demonstrate improved performance compared to multiple baselines across
most of these tasks. Our work is the first to develop response-level feedback
to sentence-level scoring techniques, leveraging sentence-level prior
information, along with comprehensive evaluations on multiple tasks as well as
end-to-end finetuning evaluation showing performance comparable to a model
trained on fine-grained human annotated labels.
