  Identifying user's opinions and stances in long conversation threads on
various topics can be extremely critical for enhanced personalization, market
research, political campaigns, customer service, conflict resolution, targeted
advertising, and content moderation. Hence, training language models to
automate this task is critical. However, to train such models, gathering manual
annotations has multiple challenges: 1) It is time-consuming and costly; 2)
Conversation threads could be very long, increasing chances of noisy
annotations; and 3) Interpreting instances where a user changes their opinion
within a conversation is difficult because often such transitions are subtle
and not expressed explicitly. Inspired by the recent success of large language
models (LLMs) for complex natural language processing (NLP) tasks, we leverage
Mistral Large and GPT-4 to automate the human annotation process on the
following two tasks while also providing reasoning: i) User Stance
classification, which involves labeling a user's stance of a post in a
conversation on a five-point scale; ii) User Dogmatism classification, which
deals with labeling a user's overall opinion in the conversation on a
four-point scale. The majority voting on zero-shot, one-shot, and few-shot
annotations from these two LLMs on 764 multi-user Reddit conversations helps us
curate the USDC dataset. USDC is then used to finetune and instruction-tune
multiple deployable small language models for the 5-class stance and 4-class
dogmatism classification tasks. We make the code and dataset publicly available
[https://anonymous.4open.science/r/USDC-0F7F].
