Stance detection is vital for promoting a trustworthy, human-centric Web by identifying biased or harmful narratives in user-generated content. Whereas recent LLM-based methods excel in accuracy, they often lack interpretability. We propose a generative stance detection approach that outputs explicit rationales and distills them into smaller language models (SLMs) via single-task and multitask learning. Our method enables Flan-T5 to outperform GPT-3.5 zero-shot by up to 9.57%. We further show that rationales enhance multitask performance and improve distillation fidelity, advancing the development of transparent, fair, and trustworthy NLP systems ¹.

Stance detection plays a key role in fostering a human-centric Web by promoting fairness, inclusivity, and accountability in online discourse. It identifies biases, discriminatory language, and harmful narratives in user-generated content, which can erode trust and exacerbate division [6]. Prior work has addressed stance detection and argumentation broadly addresses a variety of domains [2, 11, 12, 13, 16, 29], typically modeling it as a classification task over author opinions—in favor, against, or neutral—toward given claims or topics [3, 10, 20, 22, 28].

Although accuracy remains a focus, interpretability is often lacking, limiting real-world utility in high-stakes settings. Reasoning-aware stance detection is crucial for interpreting nuanced opinions, combating misinformation, and promoting ethical AI systems [27, 31], such as decoding morality on social media [25, 26]. However, collecting human-labeled rationales is expensive and domain-dependent [4]. Recent progress in large language models (LLMs), especially with Chain-of-Thought (CoT) prompting [24], has demonstrated strong reasoning in complex tasks like multihop QA and math [17, 21, 23]. GPT-generated explanations have shown promise for stance interpretability [30], but their integration into downstream models remains underexplored.

Figure 1: Framework for an explainable stance detection system. GPT-3.5 generates rationales conditioned on ground truth labels, and rationale distillation compares (a) ST-FT, (b) ST-CoT, and (c) MTL, where stance labels and rationales are generated concurrently.

We propose a generative framework that uses GPT-3.5 to produce both stance predictions and rationales, improving rationale faithfulness via conditioning on gold labels. We explore two rationale distillation strategies for small language models (SLMs): single-task CoT (ST-CoT) and multitask learning (MTL), and compare them to standard finetuning (ST-FT). ST-CoT generates rationales prior to predictions; MTL jointly learns both.

Our findings show that MTL outperforms ST-CoT and ST-FT, especially in low-resource settings, and better captures task structure. Moreover, rationale distillation boosts SLMs more than instruction tuning alone. Our contributions include (1) methods for eliciting faithful rationales from LLMs, (2) demonstrating MTL’s superiority for rationale distillation, and (3) identifying key factors driving rationale-based learning in stance detection.

Early stance detection relied on machine learning models with handcrafted features [1, 9]. Subsequent research adopted deep learning approaches, including recurrent networks [14], attention mechanisms [32], and pretrained models like BERT [7], driving performance improvements. Chain-of-thought (CoT) prompting elicits reasoning in complex tasks, with Wei et al. [24] demonstrating step-by-step reasoning using few-shot learning and Kojima et al. [15] introducing zero-shot-CoT via the prompt “Let’s think step by step,” achieving state-of-the-art performance in tasks like arithmetic and logical reasoning.

Figure 1 illustrates the framework for training a rationalized stance detection system. Human annotation for stance rationales is resource-intensive and prone to inconsistencies due to varying annotator expertise. To address this, we leverage the CoT capability of LLMs for consistent and faithful rationale generation. For integrating rationales into SLM training, we compare two approaches: single-task chain-of-thought (ST-CoT), which enforces rationale generation before prediction, and multitask learning (MTL), which separates the tasks. ST-CoT enforces rationale generation before prediction, intuitively appearing more effective than MTL, where prediction and rationale generation are separate. However, our experimental results contradict this expectation, showing MTL performs better.

Table 1: Statistics of SemEval-2016.

Prompt 1 Your task is to classify the stance of the comment on the topic as “favor”, “against”, or “neutral”. Conclude with the label.

Topic``: {Topic}

Comment``: {Comment}

Stance``:

Prompt 2 Explain the stance of the comment towards the topic by analyzing its content and relation to the topic.

Your answer should begin with: “The Comment``: {Comment} is classified as Stance``|: towards Topic``: {Topic} because ”

Prompt 3 Your task is to classify the stance of the comment on the topic as “favor”, “against”, or “neutral”.

Topic``: {Topic}

Comment``: {Comment}

Stance``:

Explain``:

Elicitation of Rationales from LLMs

Rationale Distillation into SLM

Dataset. We evaluate our framework on the SemEval-2016 Task 6 Subtask [18], containing 4,163 English tweets annotated as favor, against, or neutral across five topics: Atheism (AT), Climate Change (CC), Feminist Movement (FM), Hillary Clinton (HC), and Legalization of Abortion (LA). Table 1 displays the statistics of SemEval-2016. Following prior work, we use the official train-test split but create a new validation set by randomly sampling 10% of the training data, ensuring all topics are represented.

Evaluation Metric. We use the macro-average F1-score for favor and against ((F{avg} = frac{F{favor} + F{against}}{2}) )) as the evaluation metric, with neutral included in training and testing.

Rationale Distillation. We finetune T5 [22] and its instruction-tuned counterpart Flan-T5 [5] at small (80M), base (250M), and large (780M) sizes. For ST-FT and ST-CoT, inputs are the target and comment separated by an end-of-sentence token. For MTL, inputs are prefixed with Stance: or Explanation: depending on the task. Flan-T5 uses a unified input format, shown in Prompt 3.

Experimental Details. We train models with a batch size of 128, learning rate 5e-5, 30 epochs, maximum input length 512, and maximum generation length 256, using NVIDIA GPUs (A100, A30, A10, A6000).

Table 2: F_avg scores across models and sizes, with MTL performance reported at the optimal α. The subscripts are the standard deviation across three random runs. We establish the baseline by using GPT-3.5 to classify stances with Prompt 1, achieving an F1 score of 70.15% across three random runs.

Figure 2: F_avg scores on different α across models. Standard deviations are calculated from three random runs.

Figure 3: F_avg for training with different sizes, ranging from 10% to 100%.

Table 2 shows F_avg across tasks and models. Compared to the zero-shot baseline performance of 70.15% using GPT-3.5 with Prompt 1 (as described in Section 3.1), our approaches achieve up to 79.72%, even with the relatively small size of T5 models. We summarize our findings below.

MTL enhances ST performance across all settings:MTL consistently outperforms ST-FT by independently generating rationales, avoiding interference with prediction tasks. This fosters deeper rationale-prediction connections. T5 models benefit more from MTL than Flan-T5, with average improvements of 4.71 and 0.81, respectively, indicating that instruction tuning equips Flan-T5 with inherent reasoning capabilities, reducing the marginal effects of further finetuning.SLMs struggle with CoT capabilities:ST-CoT underperforms compared to ST-FT across all settings, as generating intermediate steps is 3,000 examples). Deviations in rationale generation often lead to failed or incorrect predictions, making rationale as an auxiliary task a more effective approach.Instruction tuning improves performance:Flan-T5, despite its smaller size, rivals or outperforms larger T5 models. For instance, Flan-T5-Small achieves 66.30 in ST-FT, surpassing T5-Base’s 65.54, and Flan-T5-Base scores 74.53 in MTL, compared to T5-Large’s 76.79. Flan-T5’s finetuning on diverse datasets enhances its instruction-following proficiency, improving stance classification performance.Weighting rationale generation varies performance:We examine the impact of the prediction and rationale generation tasks in MTL by varying the parameter α from 0.1 to 0.9, as shown in Figure 2. Optimal α values are generally low: 0.5 for T5-Small, 0.2 for T5-Base and T5-Large, 0.3 for Flan-T5-Small, and 0.1 for Flan-T5-Large, with Flan-T5-Base as an exception at 0.9. These results suggest that rationale distillation enhances MTL performance, likely due to the stronger supervision signal provided by the more challenging rationale generation task.Training data size affect performance:We evaluate generative models under limited training data by varying dataset size from 10% to 100% (Figure 3). Flan-T5 and T5 models show consistent performance improvements with larger training sizes for ST-FT and MTL. However, ST-CoT exhibits greater variability, indicating potential instability during training. Notably, T5-Base and T5-Large achieve performance comparable to full-data training with only 10-20% of the dataset under MTL, outperforming ST-FT and ST-CoT.

Effective stance detection allows AI systems to align with societal values, promoting a Web where ethical considerations guide user interactions and supporting a human-centric approach to technology. In this paper, we present an explainable stance detection system that leverages generative models to produce both predictions and justifications, reframing the task with GPT-3.5 to prioritize rationale clarity, given its strong performance in generating explanations for stance detection [30]. By providing interpretable justifications, our method enhances the reliability of automated systems in detecting biases, misinformation, and polarizing narratives, thereby fostering a safer and more inclusive online environment. We propose two rationale distillation methods–ST-CoT and MTL–demonstrating that MTL outperforms, particularly in low-data settings. Moreover, small language models benefit more from justification distillation than instruction-tuned models, emphasizing the importance of both prediction and reasoning in MTL. This work contributes to advancing stance detection by promoting informed user engagement, supporting governance against harmful content, and helping bridge the digital divide through improved interpretability and accessibility of web technologies. Future research extending these methods to larger, cross-dataset applications [20] could enhance scalable solutions for fostering trust and inclusivity on the web.

We thank the anonymous reviewers for their helpful comments.