How LLMs Understand Rhetorical Questions: A Multi-Dimensional Representation Mechanism Revealed by Linear Probing

This study uses linear probing technology to explore the internal representation mechanism of rhetorical questions in LLMs. Key findings include: Rhetorical signals emerge in the early layers of the model, and the representation of the last token is the most stable; rhetorical questions are encoded along multiple linear directions in the representation space, and probes trained on different datasets capture different rhetorical phenomena; cross-dataset transfer is detectable but has differences, revealing LLMs' multi-dimensional understanding of rhetorical questions.

Rhetorical questions are a special linguistic phenomenon whose core function is rhetorical expression rather than information acquisition (e.g., "Shouldn't we protect the environment?" emphasizes an opinion). The tension between their semantics and pragmatics makes automatic recognition complex, requiring reliance on context, tone, and intent rather than just syntactic structure. For LLMs to understand these subtle differences, they need to form internal representations that distinguish rhetorical intent.

Linear probing technology is used to analyze the internal representations of LLMs: freeze the pre-trained model parameters, train a linear classifier on the hidden layer outputs. If it can distinguish rhetorical questions from ordinary questions, it indicates that the relevant features have been learned by the model. The study was conducted on two different social media datasets to test the generality of the findings.

Rhetorical signals start to emerge in the early layers of the model, indicating that LLMs can recognize rhetorical features of rhetorical questions early when processing sentences; the rhetorical signal is most stable in the last token's representation, which is consistent with LLMs often using the last token for downstream prediction; rhetorical questions are linearly separable within a single dataset, and cross-dataset transfer AUROC reaches 0.7-0.8, indicating the existence of general rhetorical question-related representations.

Cross-dataset transfer is feasible, but when probes from different datasets are applied to the same corpus, the ranking results differ significantly (overlap of top-ranked instances is less than 0.2), suggesting that rhetorical questions are encoded along multiple linear directions in the representation space, with each direction emphasizing different clues. Qualitative analysis shows: some probes capture rhetorical stance at the discourse level, while others emphasize locally syntactically driven questioning behavior.

Rhetorical questions include multiple rhetorical strategies: emphasis type (e.g., "Who doesn't want to succeed?"), questioning type (e.g., "Do you really believe this statement?"), and sarcastic type (e.g., "Isn't this great?" in a negative context). Different types of rhetorical questions activate different internal representation modes in LLMs, explaining why a single probe cannot capture all rhetorical phenomena.

Implications of the study for LLM interpretability: 1. A seemingly single concept (such as rhetorical questions) may be decomposed into multiple dimensions, and concept probing needs to consider the internal structure; 2. Early layers capture rhetorical signals, which is consistent with the characteristic of LLMs processing language information layer by layer; 3. The feasibility and differences of cross-dataset transfer indicate that LLMs have general rhetorical perception capabilities, but their manifestations vary depending on training data.

Future research directions: 1. Develop fine-grained probing methods to capture multiple linear directions simultaneously to fully understand the representation structure of rhetorical questions; 2. Explore the relationship between the representations of rhetorical questions and other rhetorical phenomena (metaphors, irony) to see if a unified rhetorical framework can be formed; 3. Apply the findings to NLP tasks such as sentiment analysis and stance detection to improve performance.