Where Is Politeness in Japanese BERT? A Layerwise Probing and CLS Activation Patching Study

Abstract

Politeness is a central pragmatic dimension of language use, and Japanese honorifics offer a well-defined testbed for studying whether pretrained encoders represent socially meaningful distinctions. Prior BERT-based work has applied supervised models to Japanese honorific data, but we are not aware of analyses that localize honorific-level information across layers or test causal influence via activation patching in Japanese BERT-style encoders. We study these questions in LineDistilBERT using the KeiCO corpus, which labels sentences with four honorific levels. To isolate pretrained representations while still defining a task predictor, we freeze all encoder parameters and train only a lightweight [CLS] classification head as a minimal readout. We then run layerwise linear probing, training multinomial L2-regularized logistic-regression probes on [CLS] vectors from each layer to quantify linear decodability across depth and to select a best layer on development data. Finally, we test causal leverage with [CLS] activation patching, transplanting donor activations into receiver sentences at selected layers and measuring prediction transitions, logit shifts, and flip rates under standard controls. Overall, honorific level is broadly decodable across layers, and [CLS] interventions can systematically steer the frozen-encoder classifier with strong depth dependence, providing complementary evidence from probing and causal intervention for Japanese politeness in practice.

Resources

Details