Not All Models Localize Linguistic Knowledge in the Same Place: A Layer-wise Probing on BERToids’ Representations

Mohsen Fayyaz, Ehsan Aghazadeh, Ali Modarressi, Hosein Mohebbi, Mohammad Taher Pilehvar

Research output: Chapter in Book/Report/Conference proceedingConference contributionScientificpeer-review


Most of the recent works on probing representations have focused on BERT, with the presumption that the findings might be similar to the other models. In this work, we extend the probing studies to two other models in the family, namely ELECTRA and XLNet, showing that variations in the pre-training objectives or architectural choices can result in different behaviors in encoding linguistic information in the representations. Most notably, we observe that ELECTRA tends to encode linguistic knowledge in the deeper layers, whereas XLNet instead concentrates that in the earlier layers. Also, the former model undergoes a slight change during fine-tuning, whereas the latter experiences significant adjustments. Moreover, we show that drawing conclusions based on the weight mixing evaluation strategy—which is widely used in the context of layer-wise probing—can be misleading given the norm disparity of the representations across different layers. Instead, we adopt an alternative information-theoretic probing with minimum description length, which has recently been proven to provide more reliable and informative results.
Original languageEnglish
Title of host publicationProceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP
Place of PublicationOnline
PublisherAssociation for Computational Linguistics
Number of pages14
Publication statusPublished - 2021
Externally publishedYes
EventBlackboxNLP 2021: Analyzing and Interpreting Neural Networks for NLP - online
Duration: 11 Nov 202111 Nov 2021


WorkshopBlackboxNLP 2021
Abbreviated titleBlackboxNLP2021
Internet address


Dive into the research topics of 'Not All Models Localize Linguistic Knowledge in the Same Place: A Layer-wise Probing on BERToids’ Representations'. Together they form a unique fingerprint.

Cite this