Analyzing analytical methods: The case of phonology in neural models of spoken language

Research output: Chapter in Book/Report/Conference proceedingConference contributionScientificpeer-review

Abstract

Given the fast development of analysis techniques for NLP and speech processing systems, few systematic studies have been conducted to compare the strengths and weaknesses of each method. As a step in this direction we study the case of representations of phonology in neural network models of spoken language. We use two commonly applied analytical techniques, diagnostic classifiers and representational similarity analysis, to quantify to what extent neural activation patterns encode phonemes and phoneme sequences. We manipulate two factors that can affect the outcome of analysis. First, we investigate the role of learning by comparing neural activations extracted from trained versus randomly-initialized models. Second, we examine the temporal scope of the activations by probing both local activations corresponding to a few milliseconds of the speech signal, and global activations pooled over the whole utterance. We conclude that reporting analysis results with randomly initialized models is crucial, and that global-scope methods tend to yield more consistent and interpretable results and we recommend their use as a complement to local-scope diagnostic methods.
Original languageEnglish
Title of host publicationProceedings of the 58th Annual Meeting of the Association for Computational Linguistics
Place of PublicationOnline
PublisherAssociation for Computational Linguistics
Pages4146-4156
Number of pages11
DOIs
Publication statusPublished - 1 Jul 2020
Event58th Annual Meeting of the Association for Computational Linguistics - Online
Duration: 6 Jul 2020 → …

Conference

Conference58th Annual Meeting of the Association for Computational Linguistics
Abbreviated titleACL 2020
Period6/07/20 → …

Fingerprint Dive into the research topics of 'Analyzing analytical methods: The case of phonology in neural models of spoken language'. Together they form a unique fingerprint.

  • Cite this

    Chrupała, G., Higy, B., & Alishahi, A. (2020). Analyzing analytical methods: The case of phonology in neural models of spoken language. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (pp. 4146-4156). Association for Computational Linguistics. https://doi.org/10.18653/v1/2020.acl-main.381