Underreporting of errors in NLG output, and what to do about it

Emiel van Miltenburg, Miruna Clinciu, Ondřej Dušek, Dimitra Gkatzia, Stephanie Inglis, Leo Leppänen, Saad Mahamood, Emma Manning, Stephanie Schoch, Craig Thomson, Luou Wen

    Research output: Chapter in Book/Report/Conference proceedingConference contributionScientificpeer-review

    Abstract

    We observe a severe under-reporting of the different kinds of errors that Natural Language Generation systems make. This is a problem, because mistakes are an important indicator of where systems should still be improved. If authors only report overall performance metrics, the research community is left in the dark about the specific weaknesses that are exhibited by `state-of-the-art' research. Next to quantifying the extent of error under-reporting, this position paper provides recommendations for error identification, analysis and reporting.
    Original languageEnglish
    Title of host publicationProceedings of the 14th International Conference on Natural Language Generation
    Place of PublicationAberdeen, Scotland, UK
    PublisherAssociation for Computational Linguistics
    Pages140-153
    Number of pages14
    Publication statusPublished - 1 Aug 2021
    EventThe 14th International Conference on Natural Language Generation - Aberdeen (online), Aberdeen, United Kingdom
    Duration: 20 Sept 202124 Sept 2021
    Conference number: 14

    Conference

    ConferenceThe 14th International Conference on Natural Language Generation
    Abbreviated titleINLG
    Country/TerritoryUnited Kingdom
    CityAberdeen
    Period20/09/2124/09/21

    Fingerprint

    Dive into the research topics of 'Underreporting of errors in NLG output, and what to do about it'. Together they form a unique fingerprint.

    Cite this