Preregistering NLP research

Emiel van Miltenburg; Chris van der Lee; Emiel Krahmer

Preregistering NLP research

Emiel van Miltenburg, Chris van der Lee, Emiel Krahmer

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Scientific › peer-review

Abstract

Preregistration refers to the practice of specifying what you are going to do, and what you expect to find in your study, before carrying out the study. This practice is increasingly common in medicine and psychology, but is rarely discussed in NLP. This paper discusses preregistration in more detail, explores how NLP researchers could preregister their work, and presents several preregistration questions for different kinds of studies. Finally, we argue in favour of registered reports, which could provide firmer grounds for slow science in NLP research. The goal of this paper is to elicit a discussion in the NLP community, which we hope to synthesise into a general NLP preregistration form in future research.

Original language	English
Title of host publication	Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
Place of Publication	Online
Publisher	Association for Computational Linguistics
Pages	613-623
Number of pages	11
Publication status	Published - 1 Jun 2021
Event	Human Language Technology Conference 2021 - online Duration: 16 Mar 2021 → 18 Mar 2021 https://www.hltcon.org/

Conference

Conference	Human Language Technology Conference 2021
Abbreviated title	HLTCon
Period	16/03/21 → 18/03/21
Internet address	https://www.hltcon.org/

Access to Document

https://www.aclweb.org/anthology/2021.naacl-main.51

Best Thematic Paper Award
van Miltenburg, E. (Recipient), Krahmer, E. (Recipient) & van der Lee, C. (Recipient), 2021
Prize: Other marks of recognition

Cite this

@inproceedings{e5cce62278324800853fc8365a002ab8,

title = "Preregistering NLP research",

abstract = "Preregistration refers to the practice of specifying what you are going to do, and what you expect to find in your study, before carrying out the study. This practice is increasingly common in medicine and psychology, but is rarely discussed in NLP. This paper discusses preregistration in more detail, explores how NLP researchers could preregister their work, and presents several preregistration questions for different kinds of studies. Finally, we argue in favour of registered reports, which could provide firmer grounds for slow science in NLP research. The goal of this paper is to elicit a discussion in the NLP community, which we hope to synthesise into a general NLP preregistration form in future research.",

author = "{van Miltenburg}, Emiel and {van der Lee}, Chris and Emiel Krahmer",

year = "2021",

month = jun,

day = "1",

language = "English",

pages = "613--623",

booktitle = "Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies",

publisher = "Association for Computational Linguistics",

note = "Human Language Technology Conference 2021, HLTCon ; Conference date: 16-03-2021 Through 18-03-2021",

url = "https://www.hltcon.org/",

}

van Miltenburg, E , van der Lee, C & Krahmer, E 2021, Preregistering NLP research. in Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, Online, pp. 613-623, Human Language Technology Conference 2021, 16/03/21. <https://www.aclweb.org/anthology/2021.naacl-main.51>

Preregistering NLP research. / van Miltenburg, Emiel ; van der Lee, Chris ; Krahmer, Emiel.
Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Online: Association for Computational Linguistics, 2021. p. 613-623.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Scientific › peer-review

TY - GEN

T1 - Preregistering NLP research

AU - van Miltenburg, Emiel

AU - van der Lee, Chris

AU - Krahmer, Emiel

PY - 2021/6/1

Y1 - 2021/6/1

N2 - Preregistration refers to the practice of specifying what you are going to do, and what you expect to find in your study, before carrying out the study. This practice is increasingly common in medicine and psychology, but is rarely discussed in NLP. This paper discusses preregistration in more detail, explores how NLP researchers could preregister their work, and presents several preregistration questions for different kinds of studies. Finally, we argue in favour of registered reports, which could provide firmer grounds for slow science in NLP research. The goal of this paper is to elicit a discussion in the NLP community, which we hope to synthesise into a general NLP preregistration form in future research.

AB - Preregistration refers to the practice of specifying what you are going to do, and what you expect to find in your study, before carrying out the study. This practice is increasingly common in medicine and psychology, but is rarely discussed in NLP. This paper discusses preregistration in more detail, explores how NLP researchers could preregister their work, and presents several preregistration questions for different kinds of studies. Finally, we argue in favour of registered reports, which could provide firmer grounds for slow science in NLP research. The goal of this paper is to elicit a discussion in the NLP community, which we hope to synthesise into a general NLP preregistration form in future research.

M3 - Conference contribution

SP - 613

EP - 623

BT - Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies

PB - Association for Computational Linguistics

CY - Online

T2 - Human Language Technology Conference 2021

Y2 - 16 March 2021 through 18 March 2021

ER -

Preregistering NLP research

Abstract

Conference

Access to Document

Fingerprint

Prizes

Best Thematic Paper Award

Cite this