TU Darmstadt / ULB / TUprints

NLPeer: A Unified Resource for the Computational Study of Peer Review

Dycke, Nils ; Kuznetsov, Ilia ; Gurevych, Iryna (2024)
NLPeer: A Unified Resource for the Computational Study of Peer Review.
The 61st Annual Meeting of the Association for Computational Linguistics. Toronto, Canada (09.07.2023 - 14.07.2023)
doi: 10.26083/tuprints-00027661
Conference or Workshop Item, Secondary publication, Publisher's Version

[img] Text
2023.acl-long.277.pdf
Copyright Information: CC BY 4.0 International - Creative Commons, Attribution.

Download (973kB)
[img] Video
2023.acl-long.277.mp4
Copyright Information: CC BY 4.0 International - Creative Commons, Attribution.

Download (38MB)
Item Type: Conference or Workshop Item
Type of entry: Secondary publication
Title: NLPeer: A Unified Resource for the Computational Study of Peer Review
Language: English
Date: 16 July 2024
Place of Publication: Darmstadt
Year of primary publication: 2023
Place of primary publication: Kerrville, TX, USA
Publisher: ACL
Book Title: Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Event Title: The 61st Annual Meeting of the Association for Computational Linguistics
Event Location: Toronto, Canada
Event Dates: 09.07.2023 - 14.07.2023
DOI: 10.26083/tuprints-00027661
Corresponding Links:
Origin: Secondary publication service
Abstract:

Peer review constitutes a core component of scholarly publishing; yet it demands substantial expertise and training, and is susceptible to errors and biases. Various applications of NLP for peer reviewing assistance aim to support reviewers in this complex process, but the lack of clearly licensed datasets and multi-domain corpora prevent the systematic study of NLP for peer review. To remedy this, we introduce NLPeer– the first ethically sourced multidomain corpus of more than 5k papers and 11k review reports from five different venues. In addition to the new datasets of paper drafts, camera-ready versions and peer reviews from the NLP community, we establish a unified data representation and augment previous peer review datasets to include parsed and structured paper representations, rich metadata and versioning information. We complement our resource with implementations and analysis of three reviewing assistance tasks, including a novel guided skimming task. Our work paves the path towards systematic, multi-faceted, evidence-based study of peer review in NLP and beyond. The data and code are publicly available.

Identification Number: 2023.acl-long.277
Status: Publisher's Version
URN: urn:nbn:de:tuda-tuprints-276618
Classification DDC: 000 Generalities, computers, information > 004 Computer science
Divisions: 20 Department of Computer Science > Ubiquitous Knowledge Processing
Date Deposited: 16 Jul 2024 12:16
Last Modified: 10 Sep 2024 09:19
URI: https://tuprints.ulb.tu-darmstadt.de/id/eprint/27661
PPN: 520535170
Export:
Actions (login required)
View Item View Item