CoNLL-2014 Shared Task: Grammatical Error Correction

 

CoNLL-2014 will continue the CoNLL tradition of having a high profile shared task in natural language processing. This year's shared task will be grammatical error correction, a continuation of the CoNLL shared task in 2013. A participating system in this shared task is given short English texts written by non-native speakers of English. The system detects the grammatical errors present in the input texts, and returns the corrected essays. The shared task in 2014 will require a participating system to correct all errors present in an essay (i.e., not restricted to just five error types in 2013). Also, the evaluation metric will be changed to F_0.5, weighting precision twice as much as recall.

The grammatical error correction task is impactful since it is estimated that hundreds of millions of people in the world are learning English and they benefit directly from an automated grammar checker. However, for many error types, current grammatical error correction methods do not achieve a high performance and thus more research is needed.

Participating teams will be provided with common training data in which grammatical errors have been annotated. Blind test data will be used to evaluate the outputs of the participating teams using a common scoring software and evaluation metric.

Registration

Registration for the shared task has been *closed*.

45 teams have registered to participate in the shared task. We are also planning for a journal special issue on grammatical error correction after the conclusion of the shared task.

Important Dates

• November 22, 2013: announcement of shared task
• December 5, 2013: set up of shared task website
• December 27, 2013: registration begins and release of training set and scorer
• January 22, 2014: registration deadline
• March 16, 2014: test set available
• March 19, 2014: systems' outputs collected
• March 26, 2014: system results due to participants
• April 2, 2014: shared task system papers due
• April 11, 2014: reviews due
• April 14, 2014: notification of acceptance
• April 27, 2014: camera ready version of shared task system papers due
• June 26-27, 2014: CoNLL-2014 conference (Baltimore, Maryland, USA)

Shared Task Organizers

• Hwee Tou Ng (Chair), National University of Singapore
• Siew Mei Wu, National University of Singapore
• Ted Briscoe, University of Cambridge
• Christian Hadiwinoto, National University of Singapore
• Raymond Hendy Susanto, National University of Singapore
• Christopher Bryant, National University of Singapore

Program

The CoNLL-2014 Shared Task program is now available. Click here to view.

Overview Paper

The shared task overview paper can now be downloaded:

Ng, Hwee Tou, & Wu, Siew Mei, & Briscoe, Ted, & Hadiwinoto, Christian, & Susanto, Raymond Hendy, & Bryant, Christopher (2014). The CoNLL-2014 Shared Task on Grammatical Error Correction. In Proceedings of the Eighteenth Conference on Computational Natural Language Learning: Shared Task (CoNLL-2014 Shared Task). Baltimore, Maryland.

Proceedings

The shared task proceedings can now be downloaded. Click here to download.

Data Release

The test data with gold standard annotations and the official scorer are now available:

• NUCLE Release 3.2: To obtain the data, please download the license form. Print the form, sign, and have the scanned PDF file of the signed form ready. Then, please provide your particulars (name, position, affiliation, and email address) and upload your scanned PDF file of the *signed* form through the license submission page. We will try to send the NUCLE data to you within 3 (three) working days.
• Annotated Test Data
• Official Scorer (version 3.2)
• Corrected system outputs of 12 participating teams