CoNLL-2013 Shared Task: Grammatical Error Correction
CoNLL-2013 will continue the CoNLL tradition of having a high profile shared task in natural language processing. This year's shared task will be grammatical error correction. A participating system in this shared task is given short English texts written by non-native speakers of English. The system detects the grammatical errors present in the input texts, and returns the corrected texts.
This task has gained popularity recently with the organization of the HOO (Helping Our Own) shared tasks in 2011 and 2012. In the most recent HOO shared task in 2012, on error detection and correction of determiners and prepositions, 14 teams from around the world participated and 85 systems were submitted to the shared task.
The grammatical error correction task is impactful since it is estimated that hundreds of millions of people in the world are learning English and they benefit directly from an automated grammar checker. However, for many error types, current grammatical error correction methods do not achieve a high performance and thus more research is needed.
Instead of focusing on only determiner and preposition errors as in HOO 2012, the CoNLL-2013 shared task will include a more comprehensive list of error types, including determiner, preposition, noun number, verb form, and subject-verb agreement errors. Extending into more error types introduces the possibility of correcting multiple interacting errors. Examples of such interacting errors include determiner and noun number ('that cars' → 'that car' or 'those cars') and preposition and verb form ('an interest to study’ → ‘an interest in studying').
Participating teams will be provided with common training data in which grammatical errors have been annotated. Blind test data will be used to evaluate the outputs of the participating teams using a common scoring software and evaluation metric.
Registration
Registration for the shared task is closed.
Important Dates
December 17, 2012: announcement of shared taskJanuary 17, 2013: set up of shared task websiteFebruary 5, 2013: registration begins and release of training set and scorerMarch 5, 2013: registration deadlineApril 28, 2013: test set availableMay 1, 2013: systems' outputs collectedMay 8, 2013: system results due to participantsMay 15, 2013: shared task system papers dueMay 24, 2013: reviews dueMay 27, 2013: notification of acceptanceJune 3, 2013: camera ready version of shared task system papers due- August 8-9, 2013: CoNLL-2013 conference (Sofia, Bulgaria)
Program
The CoNLL-2013 Shared Task program is now available. Click here to view.
Overview Paper
The shared task overview paper can now be downloaded:
Ng, Hwee Tou, & Wu, Siew Mei, & Wu, Yuanbin, & Hadiwinoto, Christian, & Tetreault, Joel (2013). The CoNLL-2013 Shared Task on Grammatical Error Correction. In Proceedings of the Seventeenth Conference on Computational Natural Language Learning: Shared Task (CoNLL-2013 Shared Task). Sofia, Bulgaria.
Data Release
The test data with gold standard annotations and the official scorer are now available. The package can be downloaded here.
Shared Task Organizers
- Hwee Tou Ng (Chair), National University of Singapore
- Joel Tetreault, Nuance Communications
- Siew Mei Wu, National University of Singapore
- Yuanbin Wu, National University of Singapore
- Christian Hadiwinoto, National University of Singapore