Evaluation Setup
Data set
Test data: TREC-13 question answering task data
AQUAINT corpus and 64 definition questions with answers
Training data
761 manually labeled definition sentences from TREC-12
question answering task data
Comparison systems
State-of-the-art manually constructed patterns
Most comprehensive manually constructed patterns to our
knowledge
Previously proposed soft pattern in Cui et al., 2004