The amount of text data has been growing exponentially and with it the demand for improved information extraction (IE) efforts to analyze and query such data. While automatic IE systems have proven useful in controlled experiments, in practice the gap between machine learning extraction and human extraction is still quite large. In this paper, we propose a system that uses crowdsourcing techniques to help close this gap. One of the fundamental issues inherent in using a largescale human workforce is deciding the optimal questions to pose to the crowd. We demonstrate novel solutions using mutual information and token clustering techniques in the domain of bibliographic citation extraction. Our experiments show promising results in using crowd assistance as a costeffective way to close up the ”last mile” between extraction systems and a human annotator.
Authors:
Sean Goldberg, Daisy Zhe Wang, Tim Kraska
Bibtex:
@article{,
author = "Sean Goldberg, Daisy Zhe Wang, Tim Kraska",
title = "CASTLE: Crowd-Assisted System for Textual Labeling \& Extraction",
journal = "HCOM",
year = "2013"
}
Download:
[pdf]