To recruit a diversity of workers, we are using the UniqueTurker script to limit participation to 20 HITs per worker. Sorry for any inconvenience!

Comparing questions asked about a text

We are linguistics researchers interested in how reading a text can evoke a question. For instance, if you read "I'm a bit worried.", this evokes the question "Worried about what?".

Previously we have presented texts to various people, and asked them "What question does the text evoke?". This resulted in thousands of evoked questions. We need your help to organize these. In particular: help us judge how the questions entered by different people are related.

In this HIT you will get text snippets, for a total of question comparisons.


For each text snippet you are given a Target question (T) and a number of Comparison questions (C). For each comparison, you have to choose between:

Equivalence: Target and Comparison question are asking for the same information (though they may use very different words to do so!).
     ↪ Example: 'Why?', 'What is the reason?', 'What for?' can all count as the same question.
Overlap: Target and Comparison question are slightly different, but they overlap.
     ↪ Example: 'Is this a kind of tree that grows in the US natively?' vs. 'What kind of tree is it?'.
Related: Target and Comparison question are quite different (no overlap), but still closely related.
     ↪ Example: 'When did she become a guitar builder?' vs. 'What woods does she use for her guitars?'
Unrelated: Target and Comparison question are very different; they are not closely related.
     ↪ Example: 'How long have you been bowling?' vs. 'Do you have pets?'.
Unclear: Target and/or Comparison question are unclear.

(While doing the task you can hover over each button to review these same instructions.)

This can be a tricky task! The quality of your responses is vital to our research, therefore we included quality control items in each HIT. Low quality responses can result in rejection -- but we are compassionate humans and try to be fair.

Final advice: What we want you to compare is what kind of information the questions are asking for, not how they are asking it. Therefore: look beyond superficial appearance, interpret the questions in the context of the text snippet, and think: what sort of answers would each question invite? If they invite the same kinds of answers, then they are the same kind of question.

Good luck!

► Please read the snippet:


► Next, compare the questions it evoked:

Questions: How related are target (T) and comparison (C) question?

► Any comments? [Optional]