MearcAll papers

arXiv 1606.05250 · cs.CL

SQuAD: 100,000+ Questions for Machine Comprehension of Text

Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, Percy Liang

Claims were extracted automatically by reducer-1 on 2026-11-12. The paper's license does not allow republishing it, so long claims are shortened to a quotation. Read the full text on arXiv.

No reproduction recordedclm_81d3d3e4
We present the Stanford Question Answering Dataset (SQuAD), a new reading comprehension dataset consisting of 100,000+ questions posed by crowdworkers on a set of Wikipedia articles, where the answer to … (excerpt)

Abstract · page 1 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_448ae69a
We build a strong logistic regression model, which achieves an F1 score of 51.0%, a significant improvement over a simple baseline (20%).

Abstract · page 1 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_0b1b0470
Our best model achieves an F1 score of 51.0%, 1 which is much better than the sliding window baseline (20%).

Introduction · page 2 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_4842c6c1
To get a better understanding of the reasoning required to answer the questions, we sampled 4 questions from each of the 48 articles in the development set, and then manually … (excerpt)

Dataset Analysis · page 4 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_771ab463
64.1% Multiple sentence reasoning There is anaphora, or higher-level fusion of multiple sentences is required.

Dataset Analysis · page 5 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_2d21d743
The logistic regression model significantly outperforms the baselines, but underperforms Exact Match F1 Dev Test Dev Test Random Guess 1.1% 1.3% 4.1% 4.3% Sliding Window 13.2% 12.5% 20.2% 19.7% Sliding … (excerpt)

Model Performance · page 7 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedclm_981be0df
Logistic regression outperforms the baselines, while there is still a significant gap between humans.

Model Performance · page 8 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_e8686cde
The performance of our logistic regression model, with 51.0% F1, against the human F1 of 86.8% suggests ample opportunity for improvement.

Conclusion · page 9 · Automated extraction, not reviewed · 0 reviewed reproductions

Reproduced one of these claims?

Record your result in the Mearc app on your own machine, then use Publish this result with this paper's id (1606.05250) and the claim id shown next to the claim. Mearc creates a file that holds only what you choose to publish. Add it to the public claim data by pull request; a maintainer other than you reviews it before it counts.