by Prof. Nathalie Japkowicz

What is the research goal?

What question(s) is the author trying to answer?

What methods are being applied?

What methods is the author applying to answer the question?

What are the research results?

A paper can contain many different kinds of results (E.g.: applied results (efficiency
or accuracy), theoretical results (a theorem was proven), etc.)

What claims are made in the paper?

For theoretical papers, what results are proven?

How are these claims supported?

What reasonable claims and results are missing from the paper?

What would be reasonable next steps for the research?

**How
to initiate a paper discussion**

by David Jensen, 2001 (http://kdl.cs.umass.edu/courses/csna/initiate-discussion.html)

**Specific Example: NIPS Review Criteria**

**Quality** -- Is the paper technically sound? Are claims well-supported?
Is this a complete piece of work, or merely a ``position paper''? Are the authors
careful (and honest) about evaluating both the strengths and weaknesses of the
work.

**Clarity** -- Is the paper clearly written? Is it well-organized
(if not, feel free to make suggestions to improve the manuscript)? Does it adequately
inform the reader? (A superbly written paper provides enough information for
the expert reader to reproduce the results.)

**Originality** -- Are the problems or approaches new? Is this
a novel combination of familiar techniques? Is it clear how this work differs
from previous contributions? Is related work adequately referenced?

**Significance** -- Are the results important? Does the paper address
a difficult problem in a better way than previous research? Does it advance
the state of the art? Does it provide unique data, unique conclusions on existing
data, or a unique theoretical or pragmatic approach?