What is the research goal?
What question(s) is the author trying to answer?
What methods are being applied?
What methods is the author applying to answer the question?
What are the research results?
A paper can contain many different kinds of results (E.g.: applied results (efficiency or accuracy), theoretical results (a theorem was proven), etc.)
What claims are made in the paper?
For theoretical papers, what results are proven?
How are these claims supported?
What reasonable claims and results are missing from the paper?
What would be reasonable next steps for the research?

How to initiate a paper discussion
by David Jensen, 2001 (http://kdl.cs.umass.edu/courses/csna/initiate-discussion.html)

Specific Example: NIPS Review Criteria

Quality -- Is the paper technically sound? Are claims well-supported? Is this a complete piece of work, or merely a ``position paper''? Are the authors careful (and honest) about evaluating both the strengths and weaknesses of the work.
Clarity -- Is the paper clearly written? Is it well-organized (if not, feel free to make suggestions to improve the manuscript)? Does it adequately inform the reader? (A superbly written paper provides enough information for the expert reader to reproduce the results.)
Originality -- Are the problems or approaches new? Is this a novel combination of familiar techniques? Is it clear how this work differs from previous contributions? Is related work adequately referenced?
Significance -- Are the results important? Does the paper address a difficult problem in a better way than previous research? Does it advance the state of the art? Does it provide unique data, unique conclusions on existing data, or a unique theoretical or pragmatic approach?