\documentclass[10pt,letterpaper]{article}
   \RequirePackage[nohead,top=.4in]{geometry}
   \RequirePackage{fancyhdr,afterpage}
   \RequirePackage[perpage,bottom,symbol]{footmisc}
  %\RequirePackage{indentfirst}
   \RequirePackage{multicol}
   \RequirePackage{pifont,amssymb}
   \RequirePackage{natbib,url}
   %% PDF detection macro from Christian Kumpf 
   %% in Question 3.1.6 of the pdfTeX FAQ version 0.12 (July 4, 2000)
   \newif\ifpdf
   \ifx\pdfoutput\undefined
      \pdffalse
   \else
      \pdfoutput=1
      \pdftrue
   \fi

   \RequirePackage{colortbl}
   \ifpdf
     \RequirePackage{thumbpdf}
     \RequirePackage[bookmarks=true,%           
                       bookmarksnumbered=true,% 
                       bookmarksopen=true,%      
                     colorlinks=true,%   turns off box around links
                       urlcolor=blue,%          
                     letterpaper=true,%         
                     pdfpagemode=None,%         
                     pdftex=true,%
                       breaklinks=true,% allow links to span multiple lines
                     pdfauthor={J. Blustein},%
                     pdftitle={Graduate Essay Grading Principles},%
                     pdfsubject={Human Factors in On-Line Information Systems}]%
                    {hyperref} % must be last package loaded
     \providecommand\myurl[1]{$\langle$URL:\href{#1}{\url{#1}}$\rangle$}
  \else
     \renewcommand\url{\begingroup \def\UrlLeft{\ensuremath{\langle\mbox{URL:}}}%
                                   \def\UrlRight{\ensuremath{\rangle}}%
                                   \urlstyle{tt}\Url}
     \providecommand\myurl[1]{\url|#1|}   % regular URLs 
     \providecommand\href[2]{#2}          % special URLs
     \providecommand\pdfbookmark[3][{}]{}    % only for *-form ...
     \providecommand\currentpdfbookmark[2]{} % ... sections in ...
     \providecommand\subpdfbookmark[2]{}     % ... pdflatex
     \providecommand\hypertarget[2]{#2}
     \providecommand\hyperlink[2]{#2}
  \fi
  \newcommand{\defn}[1]{\textit {#1}}
  \renewcommand{\emph}[1]{\textbf{#1}}

    \bibliographystyle{unsrt}
    \renewcommand{\bibname}{Reference}

    \pagestyle{plain}

    \newcommand{\Version}{version: 17~March~2005 (1a)}
    \title{Essay Grading}
    \author{CS 6606: Human Factors in On-Line Information Systems}
    \date{17 March 2005}
\begin{document}
    \maketitle

    \section{Basic Expectations}
      \begin{itemize}
        \item length (your essay must not be much shorter or longer than
                      the specified length)
        \item format (must be appropriate for an academic essay, and
                      conform the specification in the syllabus)
        \item structure
        \item spelling \& grammar
      \end{itemize}
      Violations of any of these expectations will result in deductions or
     outright failure.

    \subsection*{Use Of References}
      Your essay should be self-contained.  References to any external
     sources should be for additional information and sources of evidence
     rather than being necessary for the reader to understand the essay.

      Plagiarism will not be tolerated.  You must give proper credit when
     referring to or using work by others.  If you require guidance about
     the mechanics of citation and what constitutes plagiarism consult with
     the Computer Science librarian, or staff at the Killam Library
     reference desk.

    \section{Content}
      According to Dalhousie's undergraduate calendar~\cite{DalCal}:
      \vspace{-\baselineskip}%
      \begin{quotation}\noindent%
      \begin{description}
        \item[`A'-level grades] reflect
          `considerable evidence of original thinking; demonstrated
          outstanding capacity to analyze and synthesize; outstanding grasp
          of subject matter; evidence of extensive knowledge base.'

        \item[`B'-level grades] reflect
          `evidence of grasp of subject matter, some evidence of critical
          capacity and analytical ability; reasonable understanding of
          relevant issues; evidence of familiarity with the literature.'
      \end{description}
      \end{quotation}

      \noindent
      I evaluate your essay for the demonstrated understanding of
      underlying issues and terminology.

      In particular, the most important aspects of your essay that I
     evaluate when grading content are:
      \begin{itemize}
        \item quality of research (how well you selected appropriate
              high-quality materials by others);
        \item quality of analysis (your contributions to making sense
              of the material you have selected);
        \item demonstrated understanding of underlying issues and
              terminology; and
        \item style (how the essay is organized, and how well the text
              flows).
      \end{itemize}

   \begin{thebibliography}{1}
     \bibitem{DalCal}
       Dalhousie University.
       \newblock Section 17.1 Grade Scale and Definitions In Undergraduate calendar.
       \newblock \myurl{http://www.registrar.dal.ca/calendar/ugrad/acaregs.htm}.
       \newblock Downloaded 14 Dec. 2002.

     \bibitem{vanL:AHfS}
       Mary-Claire {van Leunen}.
       \newblock {\em A Handbook for Scholars} (Revised edition).
       \newblock Oxford University Press, 1992.
       \newblock ISBN 0-19-506953-5 (cloth) / 
                      0-19-506954-4 (paper).
   \end{thebibliography}

  \newpage \enlargethispage{\baselineskip}

   \section{Examples of Grade Levels}
   \begin{center}
   \begin{tabular}{p{.7\textwidth}|p{10ex}}
     \multicolumn{1}{c|}{\textbf{Essay Characteristics}}  &
     \multicolumn{1}{c}{\textbf{Grade}} \\\hline\hline
     \begin{itemize}
       \item does not violate basic expectations
       \item includes introduction about importance of topic and motivation
             for its study
       \item survey of article within an area, with one section about each
             article
       \item little or no clear connection between sections
     \end{itemize}
     & \begin{itemize}\item[]$\mbox{C}$\end{itemize}
     \\\hline
     \begin{itemize}
       \item as above but with a concluding summary comparing and
       contrasting the various articles
     \end{itemize} 
      & \begin{itemize}\item[]$\mbox{B}^{-}$\end{itemize}
     \\\hline
     \begin{itemize}
       \item survey beginning with motivation for study
       \item examination of relevant articles ordered by issue and with a
             clear, rational connection between sections
       \item insights throughout
       \item conclusion that summaries key points
     \end{itemize}
      & \begin{itemize}\item[]$\mbox{B}^{+}$\end{itemize}
      \\\hline
     \begin{itemize}
       \item comprehensive survey (within reasonably defined limits)
       \item clear structure of the essay follows from the goals in
             the introduction
       \item insights throughout 
       \item conclusion that summarizes the essay and points the way for
             future work and outstanding issues
     \end{itemize}
      & \begin{itemize}\item[]A\end{itemize}
     \\\hline
     \begin{itemize}
       \item discussion of a research problem with a plan
       \item begins with an introduction to the area and explanation of why
             the issues are important
       \item all relevant technical terms are clearly defined without
             interrupting the flow of the essay
       \item survey of background material (as above)
       \item a statement of hypotheses justified by evidence in the survey
       \item plan for an experiment to test the hypotheses including
             descriptions of dependent and independent variables,
             experimental design, and planned statistical tests (see
             Appendix~\ref{app}, on page~\pageref{app}) 
       \item concluding summary that serves as a short, complete pr\'ecis
             of the essay
     \end{itemize}
      & \begin{itemize}\item[]$\mbox{A}^{+}$\end{itemize}
     \\\hline
   \end{tabular}
   \end{center}

\newpage
  \appendix

    \section{Common Problem: Insufficient Justification}
    \enlargethispage{2\baselineskip}
       The notes in this appendix are intended to help you to prepare
     better essay and theses.
       Most of the students whose essays I have read have no problem saying   
     interesting things, although a few exaggerate beyond the evidence they 
     present.
       Although you must say interesting things in your essay, the reader
     of your essay should never have reason to doubt the truth of anything
     you write.

    \subsection{Special Considerations for Experiments}
       If you are writing up an experiment remember that your are
     describing an investigation not attempting to obtain any specific
     results.
       In the case of experiments, you must justify that the experiment is
     (or was) worth conducting, and that the results will be (or are)
     credible.

    \subsection{Justifying Claims}
       Following standard scholarly practice, you must justify all of the
     factual claims in your essays, and be careful not to exaggerate.
         You can justify claims in four general ways which are
     detailed in the sections immediately below.
     
     \subsubsection{Reference to Credible Sources}
     The strongest way is to provide evidence (either the results of
     experiments that you are presenting in the essay itself, or by
     reference to work by others).
       Any reference to work by others must include a formal citation to
     the complete record in the References section at the end of your essay.

    \paragraph*{Credibility of Sources}
       Your readers will need the full reference to determine the
     credibility of the evidence.
       Refereed works (articles, certain books, etc.) are usually
     considered to be high quality sources, but unrefereed works (such as 
     technical reports, personal websites) have little or no credibility. 
       If your reader does not recognize the author's name then they will
     likely look to see where the work appeared as a way to judge its
     credibility.
       Works appearing in ACM Transactions or from major academic presses
     (the Oxford Univ., Yale Univ., and MIT Presses for example) are much
     more credible than those appearing from low-quality journals,
     technical magazines, and unrefereed workshops.

       Some readers often depend only the citation to judge the
     credibility of the evidence you provide.
       However the \emph{real purpose of the citation} is to enable
     readers to 
       (a)~verify that what your cited actually appears where you said
           it did,
       and
       (b)~to find out more about the topic~\cite{vanL:AHfS}.

     \subsubsection{Expert Opinion}
       Where evidence is not available you can cite the opinion (of one or
     more established experts, or yourself\footnote{Even if you are an
     expert about what you are discussing you must justify your reputation
     in the essay, otherwise you are begging the question of your
     expertise: `I am an expert because I wrote this essay which made
     claims based on my opinions as an expert'.}).
       Such opinions are easiest to believe if some of the reasons for them 
     are described too.
      % \marginpar{\raggedright\textsl{pers. comm.}}
       In a thesis or published article you must provide a full citation
     for all non-print sources, e.g. telephone conversations.
       The CS Librarian can show you how to cite such `personal
     communication'.
       The most important aspect for these notes is that you provide a
     reason for the reader to understand why you are citing someone else's
     opinion.
       For example: `Experimental participants were placed into one of
     two groups depending on their ABC score.  Following the accepted
     practice in $\overline{\mbox{\tiny {a relevant field}}}$ [cite
     an experiment published in a respected venue\marginpar{
       \raggedright\textsl{Venues} are
        where articles are published, e.g. journals.}]
     partitioning was based on the median score.'

    \subsubsection{Pure Reason}
        Sometimes it is appropriate or provide reasoning without hard
     evidence.
        For example: `The groups were partitioned into three groups
     (referred to as ``low'', ``middle', and ``high'') based on their
     XYZ scores.
        Scores in the two extreme groups (``low'' and ``high'')
     differed by at least one standard deviation but there was some
     overlap between each of those groups and the ``middle'' group, as
     can be seen in Fig.~X.'

    \subsubsection{Appeal to Common Knowledge}
        Finally, the weakest justification (which nonetheless may be
     sufficient) is your own impression of generally held views.
        You can write something like `Since it is generally held that the
     sky is often blue, \ldots'.

\newpage  
  \section{What Is A `Plan For An Experiment'?\label{app}}

    We will have studied several articles describing experiments before
   your term paper is due.
    In addition to the lessons you should have learned from such
   articles here are some specific points that should be explicit
   parts of your plan for an experiment.


  \subsection{Motivation \& Expected Importance}
    Always remember that an experiment is an investigation and never
   an attempt to prove anything.
    You must explain what you expect to learn from the experiment and
   why that information will be useful or interesting.

  \subsection{Hypotheses}
    Clearly state what you are planning to investigate with your experiment.
    A simple example could be that your new interface will be more
   effective than another interface.

   \marginpar{\raggedright\textsl
              {The values you are measuring are 
               described as Factors in \S\ref{fact}.}} % 
    Traditionally hypotheses come in pairs: the \defn{null hypothesis}
   which is that there is no difference between the values you are
   measuring, and the \defn{alternative hypothesis} (which is that
   there is a difference).
     The alternative hypothesis can be one-tailed or two-tailed.
     \defn{One-tailed hypotheses} are directional (the result of the
   treatment will be strictly greater than, or less-than, the control
   condition).
     The control is a condition in which everything is exactly the
   same as in the treatment except that the factors are missing.
     For example: if one of the factors is that users are predicted to
   improve with practice, then a control condition could be that you
   measure before and after practice.
     The measurements before practice are a control for the practice
   factor in the after practice condition.
     It can be appropriate to control for several factors with
   separate conditions.


  \subsection{Factors\label{fact}}
     List and describe the independent and dependent variables.

     \defn{Independent variables} are the factors that you hypothesize
   as causes.
     Sometimes these can be measured directly (such as time to
   complete a task), but sometimes they must be operationalized as 
   dependent variables instead.
     For example you might expect that users' understanding of a
   subject will be improved by using the interface you propose to
   create. But you cannot measure understanding directly, so you
   measure something that depends on understanding: how much users'
   scores in comprehension tests improves after using your interface
   for instance.

     You must account for all factors that could influence your
   experiment so that you can design adequate controls.
     For instance if you use a repeated-measures design then you will
   need to control for order effects even if you do not expect them to
   be significant.


  \subsection{Statistical Investigation}
    \begin{itemize}
      \item What descriptive statistics will you present?
      \item What statistical tests will you perform?
      \item What standards will you use the decide to accept or reject
            the null hypothesis?  (In particular, is a one- or
            two-tailed test most appropriate?)
    \end{itemize}

  \subsection{Possible Implications}
    A well-planned experiment can often make use of so-called negative
   (or unexpected) results as well as results which confirm the
   alternative hypothesis.
     You must foresee the implications of both positive and negative
   results as part of the design of your experiment.
     Include a brief description of what accepting or rejecting each
   of your alternative hypotheses would mean for further experiments
   (and for practice, if it is appropriate to your experiment).

\end{document}
