SCENARIO METHOD

© 1997 by Walter Maner (unless otherwise noted)
May be reproduced only for non-commercial educational purposes.

The outline below requires Netscape/Microsoft browsers, version 3.x or later, for correct viewing.

Revised March 15, 1997
  1. Back Top Next
    PICTURE
  2. Back Top Next
    SCENARIO DEFINITION
    A scenario is a detailed, linear, step by step behavioral description
    of ...
    1. a specific user ...
    2. driven by a specific GOAL ...
    3. who takes specific ACTIONS ...
    4. to interact with specific interface elements (e.g., buttons) ...
    5. to complete a specific TASK ...
    6. on a specific occasion ...
    7. under specific circumstances ...
    8. within a specific period of time ...
    9. guided by specific feedback from the computer system.
  3. Back Top Next
    SCENARIO TERMINOLOGY
    1. GOAL
      -- similar to the end state of a procedure
      1. DEFINITION
        the stopping condition for a task that reaches closure
      2. EXAMPLES
        1. password accepted
        2. record 34 removed from database
        3. TCP/IP defaults restored
    2. ACTION
      -- similar to a statement
      1. DEFINITION
        a simple, specific, "indivisible" low-level task
        1. has no structure
        2. involves no problem solving
        3. has no component parts
        4. may refer to hardware and/or interface elements
      2. EXAMPLES
        1. activating the check box labeled "Show all matches"
        2. clicking the mouse in the Close Box
        3. tabbing to the next data-entry field inside the Billing
          Address form
    3. TASK
      -- similar to a procedure
      1. DEFINITION
        a sequence of simple actions and/or activities intended to
        achieve a specific goal
        1. has a definite structure, often hierarchical
        2. can involve problem solving
        3. can include other tasks as components (i.e, subtasks) unless
          it happens to consist entirely of simple actions
        4. makes no specific reference to hardware or to the interface
        5. reaches closure at a specific point
      2. SOURCE OF TASK INFORMATION
        1. INTROSPECTION
          What would I want to do? What would I do?
        2. INTERVIEWS WITH USERS
          What do users want to do?
        3. OBSERVATION OF USERS
          What do users actually do?
        4. ANALYSIS OF USERS
          Logically, what would users have to do?
      3. EXAMPLES
        1. logging on to Alpha
        2. connecting to BGNet
        3. viewing last month's accounting summary
        4. responding to helpdesk query regarding weekend lab hours
  4. Back Top Next
    EXAMPLE SCENARIO
    GOAL: Have $50 cash to spend
    TASK: Withdraw $50 from an ATM
    1. The User
      A financially solvent customer of Huntington Bank who needs $50
      cash approaches the Huntington ATM in the BGSU Union.
    2. The Computer System
      A normally functioning automatic teller machine operated by
      Huntington Bank
    3. Action
      The user sees a welcome message on the ATM's screen.
    4. Action
      The user takes a valid Huntington bank card from their wallet.
    5. Action
      The user slides the bank card fully into the marked slot.
    6. Action
      The user waits five seconds for the ATM to respond.
    7. Feedback
      The ATM displays "Please enter your four-digit personal
      identification number" on the screen.
    8. Action
      Using the physical keypad attached to the ATM, the user correctly
      enters their four-digit PIN.
    9. Action
      The user presses the Enter key.
    10. Feedback
      The ATM displays a menu of options on the screen, each one
      associated with a specific physical button:
      1. Withdraw $50
      2. Withdraw other amounts
      3. Make a deposit
      4. Other transactions
    11. Action
      The user presses the button associated with the "Withdraw $50"
      option.
    12. Action
      The user waits 15 seconds before hearing the machine begin to
      count out the cash.
    13. Feedback
      The ATM displays the message "Please remove your cash".
    14. Action
      The user removes the cash and places it in their wallet.
    15. Feedback
      The ATM displays the message "Do you want to make another
      transaction."
    16. Action
      The user presses the physical button labeled "NO".
    17. Feedback
      The ATM prints out a receipt and ejects the users bank card.
    18. Feedback
      The ATM prints out the message "Please remove your card and
      receipt."
    19. Action
      The user removes the receipt, examines it briefly, and places it
      in their wallet.
    20. Action
      The user removes their bank card and returns it to their wallet.
    21. Action
      The user watches the screen for 15 seconds.
    22. Feedback
      The ATM redisplays the welcome message.
    23. Action
      The customer leaves.
  5. Back Top Next
    SCENARIOS AND THE LARGER TASK MODEL
    1. Any given scenario task should be part of a well-defined sub-task
      super-task hierarchy.
      1. The scenario itself includes actions as sub-tasks, which
        themselves may include other sub-tasks.
      2. The set of tasks, sub-tasks, and super-tasks can be drawn in
        the form of a task tree, which is organized like a structure
        chart.
    2. In addition to the information included in individual scenarios, a
      complete task model may contain many additional pieces of related
      information.
      1. a complete task hierarchy
      2. a description of the physical task environment
      3. the average number of errors during each task
      4. the average time to perform each task
      5. the criticality level for each task
      6. the frequency with which each task is performed
      7. the time constraints for completing each task
      8. the tolerance of each task for user error
    3. Checklist for Evaluating the Task Model
      1. [ ] Does each task represent a class of user behaviors?
      2. [ ] Have important variations been considered?
      3. [ ] Is the set of tasks complete?
      4. [ ] Is the task related to usability requirements?
      5. [ ] Is the set of tasks descriptive?
      6. [ ] Is the task hierarchy correctly structured?
    4. Checklist for Evaluating Scenarios
      1. [ ] Are user actions clearly distinguished from system
        feedback?
      2. [ ] Does the scenario describe a real task users would need to
        perform?
      3. [ ] Does the scenario fit the overall task model?
      4. [ ] Have sub-tasks been correctly sequenced?
      5. [ ] Is each step "on task"?
      6. [ ] Is the description strictly confined to observable user
        behaviors and observable system feedback?
      7. [ ] Is the scenario divided into discrete steps?
      8. [ ] Is the scenario linear from start to goal?
      9. [ ] Is the scenario missing any sub-tasks?
      10. [ ] Is the task concrete?
      11. [ ] Is there a clear goal?
      12. [ ] Is there a way to make the scenario simpler or faster or
        less error-prone without changing the starting place or the
        goal?
  6. Back Top Next
    USABILITY EVALUATION BASED ON SCENARIOS
    1. Step 1
      From the scenario, build a "script" (i.e., a specific set of
      directions or instructions) for users to follow.
    2. Step 2
      Pilot test and refine the script.
    3. Step 3
      Identify representative users.
    4. Step 5
      Observe these subjects as they follow the script.
    5. Step 6
      Record data.
    6. Step 7
      Analyze the data.
      1. Do users stop before completing this script? Why?
      2. Do users make errors completing this script? Why?
      3. Do users take too long to complete this script? Why?
      4. Do users make errors on particular sub-tasks? Why?
      5. Do users have difficulty because of genuine usability problems
        or is the script itself to blame?
  7. Back Top Next
    EXAMPLE TESTING SCRIPT
    1. SCRIPT SUBJECT USES
      1. Insert the bank card into the slot on the ATM.
      2. Enter 3636 as your PIN number.
      3. Choose the option of withdrawing $50.
      4. Remove the cash.
      5. Decline the option of making another transaction.
      6. Remove the receipt.
      7. Remove the bank card.
    2. SCRIPT OBSERVER USES
      1. Insert the bank card into the slot on the ATM.
        1. Subject completed this sub-task in ____ without error.
        2. Subject completed this sub-task in ____ but with ____
          errors.
        3. Subject required ____ hints from the test administrator.
        4. Subject failed to complete this sub-task.
        5. Subject's self-reported intentions, thoughts and feelings
          (if doing a talk-aloud walkthrough rather than a timing
          session):
        6. Observer's comments:
      2. Enter 3636 as your PIN number.
        1. Subject completed this sub-task in ____ without error.
        2. Subject completed this sub-task in ____ but with ____
          errors.
        3. Subject required ____ hints from the test administrator.
        4. Subject failed to complete this sub-task.
        5. Subject's self-reported intentions, thoughts and feelings
          (if doing a talk-aloud walkthrough rather than a timing
          session):
        6. Observer's comments:
      3. Choose the option of withdrawing $50.
        1. Subject completed this sub-task in ____ without error.
        2. Subject completed this sub-task in ____ but with ____
          errors.
        3. Subject required ____ hints from the test administrator.
        4. Subject failed to complete this sub-task.
        5. Subject's self-reported intentions, thoughts and feelings
          (if doing a talk-aloud walkthrough rather than a timing
          session):
        6. Observer's comments:
      4. Remove the cash.
        1. Subject completed this sub-task in ____ without error.
        2. Subject completed this sub-task in ____ but with ____
          errors.
        3. Subject required ____ hints from the test administrator.
        4. Subject failed to complete this sub-task.
        5. Subject's self-reported intentions, thoughts and feelings
          (if doing a talk-aloud walkthrough rather than a timing
          session):
        6. Observer's comments:
      5. Decline the option of making another transaction.
        1. Subject completed this sub-task in ____ without error.
        2. Subject completed this sub-task in ____ but with ____
          errors.
        3. Subject required ____ hints from the test administrator.
        4. Subject failed to complete this sub-task.
        5. Subject's self-reported intentions, thoughts and feelings
          (if doing a talk-aloud walkthrough rather than a timing
          session):
        6. Observer's comments:
      6. Remove the receipt.
        1. Subject completed this sub-task in ____ without error.
        2. Subject completed this sub-task in ____ but with ____
          errors.
        3. Subject required ____ hints from the test administrator.
        4. Subject failed to complete this sub-task.
        5. Subject's self-reported intentions, thoughts and feelings
          (if doing a talk-aloud walkthrough rather than a timing
          session):
        6. Observer's comments:
      7. Remove the bank card.
        1. Subject completed this sub-task in ____ without error.
        2. Subject completed this sub-task in ____ but with ____
          errors.
        3. Subject required ____ hints from the test administrator.
        4. Subject failed to complete this sub-task.
        5. Subject's self-reported intentions, thoughts and feelings
          (if doing a talk-aloud walkthrough rather than a timing
          session):
        6. Observer's comments:
  8. Back Top Next
    USES AND LIMITATIONS OF SCENARIOS
    1. BENEFITS
      1. may be readily understood by both client and designer
      2. may be written and used very early in the design process
      3. may enable understanding of complex behavior by dividing it
        into a series of discrete actions and reactions
      4. may focus on an ordinary user task rather than some esoteric
        system functionality
      5. may provide a useful focus for usability testing
    2. LIMITATIONS
      1. may "multiply" since each new choice made at a decision point
        can result in a new scenario
      2. may be difficult to determine when to stop dividing tasks into
        sub-tasks, with the result that the scenario may be too
        granular or, more likely, not granular enough
      3. may be difficult to distinguish between a task and a sub-task
      4. may be difficult to validate when typical users do not follow a
        simple linear path to the goal
      5. may include detail irrelevant to the design
      6. may include detail that lies outside the HCI (human-computer
        interaction) loop
      7. may model only one small aspect of the total system
      8. may not distinguish between sub-tasks that require the computer
        and those that the user can perform unassisted
      9. may not represent the most important tasks
      10. may not represent the most typical tasks
  9. Back Top Next
    SCENARIOS AND WALKTHROUGHS
    1. A scenario is an OBSERVER'S description of what a user does as
      they go about some specific task.
      1. It has an OUTSIDE focus.
      2. Emphasis is on the user's visible MOTOR activities.
    2. A walkthrough is a USER'S self-description of what that user is
      intending/thinking/feeling as they go about some specific task.
      1. It has an INSIDE focus.
      2. Emphasis is on the user's invisible COGNITIVE and AFFECTIVE
        states.
    3. To perform a walkthrough ...
      1. the user is given a specific script to follow,
      2. the user "thinks aloud" as they go through the steps required
        to complete the script, and
      3. an observer or camera records what the user does (scenario
        information) PLUS that user's own self-descriptions of what
        they are intending/thinking/feeling at the time.
    4. Following completion of the script, steps listed in the
      corresponding task scenario may be annotated with self-reported
      information about the user's cognitive and affective states.