Personal Glossaries on the WWW

4. Methodology in Brief

[skip to content]

User Interfaces

We used three interfaces within a web browser in our tests:

  1. one interface had no special features (and is not shown here);
  2. one interface presented a glossary containing pre-defined terms (shown in Figure 4-I-1); and
  3. finally an interface much like the previous one but that allows users to update the glossary (shown in Figure 4-I-2 with numbers labeling parts of the interface).

screenshot of three part glossary interface
Figure 4-I-1: Automatic glossary tool interface
[select this link to view image without surrounding text]

The more complex interface is shown in Figure 4-I-2. This second interface is for the user-updateable glossary tool. It is identical to the basic interface (shown in Figure 1) except that it includes a toolbar at the bottom of the window (labeled (4) in Figure 2) which allows its users to modify, delete, and add new entries to the glossary.

(Note that the labels ((1)(4)) are not  part of the interface but are used only for the description below.)

The interface for the user-updateable glossary tool (shown in Figure 2) is divided into four sections:

screenshot of four part glossary interface
Figure 4-I-2: User-updateable glossary tool interface (with parts labeled)
[select this link to view image without surrounding text]


Experimental tasks

We tried to simulate a real-world condition in which people would want to understand a text with unfamiliar terms. We asked experimental participants to read online articles about diseases with the goal of understanding their severity and treatment options. Participants were told to imagine that they, or a child in their care, had recently been diagnosed with the condition as motivation to understand the text.

The choice of health conditions, and whether participants began with a glossary or not (and which glossary they used) was randomly assigned.

When the participants had completed reading the articles and answering the questions about them, they were asked to complete a questionnaire measuring their subjective appreciation of the glossary tool they were given to use. A debriefing session followed where participants were asked to express any comments about their performance during the study. The debriefing allowed participants to openly express their views on the software and describe what areas they felt required improvement. It also proved useful in the collection of subjective data from the participants.

Participants and materials

Forty participants from various backgrounds took part in the study. All participants were familiar with the use of computers and as such, represented the target audience for the glossary tool. Self-selection bias was not expected. Therefore, the presence of any skewing due to selection bias is not expected in the results.

The articles used for the study discussed asthma, bronchitis and influenza. These are health conditions that affect a considerable part of the general population. A large number of individuals may therefore be interested in reading about these conditions. It is likely, however, that these individuals are not familiar with the specific details presented in the articles chosen. This would give the participants incentive to read the articles with greater attention to detail.

Articles were adapted from the WebMD, [2002] website. Glossary terms were chosen manually and definitions extracted algorithmically [Kaindl et al., 1999].

Sketch of experimental protocol

Experimental design

The experiment was a mixed design: the comparison between the glossary and non-glossary treatment was within-subjects (in which every participant acted as their own control), but the comparison between the two glossary tools was between-subjects. A total of forty participants completed the study. Every participant undertook two sessions, each with a different glossary type and document.

Measures

We recorded:


References

References for works cited in this text chunk appear below. References for all works cited are available in a separate chunk.

[Kaindl et al., 1999]
Hermann Kaindl, Stefan Kramer, and Papa Samba Niang Diallo. Semiautomatic generation of glossary links: A practical solution. In HT'99, pages 3 – 12, 1999.
<DOI:10.1145/294469.294473>.
[WebMD, 2002]
WebMD Corporation. WebMD About Us. Copyright 2004, Last retrieved 05 September 2004.
<URL:http://www.webmd.com/corporate/index.html>.

This document is written in valid XHTML 1.0 & This document makes use of cascading style sheets.

[Up to navigation links]