J. Blustein's PhD Thesis in Brief

Back to [ Homepage (Experimental) | Homepage (Official) | Hypertext Refs ]

Introduction

Electronic versions of journal are already appearing. I believe that scholarly articles with automatically generated hypertext links can and will be more useful to readers than versions without links. However, as with all hypertexts, we must ask if the additional features of hypertext documents are worth the price in cognitive overhead.

The overall objective of my PhD thesis was to review, develop and evaluate ways of automatically incorporating hypertext links into pre-existing long scholarly articles. I chose to work with lengthy articles because preliminary experiments with short articles seemed to show that hypertext links did little to improve them. The completed thesis presents the background of my work (what is hypertext, why might it be helpful, how can we automatically generate it, a review of research about reading from computer screens and using hypertext), a description of the methods used to generate hypertext links in scholarly articles, the results of experiments with people using the hypertext thus generated, a discussion of the importance of those results, and some directions for future work.

Method in brief

Link creation

[forward to `Evaluation']

I developed my method of generating hypertext links from work James Allan did for his PhD dissertation. My method generates HTML versions of scholarly journal articles. The articles contain three types of links:

structural links

which mimic automates non-linearity already present in traditional documents, e.g., cross-references and footnotes

definition links

which connect the use of a term with its definition in the document

two types of semantic similarity links

summary links

which connect summaries to the beginning of the passage they are about

related links

which connect discontiguous passages that are about the same topic

Each link type is created by applying a rule. The quality of the links depends on how good the rules used to create them are and how well those rules are applied.

Structural links were created mainly through the interpretation of mark up in the text. Candidate targets for definition links were identified manually. Semantic links were generated two ways, each using a different information retrieval system:

1. using a variation of Cornell's SMART system
2. using Bellcore's latent semantic indexing (LSI) approach

Evaluation

[back to `Link creation']

Tests were performed to evaluate how useful such links are to readers. I evaluated the performance of the system for reader comprehension (to ensure that the system is really helpful) and measure readers' satisfaction with the rules used to build links. I also collect some additional information about about the readers in case such information is needed for future work.

Each participant read three HTML documents: one with links created using SMART, one with links created using LSI, and one with only structural links. The third version served as a control for the other two. I used a total of nine documents (three articles, each in three forms). No reader saw two versions of the same document and all orders of presentation were used.

While they read each of the documents, readers performed a comprehension task: answering questions about the article and briefly summarizing it. The summaries were scored for accuracy and thoroughness. After they have read each of the documents, the readers rated the usefulness and appropriateness of a selection of links in the document.

I wanted to determine how well the rules worked and how well the two programs are at detecting when to apply the rules. A fractional repeated measures design was used.

Completed document

I (successfully) defended the completed thesis on 12 April 1999. A copy of the completed thesis, titled Hypertext Versions of Journal Articles: Computer-aided linking and realistic human-based evaluation, is available for download from the UWO Computer Science Department's FTP site (at <URL:ftp://ftp.csd.uwo.ca/pub/thesis/Blustein.PhD.Thesis.ps.gz>). The file is in Postscript form and compressed with the gzip program.

A copy of the abstract is in my publications list.

What next?

It should be no surprise to anyone who has completed a thesis that the experiment could be improved through further iterations. The future work chapter of my thesis describes some of the experiments that I'd like to do. In particular I think that the experiment should be rerun with a longer reading time, and that the readers should be selected with greater precision. For instance readers could be more interested in the topics of the article, or have high spatial ability.

I am more interested in pursuing the new interface and annotation parts of the future work. If you found those sections of the thesis interesting then you might be interested in the interface projects listed on my what next page.

See Also

http://www.csd.uwo.ca/%7ejamie/Official/Proposal/thesis-brief.html

Homepages:

official

experimental

This document is copyright by its author, J. Blustein.