For a current list see <URL:http://www.cs.dal.ca/~jamie/pubs/>.
[ Jump to: detailed list | address | short list ]
[Jump to: detailed list | address | top of document]
Titles are links to details below.
Titles are links to details below.
Titles are links to details below.
Titles are links to the documents (at this site).
[Jump to: top of document | short list | address]
This detailed list is in chronological order, from most recent to earliest.
The overall objective of this work is to develop and evaluate ways of automatically incorporating hypertext links into pre-existing scientific articles. Some readers like hypertext even when it is not as useful to them as the linear document from which it was generated. Hypertexts must therefore be evaluated for usefulness and acceptability. We describe rules for making links and an experiment using two methods of applying those rules, to show how such rules should be evaluated, and to see if they truly help people. In addition to measures of performance we also collected measures of preference. The effectiveness of these links was evaluated by testing with people. Performance was determined by measuring the accuracy and inclusiveness of answers to questions about the article, and written summaries. Readers judged the quality of links (and thereby the quality of the rules used to forge them) and the overall effectiveness of the hypertext. Most readers did not read the entire articles in the time allotted. Readers had no preference for articles with or without novel link types, but they did have a strong preference for definition and structural links over (novel) semantic links. Readers of documents with only structural links had comprehension scores that were inversely proportional to their satisfaction ratings. No performance difference was detected.
My overall objective is to develop and evaluate ways of automatically incorporating hypertext links into pre-existing scholarly journal articles. I describe a rule-based approach for making three types of links (structural, definition, and semantic). Structural links are a way of making explicit some connections between parts of the text. Definition links connect the use of a term, defined elsewhere in the document, to that definition. Links that connect parts of text that discuss similar things are semantic links. I distinguish several types of semantic links.
I use two information retrieval (IR) systems (Cornell's SMART system and Bellcore's Latent Semantic Indexing) to select links based on the content of the articles. I conducted an experiment to compare the performance of the links forged using these two systems.
The effectiveness of the links (and the rules used to make them) is tested by people reading the hypertext versions for information under a time constraint. A within-subjects experimental design was used. Each of the nineteen experimental participants read one version of each of three scholarly articles in a different hypertext form (one had only simple links, the others had definition links and semantic links selected using one of the IR systems). Subjects' preferences were also measured.
Although I used three survey articles from published sources for my evaluation experiment there was no difference in reader preference or performance on the basis of article. Subjects ratings of the utility of the various links shows a significant preference for structural links over semantic links. Definition links were preferred to structural links, although the result was not significant. No difference between the links created using the two IR systems was detected.
However there were significant differences in the times that readers spent on documents created using the various treatments When they read in documents with only structural links readers were more likely to have read the whole article, and their satisfaction scores were inversely proportional to their comprehension score.
The method of evaluating hypertext versions of journal articles for use by researchers may be applied to other hypertext versions.
In this paper I present an experimental approach to the evaluation of a type of hypermedia application. My overall objective is to develop and evaluate ways of automatically incorporating hypermedia links into pre-existing scholarly journal articles. The focus of this paper is the evaluation method. My method allows the results to be applied to other documents than just those tested.
To properly convert ordinary documents into useful hypermedia two constraints must be satisfied: the links must be useful to the readers and the risk of disorientation introduced by the new structure imposed by the links must be minimized. I describe a rule-based approach for making links. In my experiment I use two methods to detect when the rules should be applied. The effectiveness of the links is tested by people performing realistic tasks. Readers judge the quality of links (and thereby the quality of the rules used to forge them) and the overall effectiveness of the hypermedia.
My overall objective is to develop and evaluate ways of automatically incorporating hypertext links into pre-existing scientific articles. Hypertext can support all of the ways we believe people use printed versions of articles and can add additional useful features. However many readers find hypertext confusing and not all texts are suitable for conversion to hypertext. Some readers like hypertext even when it is not as useful to them as the linear document from which it was generated. Hypertexts must therefore be evaluated for usefulness.
To properly convert ordinary documents into useful hypertext two constraints must be satisfied: the links must be useful to the readers and the risk of disorientation introduced by the new structure imposed by the links must be minimized. I describe rules for making links and two methods to detect when the rules should be applied.
I propose to provide links by applying rules based on the content of the articles using two methods: the Cornell's SMART system and Bellcore's Latent Semantic Indexing. I will evaluate the effectiveness of these links by testing with people. Readers will judge the quality of links (and thereby the quality of the rules used to forge them) and the overall effectiveness of the hypertext.
We present two methods for evaluating automatically generated hypertext links. The first method is based on correlations between shortest paths in the hypertext structure and a semantic similarity measure. Experimental results with the first method show the degree to which the hypertext conversion process approximates semantic similarity. The semantic measure is in turn only an approximation of a user's internal model of the corpus. Therefore we propose a second evaluation method based on measuring user's performance using hypertext. Finally, we discuss the advantages and disadvantages of computer versus human evaluation, respectively.
Bit vectors provide an extremely space- and time-efficent means of implementing arrays of Boolean values.
A program to compute descriptive and analytic statistics for the TREC IR trials.
A statistical analysis of the TREC-3 data shows that performance differences across queries is greater than performance differences across participant runs. Generally, groups of runs which do not differ significantly at large, sometimes accounting for over half the runs. Correlation among the various performance measures is high.
Useful hypertext is constrained by the need for users to be able to find documents about similar topics without extensive navigation. We show how examining the properties of a graph built by a document's hypertext links can be used to evaluate the usefulness of the document. To formally measure the quality of hypertext linking in a corpus, we compare the semantic similarity of pairs of documents with the minimum number of links between their corresponding nodes in an analogous hypertext graph. We use the measure of document-to-document similarity computed using latent semantic indexing as our measure of semantic similarity. Our method has been applied to a corpus composed of Usenet messages.
I have made some minor corrections to some of the tables since the presentation. A more extensive analysis appears in the IP&M paper.
Methods for automatically converting semi-structured text (Usenet messages) into hypertext form using information retrieval methods were investigated. The methods were evaluated using statistical means to determine which will produce hypertext best suited to browsing and searching. Methods were evaluated by comparing a measure of semantic similarity of all document pairs with the shortest path in a graph formed by hypertext links between those documents.
I have made some minor corrections since publication.
A manual for the Lathrop et al.'s LINKAGE programs for genetic linkage analysis (See `Construction of Human Linkage Maps: likelihood calculations for multilocus linkage analysis' Proc. Natl. Acad. Sci. USA 81:3443-3446, June 1984) It was written for versions 4.6 to 5.0 of the microcomputer version but can also be used with mainframe versions. The trouble shooting section (pages 18 - 27) is probably the most useful.
The latest version is 2.01 August 28, 1989.
[Jump to: top of document | short list | detailed list]
For a current list see <URL:http://www.cs.dal.ca/~jamie/pubs/>.
This document is copyright by its author, J. Blustein.