The World-Wide Web Virtual Library [Alphabetical || Category Subtree || WWW VL database|| WWW VL Global Search]

Edited by Dr T.Matthew Ciolek

Est.: 8 Apr 1997. Last updated: 1 Aug 2001.

This page discusses rationale and methodology underlying the rating/evaluation process for quality or usefulness of intellectual content of a number of web sites listed in the Asian/Pacific Studies WWW Virtual Libraries, and - especially - in their two electronic journals, Asian Studies WWW Monitor and Pacific Studies WWW Monitor.

Several of the WWW sites listed in these Virtual Libraries have received a tag (such as a word-label or a graphics file with a couple of "stars" indicating their (a) online visibility; (b) quality or goodness of their intellectual content.

As the editor of this and related VL pages I would like to acknowledge that evaluation and rating of online information resources is both a complex and complicated enterprise. See, for example, extensive discussions and proposals listed in a section of the Information Quality Virtual Library dealing with Evaluation of Information Resources. See also an overview of some of the related methodological problems enumerated in my 1996 paper on "Current Approaches to Information Quality in WWW Resources".

Essentially, there is a strong tension between methodological and operational soundness of a rating procedure. Procedures which are relevant and thorough tend to take, as a rule, a lot of time and conscious effort to consistently apply them in everyday situations. On the other hand, the large volume of online information that arrives daily at our electronic doorsteps require that its analysis or evaluation should be carried out as quickly and as efficiently as possible, without creating backlogs, variations or irregularities in the way information is treated.

While such an ideal methodological tool combining soundness with accuracy, consistency and speed may eventually be constructed and disseminated on the web, it is important that some minimal evaluative operations be performed today on the materials one works with, so that we can start training ourselves in critical thinking about our online work environment and about the manner with which we go about our electronic publications.

Therefore, for the purpose of electronic activities of the Asian Studies WWW Virtual Library I have developed a simple procedure.

(a) The number of "External links to the resource" uses data provided (in response to a command link:URL -host:URL) by Altavista Search Engine (www.altavista.com).

(b) Also, an online resource may receive, on the strength of the integrity, trustworthiness and SCHOLARLY usefulness of its content ( = I am leaving aside separate and immense questions of the site's navigational structure and accessability as well as those of presentation, layout or aesthetics) one of the following five tags:

Essential	a 5 star site
Very Useful	a 4 star site
Useful	a 3 star site
Interesting	a 2 star site
Marginal	a 1 star site

Some further clarification, however, might be in order here. An ESSENTIAL site or resource (from the point of view of a scholar, a postgraduate student, a researcher) is one in which the online information is

current

factually accurate

clearly attributed to a particular author and his/her institution

annotated

supported by adequate references, and

ample

Please note that the sites dealt with in this Virtual Library are evaluated in their own terms, that is without reference to other information systems/pages/documents dealing with similar issues or topics.

This means that in trying to decide whether a resource merits this or that rating I ask myself how much and how confidently could an experienced researcher carry out online work on the basis of a given set of online materials. Normally an answer to such a limited question can be reached within a couple of minutes of a quick, but attentive inspection of the intellectual content of a given site.

Obviously, this approach is quite different from asking onself how does the material compare with things seen elsewhere on the Web ?

In sum, what I currently use is a very rough yet handy measure of the research usefulness of the encountered online materials. I do not compare between sites, instead I evaluate each of them as if there were individual and unrelated specimens.

This is the measure which I will try to apply as consistently and as dispassionately as possible, until the time when a more rigorous, yet equally quick method is developed.

Please, do let me know, if you hear about a superior tool, procedure or technique. I would be very interested in trying it.

T.Matthew Ciolek