[Est.: 5th January 1995. Last revised: 15 February 1995. This facility is provided by the Australian National University (ANU). Usage statistics is listed here.]
by Dr T. Matthew CIOLEK, and Dr Warwick CATHRO
Introduction
On 18 October 1994 the Prime Minister launched the Government's Cultural Policy - the Creative Nation statement. It was a broad statement, dealing with more than what many people normally think of when they think of culture. And it included, amongst other things, important statements on information infrastructure and multimedia services.
The Creative Nation statement includes a chapter on Information for all Australians. That chapter stresses that information is a key resource for the Australian community; that the provision of information will be significantly affected by new technologies, and especially the Internet; and that libraries and museums are improving access to information through cooperative activities - or Distributed National Collection activities.
The statement also referred to multimedia services. It gave considerable prominence to CD- ROMs, and committed funding to the development of the Australian multimedia CD-ROM industry. However, the statement also said that the future of multimedia services lay not so much with CD-ROMs as with online, interactive, broadband services.
Now what has all this got to do with NDIS? First, when it referred to information infrastructure, the Creative Nation statement referred to the NDIS Project as an example of infrastructure improvement within the library sector, and one which would take advantage of new technologies and underpin cooperative DNC activities.
Second, the NDIS will be positioned to contribute to the future delivery of multimedia services, because its hardware and software underpinnings will be exactly of the right kind to support these services.
The NDIS Service
It will be useful to recall some of the key general characteristics of the proposed NDIS service.
NDIS is aimed at facilitating access to documents. In particular, it is aimed at assisting libraries and individuals to obtain documents when their local resource needs to be supplemented - when the local library or information resource does not have the document, or all the documents, that are needed. The NDIS should be particularly relevant when the search for a document needs to be broadened to a nationwide or international search. And by "document", we mean of course, not merely printed documents, but also multimedia and electronic documents.
NDIS will replace current services such as ABN and Ozline, and similar services in New Zealand. It will also extend these services: it will not only enhance them, but also provide services not covered at all by ABN or Ozline. For example, NDIS will provide online access to directory information, including the Conspectus database; it will provide direct full text access to some Australian and New Zealand document collections; it will provide sophisticated interconnection services such as uploading data and transparent search gateways.
NDIS will take advantage of the Internet. It is not intended in any sense as being a competitor to the Internet. The Internet is likely to be the most common access method; the NDIS communication protocols will be entirely Internet compatible; and in its gateways services NDIS will use and build on Internet-based links to other document resources.
Finally, NDIS will provide some shared infrastructure for the Australian and New Zealand library communities - and indeed for the wider Australian and New Zealand communities. It will support the development of a truly trans-Tasman library network, including shared cataloguing, interlibrary loans and possibly projects such as cooperative indexing occurring across the Tasman.
The Market For NDIS
One important difference between ABN and NDIS is that NDIS will place greater emphasis on services to end-users. We envisage that many people who need to get access to documents which they can't obtain from their local resource will use NDIS to search for, locate and access these documents. These people might include groups such as university teachers and researchers, students, and people who work in information-intensive professions such as doctors, lawyers, consultants and journalists.
Between December 1994 and March 1995 we will undertake a business planning and market analysis which will give us a greater understanding of our preferred markets for NDIS, and of particular databases and charging strategies which will be required for those markets.
However, we wish to stress that libraries will remain our core market. Furthermore, in most cases the services to end-users will be mediated through institutions which the end-user is affiliated with, such as the university or university library, or through a network to which the user might subscribe.
Specific NDIS Services
NDIS will offer the following groups of services to this market:
Document delivery services have been mentioned first because they are the key outcome, reflecting the goal of "facilitating access to documents". The search and gateway services will provide the necessary prerequisite to document delivery, and the data creation services will provide the essential underpinning for the search services.
It should also be noted that, in respect of each of these groups of services, NDIS will be an open system. That is, we will design NDIS so that it is possible to create a document delivery request either directly on the NDIS system or on another system, such as ILANET or a regional or local system. In the latter case, the request will be uploaded using the international standard Interlibrary Loan Protocol (ISO 10160/ 10161).
Similarly, we will design NDIS so that it is possible to create a search either directly on the NDIS system or on a local system or search client software. In the latter case, the search will be uploaded using the US and international standard Information Retrieval Protocol (ANSI Z39.50).
Further, we will design NDIS so that it is possible to create cataloguing or indexing data either directly on the NDIS system or on a local system. In the latter case, the data will be uploaded using standacols and data formats.
The document delivery services will be delivered in several ways. We will point to documents that are stored electronically on other computers; we will point to documents that are stored in traditional print form in libraries; and we will also host on the NDIS system a sizeable collection of Australian and New Zealand electronic documents which are not stored elsewhere. The NDIS will aim to support the delivery of these different types of documents, to support the process of requesting them (where they can't be obtained immediately) and to manage any statistics related to copyright royalties.
The gateway services will, wherever possible, use the standard Information Retrieval protocol (ANSI Z39.50). NDIS will support Z39.50 access to other services, and will also provide services using this protocol.
Data accessible through NDIS
NDIS will make available the following types of data:
The NDIS Solution
Last December, the two national libraries received tenders from five systems integration companies. From these five, we selected the tender by CSC Australia as providing the best solution for the future NDIS Service. The solution proposed by CSC is highly integrated; it will use state of the art computer technology; it will meet all of our essential requirements; and it provides the potential to respond to future changes in our requirements and in the technology environment.
Broadly speaking, the NDIS system will consist of three layers:
This design is an example of what is known as "Client Server" architecture. In this architecture one computer (the client) seeks services from another computer (the server). (In our case, we actually have two servers: an Application Server and a Database Server). Client server architecture allows improved performance by distributing the computer processing across more than one tier of computers. It also enables the cost effective addition of users to the network whilst maintaining performance.
A frequent use of the client computer is to provide a graphical user interface to the service. Typically, in the NDIS Solution, software on a client computer on the user's desk will be used to provide this interface.
A key issue which we have been debating is the kind of client software which will be used with NDIS. The libraries intend to develop NDIS-specific client software, which will provide value-added access to NDIS services, which will provide access to third party services using the Z39.50 protocol, and which will be distributed as freeware. We also recognise that some client software packages, such as Mosaic and Netscape, are already in widespread use. We intend that such packages will be able to access the NDIS for the more limited range of services which they will support.
The Hardware
The Application Layer and the Database Layer will run on a central computer, located at the National Library of Australia. This computer will be based on what is known as "massively parallel processing" technology. This technology allows further improved performance at relatively low cost by spreading the tasks across large numbers of processors. It also gives excellent "scalability": as the demand on the system grows, it is very easy to add more processors, to scale up to the next level of demand.
There are a number of new massively parallel processing platforms around. CSC has selected the IBM SP2 (or Scalable PowerParallel 2) which was released early in 1994, and which will meet our needs for NDIS performance at a competitive cost.
Typically, the Client Layer will run on a microcomputer on the user's desk. We are aiming to support a variety of client devices, including Windows-based PC's, Apple Macintoshes, and X terminals. We will be phasing the delivery of NDIS services, and that will include phasing of the NDIS client software. In the earliest phase, we will be supporting Windows-based PC's, and to run this client (and to make fullest use of the NDIS Service) it will be necessary to use a 486 PC, with a speed of at least 33 MHz and memory of at least 8Mb. If you wish to take full advantage of NDIS at the earliest possible stage, we recommend that you plan to purchase such a PC for each user of NDIS.
As soon as we can, but not in the earliest phase, the Project will deliver versions of the client software to run on Macintoshes and on X terminals. These devices will still be able to use NDIS in the earliest phase, but not as clients, and only with a text based interface.
Where there is a cluster of users at the local site, there will be performance benefits in having a further processor to act as a "controller" or communications front end. This controller may be attached to a local area network. The controller software would run on a Unix processor or on a 486 or Pentium PC, with a speed of at least 66 MHz and memory of at least 16Mb.
The database software
The Libraries' requirement for a powerful data management and retrieval facility will be met by a package of three software components to be supplied by the Oracle Corporation. The first of the components, the Oracle 7 relational database software, will support searches on structured data fields such as dates and coded cataloguing data, including searches on ranges of values.
Oracle 7 can also store multimedia data, such a pictorial images and sound, as Binary Large Objects or 'Blobs'. It uses an extension of the SQL standard based on the planned international SQL3 Multimedia Committee specification. Oracle can store or retrieve images in any format, including compressed formats. Oracle has developed a Multimedia Toolkit to support digital video and other new media types. This, combined with the massively parallel processing technology, gives some excellent prospects for the future use of the NDIS system to host images (still or moving) and sound.
The Oracle 7 software is totally compatible with the massively parallel processing architecture described to you earlier. The database can be physically partitioned, but searched as a complete entity.
The second component, the Oracle TextServer3 software, will support searches on text, including Boolean searches on keywords, with features such as word truncation, word proximity (one word adjacent to or near another) and relevance ranking. It also incorporates thesaurus support. This text retrieval software is fully integrated with the Oracle 7 relational database software, allowing searches on text to be combined with searches on structured data. It would also allow multimedia data (such as an oral history sound recording) to be linked with appropriate text (such as the transcript of the recording).
When we first investigated the redevelopment of ABN in 1989/90, we thought we would need to acquire and integrate two separate products: a relational DBMS and a text retrieval product. We now find that this integration has been achieved by Oracle.
The third component, the Oracle ConText software, will support advanced text analysis. It can identify the themes in unstructured text, and assign a weight to each theme. It can, for example, deduce from phrases like 'interest rates' that a document is a banking document. Its analysis capabilities are supported by a 600,000 word dictionary, in which each word can have up to 1,000 attributes or meanings. In addition, ConText uses a "linguistic parser" to analyse the grammar of each sentence and to determine whether a word (such as "ring") is used as a noun or a verb. ConText can also create abstracts or summaries, to any desired level, from the full document text. The use of the ConText software within NDIS is yet to be fully determined, but it will assist in improving the precision of searches by filtering out documents whose main theme is not germane to the search.
To summarise, the database software selected for NDIS has the following advantages:
Management of the NDIS Project
A Joint Project Director (Jan Rosi) manages the NDIS Project on behalf of the two national libraries. Jan is responsible for delivering to the Australasian marketplace an NDIS system which:
Over the past 17 years Jan Rosi has had considerable success in delivering leading edge technology products to the international financial information marketplace. She believes that the keys to successful project management include:
These principles will be applied to the NDIS project
Progress to Date
Observers of the NDIS Project will know that the two national libraries completed and issued the Request for Tender in October 1993. Following evaluation and negotiation, the selection of CSC Australia as the Prime Contractor was announced in September 1994. This means that CSC Australia will be consolidating and tailoring products from a number of vendors such as IBM, Oracle and Novell.
In late 1994 and early 1995, the two national libraries were negotiating with CSC a Systems Integration contract for all NDIS deliverables. The contract is expected to be signed by the end of February. In the meantime at the end of October the libraries signed an initial consultancy contract with CSC so that work could commence immediately. The two libraries have also been busy defining the requirement priorities so that we could break the project into phases and provide an early deliverable.
Phase Zero - The Analysis and Design Phase
Work has commenced on the analysis and design of the NDIS system. This work forms the foundation for the development of all phases of NDIS. It is a bit like building the foundation for a house - it is essential but you don't actually see anything which you can use. We have called this Phase Zero, and it will take five months to complete. It will result in:
As with the foundations of a house, it's important that we do this planning stage very thoroughly to ensure that we will be able to deliver all the services which users need.
Phase One - an early deliverable
To provide an early deliverable to users, and to reduce the risk to the project, we have defined the minimum requirements for an initial release of NDIS. We have called this Phase One. It is aimed at search intermediaries, academics, researchers, and professionals (such as doctors and lawyers) who need sophisticated information services at their workplace. Phase One requires further discussion with CSC to confirm what is technically feasible, and we will be announcing the confirmed contents of Phase One early in 1995.
However, these are our requirements for Phase One:
Our analysis has shown us that migration of the ABN and NZBN data to NDIS will be a complex and lengthy task. We have already started work on preparing this data for extraction from the current systems, and we plan to initially transfer only the data with holdings. However, if this data is not migrated in time for Phase One, we will release it to users after Phase 1 has been implemented.
First and foremost, we wish to deliver our services in Phase One via a graphical user interface - a pictorial point and click interface such as Windows or the Macintosh interface. This has been widely accepted as a user friendly means of interfacing to information systems. This will meet the market expectation for sophisticated services and the Graphical User Interface will enable the support of the highest level of functionality for NDIS services.
Because of technical limitations, the initial release of the Graphical User Interface client software will be made available only on a 486 PC, as described above. After Phase One, the Graphical User Interface will be made available for other platforms, such as Macintosh and X-terminal platforms.
The Text Based Interface
We will also provide, in Phase One, a text based interface for those users who do not have suitable 486 PC machines available. The Text Based Interface will facilitate the migration of users by not forcing them to upgrade their terminals. It will enable the ad hoc user to access NDIS from a broad range of terminals (with basic teletype or VT100 emulation). It will also provide support for Macintosh and X-terminal users in Phase One.
We do not, however, intend to limit the full Graphical User Interface service to the level of functionality that can be provided on the Text Based service. Hence this means of interfacing to NDIS may be limited in some areas.
Subsequent Phases
The detailed requirement priorities for subsequent phases will be confirmed closer to the development of each phase, but will include:
User Involvement
We see the early and continued involvement of users as essential to the success of the NDIS project.
We will continue to use the existing advisory groups in both countries to obtain information on user requirements and to confirm that our developments will meet these needs. We will also be inviting a small number of expert users in each country to provide detailed knowledge and feedback on particular aspects of the development (we are calling these "focus groups"). In addition, we will make early prototypes of services available to as broad a group of users as possible, via the Internet. Finally, towards the end of each Phase, we will make early stages of the services available to pilot users to confirm that the service will meet requirements.
Naturally, we intend to keep all current users fully informed about the Project by a variety of methods including:
Progress report for the 3rd National Roundtable on Libraries and Asia
The CJK Project and the Roundtable
The project which has become known as the National CJK (Chinese Japanese Korean) Project was foreshadowed at the 1st National Roundtable in 1991, at which the following resolution was passed:
That the National Library, the Australian National University, the University of Sydney and the University of Melbourne meet to explore the concept of a shared local system which would support CJK and other vernacular script data.
Progress since the 1st Roundtable
1992 ARC grant to ANU for a feasibility study
1993-4 ARC grant for identification and acquisition of a suitable system, to ANU, as the lead institution of a group of four: ANU, University of Sydney, University of Melbourne and the National Library.
Monash University, Murdoch University, Griffith University and University of Queensland joined the consortium.
1994 Innovative Interfaces and Digital selected as the successful tenderers for supply of the system.
Jan. 1995 Contract negotiations with Innovative Interfaces in progress.
Mission and services
Along the way the CJK System acquired a mission statement:
To support Australia's closer relationship with Asia by providing Australian libraries and their clients with improved access to the distributed national collection of Chinese, Japanese and Korean library materials. The services to be provided by the CJK System are to:
Current status of the Project
At the time of writing this paper - 16 January 1995 - contract negotiations with Innovative Interfaces were still in progress. The implementation date will be April 1995 or later, depending on the contract negotiations.
Governance
The CJK Project is presently governed by a Steering Committee, with and advisory Technical Committee. Membership of the Steering Committee is as follows:
Michael Evans Australian National University Jeanette Rosengren Griffith University Edward Lim Monash University V. Nadanasabapathy Murdoch University Tony Arthur University of Melbourne Judith Wilson University of Queensland John Rodwell University of Sydney Warwick Cathro National Library of Australia Eric Wainwright National Library of Australia (Chair)Governance of the CJK System after implementation has been a topic of lively discussion. At the time of writing this paper, discussions were continuing.
Extension of Use of the CJK System to other libraries
The present eight members of the CJK consortium wish to encourage other libraries to join and to use the System. Efforts to market the System have so far been low-key but will be stepped up once the System is operational. At least two levels of use will be offered - a level allowing database searching only, and a level allowing contribution and extraction of data as well as searching.
Overseas libraries have shown interest in the CJK System from various points of view - as an experiment in multilingual systems, as an example of co-operative infrastructure, and as a database that will be accessible from overseas. It may be worthwhile to encourage this interest, both to encourage use, and thus revenue, and as a possible basis for document supply.
Charging
Charges have not yet been set, but it is likely that the charges will be structured as an all- inclusive monthly subscription, according to the number of concurrent sessions the subscriber expects to use and the level of use.
Access
Access will be available through the Internet, and for ABN users through a gateway from ABN. Access from ABN will be limited to a Roman-script-only view of the database. The National Library will provide, in the Asian Collections Reading Room, a PC which readers can use to search the CJK System free of charge.
Equipment
Users who wish to input CJK characters in their searches or cataloguing data will need a microcomputer or X terminal, third party software, and a connection to the Internet. Three third party products - ETen, Join and MASS - were evaluated by the CJK project. MASS is recommended, though any of the three may be used. For MASS, an X terminal or a 486 PC running an X Terminal emulation is recommended.
Data sources
Although the CJK System libraries will be able to contribute original cataloguing, the bulk of the database will be acquired from other agencies and networks. The initial data load will contain:
Standards
Because of the variety of data sources, the CJK System will face problems of differing standards. The problems will include different character sets, Wade-Giles vs Pinyin, Pinyin word division, and different cataloguing rules and subject thesauri. There is no easy answer to these problems and they will inevitably have an effect on precision and recall of data on the System.
The CJK Project - Links to Overseas Resources
The National Library of Australia is attempting to strengthen
resource discovery facilities for Asia-Pacific (and other) documents
through two major projects:
URL http://coombs.anu.edu.au/SpecialProj/NLA/AP-Net-Futures2.html
Copyright © 1995 by Coombs Computing Unit, ANU & Australian National Library. This Web page may be linked to any
other Web pages. Contents may not be altered.
[ Asian Studies WWW VL ]
[ Coombsweb ]
[ Social Sciences WWW VL]
Vernacular Scripts - Internet resources
These projects need to be seen in the context of a broader national
collection access strategy, also known as the Distributed National
Collection strategy. They also need to be seen as complementing, and
linking with, Web pages and other Internet discovery tools. These two
projects mean that, for most of the late 1990's, there will be physically
separate (though linked) national union catalogues for materials in East
Asian scripts, and materials in roman scripts. However, it remains the
firm policy of the National Library, once the NDIS system is developed, to
work to extend that system so that it incorporates support for East Asian and
other vernacular scripts, using an integrated universal character set
based on Unicode.
Recommendations to the 3rd National Round Table
[Draft]
Bibliography and Further Reading
Introduction *
The Internet - A Brief Overview *
Growth of Internet in the Asia-Pacific region *
Asian Studies resources available over the Internet *
National Library of Australia projects *
Vernacular Scripts - Internet resources *
Recommendations to the 3rd National Round Table *
Bibliography and Further Reading
Maintainer: Dr T.Matthew Ciolek (tmciolek@coombs.anu.edu.au)