Blog

The second session this morning once again returns us to the International Internet Preservation Consortium (IIPC). Julien Masanès, IIPC coordinator from the French National Library, will team up with Monica Berko, Director of the Applications Branch at the National Library of Australia. Julien begins by speaking on the IIPC's techniques for deep Web acquisition - the archiving of resources which are deeply hidden within Websites and which often constitute the richest content on the Web (and therefore form a crucial task for Web archiving). Originally, much of this material was inaccessible to Web crawlers, but smarter tools have now changed this.

While the major part of this conference finished yesterday, we've still got another day to go. Billed as the 'information day', today will cover many of the technologies and projects which have been mentioned over the last few days. I'll try and take in as much of this as I can, but I do have to run off to the airport by 4 p.m.; this means I will miss some of the talks on what's happening at the National Library of Australia which very humbly have been placed last on the programme. Turnout today is somewhat smaller than the 200 or so delegates over the last few days, but still very good - I also have a feeling we'll be suffering from acronym overload by the end of the day…

On to the final session now - and in fact the final session of the conference proper (tomorrow is billed as an information day on the various archiving projects). Speaking now is David Seaman from the Digital Library Foundation (DLF); his organisation is involved in a wide range of projects across the many topics and issues raised in the conference.

He notes that 'the chaos isn't slowing down', where new and possibly important formats and genres of Web content are constantly arising (but where it is difficult to work out what is relevant and likely to continue further and what isn't). Libraries, at least, may have some degree of expertise in this field and will be able to make some useful guesses if nothing else. There is therefore also an imperative to collaborate, it really is a survival skill for libraries and related organisations, but that doesn't necessarily make it any easier.

Some very interesting discussions over lunch, especially with Ian Oi from Blake Dawson Waldron (lawyers to the National Library of Australia as well as key collaborators with QUT on the translation of the Creative Commons framework into the Australian legal context). Talking to Paul Koerbin from the NLA also reminded me that the changes we've made to the server setup of M/C - Media and Culture (now using the three sub-domains more effectively) recently may mean that the archiving of the site by PANDORA as it's happened so far may now not work so well any more - I'll have to check back with the NLA to make sure there are no longer-term problems.

The next session is chaired by Ross Gibbs, the Director-General of the National Archives of Australia. We're now moving into issues especially also around archives (as opposed to libraries). Hans Jansen from the Royal Dutch Library makes a start. Like many others, the library is charged with preserving all publications by Dutch publishers, but there are no legal deposit requirements in the country, so voluntary agreements with publishers have been made. More recently, of course, the rise of electronic publishing has further complicated the library's activities. Since 1994, it has been involved in developing e-Depot, a deposit system, in partnership with IBM (the system is also commercially available under the name DIAS). It now has a load capacity of some 50,000 articles per day, and contains some 4 million electronic journal articles. (So, the focus here is on archiving deposited materials, not the wider Web as such.)

The last session for today has started. Colin Webb (how appropriate!), Director of the National Library of Australia's Preservation Services sets the scene, noting that 'preservation' means maintaining the ability to access content. Layers of responsibility include byte stream integrity, byte stream identity, and the preservation of intellectual content for each digital object that is preserved, but also the preservation of original context, current context, and 'significant properties' or essential characteristics.

However, there are some reasons for hope here: the incentive is one of taking steps and building collections, and this has driven some very promising projects already. Also, the preservation problem may break down into some more manageable segments: byte stream protection, means of access, and metadata and systems. Additionally, it is possible to make informed decisions given the limitations of known means of access; we can work on specifics and push towards automation and towards a collaboration beyond research (building networks of capacity).

Nice Shot?

We're on to the post-lunch session, now, and had our group photo taken as well. Tom Delsey begins by discussing issues around resource discovery and archived resources. A lot of this is connected to metadata - and importantly, given the sheer size of their Web resource collections, are archives and libraries able to sustain the creation of metadata about collected resources as they have done it in an offline context?

And we continue with another session on harvesting approaches for Web content archiving and preservation. We begin with Martha Anderson from the Office of Strategic Initiatives at the Library of Congress. Its problems are probably larger than those of most other libraries - there is no clearly defined country domain, and the volume of material is of course significantly higher than in most other countries.

The library therefore takes a thematic approach to its collection - both identifying ongoing themes and time-bounded issues (elections, the 11 September attacks, etc.). The goal of such selection is to save as much as possible with limited resources, and preserving the context of content as much as this can be achieved. At this point, the LoC is required to seek permission to display and (in the case of event-based harvest) to collect. In doing so, it attempts to leverage its institutional resources to achieve a higher volume of coverage.

We've now moved into the second day in Canberra; this is kicked off by Abby Smith, Director of the Council on Library and Information Resources in the US, speaking on the future of Web resources. She suggests that the strategies for selection and preservation by libraries will need to be rethought; here, the barriers to the creation of content are now unprecedentedly low, while those to persistence of information and unusually high, while library approaches so far have been based on a scarcity of archivable material, but relatively easy archivability.

The Web is massive in scale, highly dynamic and unstable, and riddled with hardware dependencies. How is it to be dealt with - what to preserve, for how long, and for whom? Cooperation and coordination between collecting institutions here will be difficult, even if it may be desirable; access is always a service for a specific community, and there may be no universal, global needs upon which to build. Cooperation is highly problematic in collecting, and at best it may mean that the aggregation of local collections will enable a solid combination of material in a broad range of fields.