ALEG

What I've done

• Megan (before she left for the National Library) and Marlene (about to leave UC) reviewed the 250+ EAD files I produced for the Lu Rees archives and proposed dozens of changes. I finally implemented most of the requested changes and zipped the results to Marlene.

• Started work on the Advanced Search. The Advanced Search selection user interface is almost ready for testing, but a lot has to go on "behind the scenes" yet to search for the new attributes exposed by the search. Hopefully this will be completed next week. I've taken the approach of not using client-side javascript which, whilst a bit more cumbersome (for the system and the user), should allow all users to access the Advanced Search intuitively and browser-independently.

  There are some tough decisions to be made regarding boolean combinations, as I can't see any logical and simple way of implementing these until we support combining search results. For example, if someone searches for:

       title:  Bush
  and  born:   1920
  or   place of birth: England
  and author:  Fred
  

  then does the user mean:
  

     (Bush and 1920) or (England and Fred)
  

  or do they mean:

     Bush and (1920 or England) and Fred)
  

  ?

• Now that the site is public, it has been found by several crawlers. The "ArtsInfo" crawler (originating from www.artsinfo.net.au aka www.acn.net.au) has been particularly enthusiastic, and doesn't seem to honour the robots exclusion meta tags (as does, for example, the GoogleBot). So, I've taken to excluding this agent from some parts of the site, as it was attempting to crawl everything, which would amount to over 2 million pages (and would result in tens of thousands of Kinetica lookups!). This accounts for the thousands of "403 Access forbidden" status responses.

  Although we have been heavily crawled by Google, it could be up to 4 weeks before we turn up in their index.

• Report of most criticised agents to help drive Roger's work.

• The system rebooted itself late Wednesday. Unlike the last spontaneous reboot a few weeks ago, it managed to log some diagnostics (clearly describing a hardware error) and "sync" the disks before rebooting (so when it came up again, the filesystems were "clean").

  Fran reported the incident to Sun, who thought that the diagnostics did not provide enough information to determine the cause, and want us to wait until it happens again (this was the 3rd spontaneous reboot since we started using this machine).

  After asking a "Sun Help" news group for reports of similar incidents it seems that this model (Sun Blade 1000) has a history of problems caused by the CPU and memory not being seated properly on the system board.

Next Week

• Complete Advanced Search (which will include keyword anywhere searching).

• William noticed that some of the place-of-publication data for loaded records is wrong where the name of the place of publication (town or city) occurs in more than one state (eg Richmond, Glebe). I'll investigate when we map the spatial thesaurus. Dan, Chris and Terry have also noticed similar incidents, so I think a very careful look at all place assignments to places occuring in multiple states/countries is warranted. (And Joan noticed Aberdeen NSW -v- Aberdeen Scotland!)

  I'll produce some reports/tools to help diagnose and correct these problems.

• Merge subjects

• Multiple creation events for a work as a mechanism for allowing date ranges to be associated with agents responsible for works, eg editors of a periodical.

Links of the week

• KurzweilAI.net

  When we were thinking about visualisation techniques for all the data we were planning to store almost 18 months ago, one of the models and implementations we looked at was The Brain.

  It seems to have been refined since, and the implementation at Raymond Kurzweil's AI site is quite intriguing. I what if any of the data we have would benefit from such a display?

  I find our current thesaurus display a bit strange, with the narrower terms "at the top", and the siblings "mixed in". An alternative "spatially consistent" view works perhaps better (for me, anyway) when a term has just one parent, but maybe not so well for terms which appear at many places in the hierarchy - ie, when the expected hierarchy breaks down.

  The funny thing about "The Brain" approach is that up/down/left/right don't seem to have any meaning as far as I can discern, but maybe that doesn't matter - maybe thinking "hierarchically" is limiting and "thinking old"!

  The thesaurus is only one, relatively simple part of our "web" of data. What if we were to display works related to thesaurus topics, or link thesaurus topics based on the number of works containing groups of terms? Would it be enlightening or just confusing? And then there is the relationships between agents and works and our other dozens of attributes!

  Maybe we could just use it to show one set of relationships at a time, such as "influenced by" agents and works. Clicking on a node would "centre" it and show the agent/work in the bottom pane... We certainly have our data in the right format to build displays such as this!