ALEG

What I've done

• ...nothing much new, just a lot of little fixes and enhancements and tuning of the database queries, which is all quite time-consuming

• Set up some scripts to automate daily and semi-daily routines - database backup, index and statistics housekeeping, and log rotation

• Fixed a major problem of the same title topic being 'shared' by many works loaded from the original Austlit. This came to light when a single 'Untitled Review' title was updated, and it changed all the 'Untitled Reviews'!

• Started coding a program to process the illustrators from the BAL load on Tuesday, but haven't had a chance to return to it yet.

• Added phrase searching: enclose the phrase in double quotes, eg "merry christmas"

• Add general edits based on XSLT Schema (the Schematron approach).
• Revisit output formatting (especially to show all the newly added attributes and relationships) and search screens.
• Drama data - work out what we need to store to record a "performance"
• AMLC bib data load
• Lu Rees Bib load (MARC format)
• WA writers data load

• System documentation

• BAL loading

• Investigate a problem with diacritics (AMLC bio's) being mangled when updated

• Discuss the approach we'll take to the Lu Rees non bibliographic data

• Enhance expression/manifestation selection search

• Remove the test works and agents added during training

• A very busy week responding to queries and feedback. We've noticed that query response times are very sensitive to the Oracle optimizer's interpretation of the current statistics which 'profile' the database and help it to determine the best query path. I spent a lot of the week fiddling with the knobs, generating different statistics and measuring the results. My aim is to establish a stable environment with predictable performance. I think performance right now (Friday afternoon!) is quite good, and next week I'll be making simple changes one at a time to see what destablises it.

Links of the week

• Clay Shirky Explains Internet Evolution

• Response of scientist at Carnegie Mellon University to legal threat from the Motion Picture Association to remove information about the DeCSS algorithm

  Movie DVD's are (poorly) encrypted. When someone published a program to allow them to be played by anyone (not just specially 'licensed' hardware), the MPAA got very upset (background on the hack is available here; you can even buy a 'forbidden circumvention device' t-shirt). Dr. David S. Touretzky of CMU has a point to make on the futility of the MPAA's legal approach. I'm sure this response had the MPAA's lawyers scratching their heads...