ALEG
Weekly Report - Week 34, 21 December 2001
What I've done
- The changes made to add customers and customer groups have been
integrated into the system authentication procedures. Authentication by IP-address
or userId/password is now operational.
- Simple, guided and advanced searches are now being logged. Information
recorded is: date/time, userid, search terms, search type (agent/work), SQL
query time (measures time to first hit), total query time (measures time
to last hit), number of hits.
A query facility has been implemented to allow
data maintainers to select, sort and view search histories.
- There have been a few problems with the system this week. Over the weekend,
an address in Italy used a "web crawler"/"offline browser" to crawl the site.
Over 100,000 pages were delivered by 5am Sunday, when the web server hung with the
same problem we've seen before (the Java Virtual Machine hanging - Sun bug
number 4472904).
After restarting the web server, the crawl from Italy continued, so I blocked
the IP address and user agents claiming to be crawler software package.
Although possibly a bit antisocial (the crawler launched 10 simultaneous
requests against our site, tending to block other users), such activity
should not be "life threatening", and the hanging of the JVM is becoming a real
problem. It seems to be particularly vulnerable under heavy load or having just
been restarted - although it coped all over the weekend, it hung several times
Monday whilst I was restarting the web server, and has hung on a total of 4 occassions
this week.
In response, I've implemented a script which checks the web server every
5 minutes, and if it doesnt get a response within 12 seconds, kills the web server
and restarts it. This is a bit like putting a bucket under the leaking roof rather
than fixing it, or topping the engine up with oil, but until Sun fix the problem
in the JVM, we have little option. Other JVM's (such as IBM's) are not available
for the Solaris platform.
Next Week
- Finalise implemention of customer accounting - count access to works and agents
and develop reports for customers.
- More tasks from Kerry and Annette's list.
Next few weeks
- Program to help linking of full text images to works.
- Multiple creation events for a work as a mechanism for allowing date ranges
to be associated with agents responsible for works, eg editors of a periodical.
- Refining NBD Holdings searches.
- Combining searches