Federated Search limitations

 Digital Libraries, Innovative Interfaces  Comments Off on Federated Search limitations
May 242006

Today we got to see first hand the fragility associated with the current state of federated search technology. As a beta site for Innovative, we have been testing their 2006 general release software prior to general distribution and an update last night caused ports to fail, resources to be limited and screwed up the keyword index searching leading to wonky results. The problems were first noticed by our Technical Services group — but were seen later in LibraryFind — which searches our III catalog remotely. The problems introduced by the updated have essentially crippled outside interaction with our ILS — and while this will be quickly corrected by Innovative (I hope) — it does underline something that I’ve been aware of throughout this process of working on LibraryFind — that being our current model, the federated search model, is broken.

Why broken? Well, within the federated search model, there are just to many unknown variables that the system literately has no control. Start with query normalization. Targets will interpret user queries differently across the board meaning that a query at on target will search very different from a query at another. This leads to some difficulty as any tool created then must either code specifically for each target or provide a generalized search that normalizes to the most common target format. Likewise, as seen with our ILS, sometimes targets fail. And when they fail, there is really nothing that the system can do but report the error to the user.

So how do we fix the model? This is why LibraryFind is as much a harvester/indexer as it is a federated search tool. Our underlying belief while developing this program is that we want to harvest and index as much data as possible — so the tool is setup that way so that as vendors become more comfortable allowing their data to be harvested — we can take advantage of it. Of course, this can bring its own set of issues to the forefront — but I would gladly deal with database/indexing related issues over the current federated searching issues.

So what are we planning on doing with our ILS? Well, at this point, we will continue to remotely query our data — but we are in the process of looking at ways to simply harvest all our data and index it locally. The problem of course, is that this needs to be a process that is is in fairly real time — and I’m not sure if we will find and exporting method that can support those requirements, but we shall see.


 Posted by at 6:53 am

Extra, extra…LibraryFind kicks MetaFind’s, um, bottom

 Digital Libraries, Innovative Interfaces  Comments Off on Extra, extra…LibraryFind kicks MetaFind’s, um, bottom
May 222006

Actually, that’s probably a bit overstating it and its still fairly early, but with MetaFind over the past year, our average search stats on a good day was about 70 searches.  Since Friday (include the weekend) with no publicity and a soft launch — we’ve had 798 cached results and ~1200 searches.  I’m hoping that these numbers actually increase quite a bit.  Looking at our OPAC, we receive ~8000 searches a day on our OPAC and I want all of them.  I’m hoping that by the end of the fall term, we’ll be handling over 1/2 the searches being done in the library on LibraryFind — though I have a feeling that a more realistic number may be a 1/4 to a 1/3.  We’ll see.  Hopefully the additional searching hasn’t been just curiosity over a new service.  I’ve also only seen 4 target errors (each one was our ILS) and have actually received a little bit of unsolicited and positive feedback from folks here at the IUG conference as news of our release has filtered out.


 Posted by at 10:21 pm

Believe it or not…

 Uncategorized  Comments Off on Believe it or not…
May 222006

So, I’m getting ready to come home after the IUG and the Denver Hyatt is hosting a new conference, and I’m not making this up, the Society of Government Meeting Professionals.  Ugh…I’m sure its different than the picture in my head but…

Updated:  I stand corrected — it was actually worse than I thought.  I made the mistake of looking up the conference session list…well, I don’t actually know how to say this nicely…we librarians are obviously amateurs at this meeting stuff.  On the bright side, I’m actually looking forward to getting back to my library and attending some meetings. 🙂


 Posted by at 8:35 pm

IUG — new stuff in 2006

 Innovative Interfaces  Comments Off on IUG — new stuff in 2006
May 212006

Given that I sometimes give III a bit of a hard time (which I think is part of our job as users) — I thought I would note some of the very cool things that are out in the 2006 general release:

  • RightResults(tm): III’s new relevance ranking algorthem.  I’d talked about this earlier — but this is very cool.  Actually getting your results back in some kindof useable order — what a concept.
  • Spelling Suggestions:  This was very well done.  They took the basic spell checking format and then enhanced it allowing for multiple suggestions and combinations of suggestions.
  • Web Edit:  This is a web-based cataloging client.  Its basically a poor man’s Connexion — but for what it does within the catalog and the potential areas of use — this is a pretty nifty new product. 
  • RSS (input and output):  For free, III will include an RSS reader in the new WebPac.  For a fee, you can purchase a new product called Feed Builder to create newbook lists, etc.  I’m not too thrilled that this is a fee-based product, but the fact that they have finally got this working is great.
  • User Reviews:  Users can now put reviews on books (in addition to ratings (from 2005))
  • Research Pro updates:  Metafind has been renamed and improved.  They still are licensing the software/connectors from Muse Global — but they have done some development that has made their software better and allowed users a bit more flexibility in how the interface is developed.
  • WebPAC Pro — this is the new webpac platform

III has released a number of other new features and enhancements — but these have been the one’s that caught my eye (and that I’m remember off the top of my head).  I have been pretty impressed by some of the ILS improvements that they have pushed out this time around. 


 Posted by at 3:50 pm

III WebPAC Pro 2006 and more

 Innovative Interfaces  Comments Off on III WebPAC Pro 2006 and more
May 212006

I just finished sitting through a session on the current upgrade to the WebOpac.  They are actually looking to do some pretty cool stuff in the realms of adding community tagging, rss feeds (though it is additional for fee product if you want to publish data from your catalog) and faceted searches.  Its really be developed in two phases.  The first phase is this year (and is already available to beta sites), and includes the rss and community reviewing componets — but next year we will see the release of a new development platform that will sit ontop of the opac and more tightly integrate III services.  For III libraries that are completely III libraries (i.e., use Millennium, Webbridge, Metafind (now called Research Pro), Symposium (III’s IR software)) — this platform sounds fairly interesting in that III is looking to provide a single search box which integrates all the III results.  However, it looks like libraries that have gone outside of III for services will be penalized to some degree as only parts of the new platform would be available to those libraries.  For example, in the meeting today, someone asked how the system would work for users that do not have Webbridge or Research Pro.  The answer — the platforms functionality would be drastically reduced — which seems to imply that if you want your services to work together, you should be planning on purchasing III’s products.  The only potentially good thing about this is that given that this new platform will sit ontop of the current opac, there is a chance that III might start releasing some API information detailing how their service is interacting with their system.  If they do make such documentation available — I’ll be one happy camper.  Otherwise, it might make our current metasearch/openURL development a harder sell — depending on how much outside interoperability III still allows vs. the ease of being an all III shop. 

Anyway, since I’m unsure how much information III is making available about this development product at this time (outside of this very general information which has been available since MidWinter) — I’ll leave it at that and say that anyone interested should visit their booth come ALA where they hope to be demo’g parts of this new product platform.


 Posted by at 3:28 pm

IUG Reception

 Innovative Interfaces  Comments Off on IUG Reception
May 212006

I enjoy the IUG reception — its a great time to get to chat with some of the folks from III and chat with friends.  Last night’s reception was really good — partly because I got to have a good conversation with Ted Fons — one of the many III product managers.  I like Ted, partly because you can tell that Ted does really understand the products that he manages — and it shows to some degree.  My favorite product that he manages has to be their ERM product — partly because of the shear amount of time it seems to be saving — but that’s neither here nor there.  I chatted with Ted a lot of about interoperability — or III’s lack their of.  The main thing that we talked about was just simply doing some little things to make III a little bit easier to work with — maybe an SRU server or folding the xml server into the ILS by default.  Little things that could help III libraries look to do a little outside development — maybe place widgets into their opac or develop handy new tools that other III libraries would find interesting.  Its really win-win for III and I hope they eventually realize it.  Its always a good model when you have your user community participating in the development/extension process and I know that there are a lot of folks in the III community that would relish the opportunity to work outside the III black box.

I don’t necessarily blame III either for their model — partly because its a model that the library community has fostered and developed for years.  To some degree, librarians have asked our vendors to do everything and make it as transparent as possible.  This transparency has come at the cost of interroperability.   I think at this point, I’d like to see libraries pushing vendors to move to make the ILS and its associated parts simply commodities.  This way you could pick and choose the best modules from all the vendors and they would work because they all provide a set of standard api.  Not likely to happen, I know — so maybe we start smaller by getting our own vendors to at least be more open with their own customers.  Unfortunately, this is a change that I fear will happen very slowly.

Anyway — it was a good chat — and I hope Ted finds a little time to enjoy the conference between presentations.



 Posted by at 12:22 pm

Saturday, May 20, 2006: IUG Keynote

 Uncategorized  Comments Off on Saturday, May 20, 2006: IUG Keynote
May 202006

The IUG conference started out this morning with an initial presentation by III CEO, Jerry Kline. For as much as some of III’s pratices frustrate me, I actually like Jerry quite a bit. He’s very approachable and I generally enjoy chatting with him — even when I disagree. Anyway, Jerry’s conference touched on a number of numbers and then new products that he was looking to highlight. First the numbers — Jerry discussed how III has faired in the ILS market to other vendors. I’ll actually admit to being pretty surprised here — but according to his numbers, III has done very well in the ILS market picking up a number of new customers and gaining over 70 new systems from other vendors. And while the numbers were impressive, I found it a little telling that all the numbers related to the ILS product. Personally, I think III has a great ILS. There are some things that I wish it did better (i.e., the xml server should be a part of the general product in my opinion) and I’m still disappointed to some degree by their millennium java clients since it has cost our shop some productivity due to the clunky nature of some processes — but overall, I’m fairly happy with it. Its their ancillary products that leave much to be desired. Webbridge, Metafind, etc…these products all fall short of various competitor products.

Anyway, Jerry underlined a couple of new features coming to the ILS and to Metafind. First the ILS. The ILS will have a new interface called WebPac Pro. Jerry underlined a number of pretty cool enhancements that they are making to the ILS in the form of WebPac Pro. Here are the highlighted enhancements:

  • RightResults ™ — One of the things that III (and other ILS vendors) have done a terrible job at is providing relevance ranking within their ILS systems. Most vendors tend to report results in date sorts or provide pretty lousy relevancy — well — III is looking to change this. They are implementing a new sorting method call RightResults(tm) that will provide a more Google-like relevancy to their search tool. There are currently a few sites currently using this new sorting method so they work great for showing some examples of how this will work. For example, a good test case would be the Yale law school catalog and the Harvard hollis catalog. If you do a search for Harvard Law Review (as a keyword search) — the Yale law school, using III’s new ranking, shows the title first. Hollis however, I don’t see it in the first 250 results. hollis.pngyale.PNG
  • Spell Checking:  III will be implementing a “did you mean” feature though with a little twist.  Definitely will be appreciated when this has been released.
  • Adding Reviews to the OPAC — the WebPro Opac will allow users to include reviews on books.  This looked very much like Amazon’s implement.  Very cool.
  • Research Pro:  III agreed with every — Metafind stinks.  They are making big changes to MetaFind — though its still quite a way behind ExLibris’s product in my opinion

The best part of all the above noted enhancements — Jerry has said that they will be free.  Now that’s something that is very unIII and welcome in my opinion.
Anyway, more later.


 Posted by at 12:23 pm

IUG 2006, Denver, CO

 Innovative Interfaces  Comments Off on IUG 2006, Denver, CO
May 202006

Its about that time again for the IUG’s annual usergroup meeting.  This year, the annual meeting is at the Hyatt Regency in Denver, Co and will last for a few days.  I got in yesterday and the weather was definitely gorgeous.  It was 87ish yesterday and almost 90 today.  A little hot for my tastes this early in the year — but definitely better that freezing rain or something. 

Anyway — I’ll post my throughts throughout the conference and will post a conference wrapup sometime after I come home.



 Posted by at 11:50 am

Turning comments off for now

 Uncategorized  Comments Off on Turning comments off for now
May 192006

If someone knows of a good wordpress plugin for automatic spam management — I’d appreciate someone dropping me a note at: terry dot reeset at oregonstate dot edu.  I’m getting close to 70 spam comments a day that I’m having to wade through to moderate and I’m getting tired up it.  So until i get something to help reduce that flow, I’m just turning commenting off for a little while.


Thanks folks.  I got lots of recommendations for akismet so I’ve downloaded it and will give it a go.



 Posted by at 6:12 am