Collaborative Filtering January 16, 2007Posted by Andre Vellino in Collaborative filtering, Digital library, Recommender service.
One area of research that I think will be fruitful and also beneficial to the end-user of a science library portal is a collaborative filtering (CF) system. The general idea is to take information about the user’s past usage statistics to help cluster / rank search results, offer serendipitous recommendations and automatically update library “alerts”.
Things like that have been tried before, mostly for commercial purposes. A9 (owned by Amazon) and the Google portal do it, why not a digital library? One challenge in applying this kind of technique fruitfully in a sparsely frequented scientific e-library is that there may not be enough data to draw meaningful conclusions without additional information. It will probably require a combination of other techniques as well, such as text-analysis on retrieved items, query analysis and explicit personal profiles.
I like some things about the suggester at The Library Thing, (though not the unsuggester, except, perhaps, for entertainment value.) What I like best about the portal version of the “suggester” is the explanation feature – why the recommender system made the recommendation. This could be especially useful for eliminating unwanted collections of suggestions in those instances where the sample space is sparse and heterogeneous. Which leads me to wish for a refinement on explanation feature – recommendations could be returned in clusters, perhaps even a hierarchical cluster, based on the reason(s) the recommendation was made.
People used to be worried about the privacy issues with data from search analytics, but I think it’s clear that if the value to the user reaches a certain level, privacy no longer matters (much.) Furthermore, there may even be some general social acceptance and understanding about what an automated recommendation service can do for you. Chris Anderson, in The Long Tail, credits Amazon’s CF system for the rediscovery of forgotten book treasures such as Touching the Void. People now seem to want software to help them, providing it doesn’t look like a paper-clip and beep at you :-).