Readings - Relevance Feedback and Query Expansion

I found the readings this week to be informative, as I had never considered the details of how user-feedback is or could be incorporated in to IR. I specifically found the discussion of pseudo and implicit relevance feedback to be interesting. I wonder about the tradeoffs between query efficiency and retrieval success in pseudo relevance feedback given that one must, presumably, run two queries to get one result. Is  this efficiency not really an issue?

I also found the discussion of thesaurus-based query expansion to be interesting. I have seen some of this in my work with bibliographic databases, but might look into it a little more now that I understand how it works.

Muddiest Point - Unit 6

I am still interested in hearing why, when comparing results of IR systems using MAP and other averaging statistics, the IR community does not also look at variability about the mean. Simple statistics can tell us a great deal about whether things are truly 'different' in systems like this or how significant those differences are. Is this truly not an issue that is discussed in this field?

Readings - Evaluation

The readings on evaluation this week were interesting, if a little too, i think, focused on objective evaluation. In IIR the discussion focused on standard test collections, precision and recall statistics, and a number of other statistics such as the precision-recall graph and mean average precision. This was followed by discussion of relevance and the problems associated with human-based evaluation. A few things in this reading came to mind that I wouldn't mind seeing discussed in class. First, given that most of the test collections discussed in the reading are from news wire or other news sources, is there any concern, and has there been any study, about potential bias of evaluation statistics based on the content of these documents? Second, the reading seemed to downplay pretty heavily the utility of human-based evaluation. Isn't there a place for this subjective evaluation given that this is how these systems are evaluated in the end-game (by human users)?

Muddies Points - Unit 5

No muddiest points this week. The lectures have been doing a great job of clarifying the readings.