The information science research community is divided between the system-centered cluster working on information retrieval (IR) algorithms and the user-centered cluster working on user behavior. A criticism from the user-centered cluster on the system-centered cluster is that document relevance should be regarded as a subjective, dynamic, and multi-dimensional construct, rather than a static and objective one.

Building on past user studies, this study proposes two hypotheses: firstly, the uni dimensional treatment of relevance in current relevance feedback IR systems will perform as well as a feedback system based solely on topicality judgment. Secondly, an IR system, which takes into account users’ novelty perception, will perform better when the novelty judgment is allowed to age – some documents are regarded as redundant if they have been evaluated long ago – than when redundancy is not incorporated into novelty judgment.

Four systems were designed to test these hypotheses and results were collected from the simulation and user study tests. The results of the simulation support the two hypotheses whereas the results of the user study support only the first hypothesis and not the second, which may be due to the experimental design.