chapter 4~.ppt

Modern Information RetrievalChapter 4Retrieval Evaluation  性能评价（performance evaluation） 检索性能评价（retrieval performance evaluation） 查全率(R, Recall Ratio) 查准率(P, Precision Ratio) 漏检率(O, Omission Ratio) 误检率(M, Miss Ratio) 交互会话（interactive session） 用户负担（user effort） 信息性（Informativeness） 面向用户（user-oriented） 覆盖率（coverage ratio） 新颖率（novelty ratio） 相对查全率（relative recall） 查全率负担（recall effort） 参考测试集（reference test collection） 主观性（subjectiveness） 优劣程度（goodness） 信息性测度（informativeness measure） Chapter 4 Retrieval Evaluation（检索评价） 教学目的与要求： 1. Make students comprehend trends and research issues 2. Make students familiar with reference collection 3. Make students master retrieval performance evaluation 教学内容： 1. Retrieval Performance Evaluation 2. Reference Collection 3. Trends and Research Issues 重点与难点： Recall, Precision, Omission, Miss 4.1 Introduction 4.13retrieval metrics 4.3.1 Precision and Recall 档案班 下转60页 There is an inherent trade off 折中between space complexity and time complexity which frequently allows trading one for the other. In a system designed for providing data retrieval, the response time and the space required 响应时间和所需空间are usually the metrics of most interest and the ones normally adopted for evaluating the system. In this case, we look for the performance of the indexing structures (which are in place to accelerate the search), the interaction with the operating system, the delays in communication channels, and the overheads introduced by the many software layers which are usually present. We refer to such a form of evaluation simply as performance evaluation. 我们不但要考虑索引结构的性能（可以适当地加快检索速度）、与检索系统的交互性、通信信道的延迟、还要考虑常见的由软件层所带来的开销等因素，我们将这种评价简称为性能评价 Thus, information retrieval systems require the evaluation of how precise is the answer set. This is referred to as retrieval performance evaluation. Such an evaluation is usually based on a test reference collection测试参考集 and on an evaluation measure评价测度. test reference collection参考测试集