Independent proprietorship and competition in distributed Web search architectures.pdf

In Proceedings of the Seventh IEEE International Conference on Engineering of Complex Computer Systems (ICECCS 2001) Independent Proprietorship and Competition in Distributed Web Search Architectures ? Rinat Khoussainov, Tadhg O’Meara, and Ahmed Patel Computer Networks and Distributed Systems Research Group Department of Computer Science, University College Dublin Belfield, Dublin 4, Ireland {rinat,tadhg,apatel}@cnds.ucd.ie Abstract The predominant Web search model attempts to use mul- tiple computers under centralised management to act as one search engine for the entire Web. As the quantity of on- line information increases, systems based on this model be- come prohibitively expensive for all but the largest organ- isations. We advocate the use of distributed search archi- tectures where multiple independently owned and managed search engines act as one search system. This approach has significant advantages including low market entry cost for individual search providers and the potential to stimulate the provision of high-quality services through competition. The low entry cost allows small organisations and even in- dividual users to influence service features and quality by establishing specialised search services. However, indepen- dent proprietorship also greatly complicates the search sys- tem design. The potential for competition between engines requires new approaches to effective engine management. Many new issues arise such as deciding what information an engine will index. In this paper, we analyse the sources of complexity in distributed Web search architectures with in- dependent proprietorship and competition between engines. We outline possible ways to cope with this complexity using techniques from the field of computational economics such as game theory. 1. Introduction Effective search for information resources on the Internet has become a popular and highly motivated research topic in the recent years. The huge size and continuing growth of the