Twitter从支撑千万到万亿级索引的有哪些信誉好的足球投注网站引擎架构演化.pptx

The Road to a Complete Tweet Index Outline 1. Current Scale of Twitter Search 2. The History of Twitter Search Infra 3. Complete Tweet Index 4. Search Engine Applications 5. Outlook The Road to a Complete Tweet Index @yz Outline 1. Current Scale of Twitter Search 2. The History of Twitter Search Infra 3. Complete Tweet Index 4. Search Engine Applications 5. Outlook The Road to a Complete Tweet Index @yz More than 2 billion search queries per day. @yz The Road to a Complete Tweet Index Current Scale of Twitter Search Hundreds of million Tweets are indexed per day. @yz The Road to a Complete Tweet Index Current Scale of Twitter Search @yz The Road to a Complete Tweet Index Current Scale of Twitter Search Hundreds of billions of Tweets have been sent since company founding in 2006. @yz The Road to a Complete Tweet Index Current Scale of Twitter Search Our Complete Tweet Index is served by thousands of instances, each with 256GB RAM and 2TB SSD. @yz The Road to a Complete Tweet Index Current Scale of Twitter Search But … our search infrastructure is currently supported by only a small number of engineers and SREs. We are hiring! Outline 1. Current Scale of Twitter Search 2. The History of Twitter Search Infra 3. Complete Tweet Index 4. Search Engine Applications 5. Outlook The Road to a Complete Tweet Index @yz @yz The Road to a Complete Tweet Index 2010 Realtime Search Powered by replicated MySQL instances and MySQL text matching. @yz The Road to a Complete Tweet Index 2010 Realtime Search Powered by MySQL.      Hundreds of Tweets per second. A few thousand of queries per second. Basic text search: no fancy tokenization, no search assistance, slow geo search etc. Many incidents and down times. MySQL master/slave dying was particularly problematic. @yz The Road to a Complete Tweet Index 2011 Launched Lucene-based search engine: Earlybird*.    Lucene API, but custom data structures optimized for in-memory operations and Realtime search. Novel concurrent a