AMPLab的大数据研究.pdf

Big Data Research in the AMPLab: BDAS and Beyond Michael Franklin UC Berkeley st 1 Spark Summit December 2, 2013 UC BERKELEY AMPLab: Collaborative Big Data Research Launched: January 2011, 6 year planned duration Personnel: ~60 Students, Postdocs, Faculty and Staff UC BERKELEY Expertise: Systems, Networking, Databases and Machine Learning In-House Apps: Crowdsourcing, Mobile Sensing, Cancer Genomics AMPLab: Integrating Diverse Resources •  Machine Learning, Statistical Methods Algorithms •  Prediction, Business Intelligence •  Clusters and Clouds Machines •  Warehouse Scale Computing •  Crowdsourcing, Human Computation People •  Data Scientists, Analysts Big Data Landscape – Our Corner 4 Berkeley Data Analytics Stack Shark BlinkDB GraphX MLBase AMP (SQL) Alpha or Soon Spark Streaming ML-lib AMP Released Apache Spark BSD/Apache rd Tachyon 3 Party Open Source HDFS / Hadoop Storage Apache Mesos YARN Resource Manager Our View of the Big Data Challenge Something’s gotta give… Time Massive