2013.11WhenBigDataMeetsBigSmog-ABigSpatio-temporal DataFrameworkforChinaSevereSmogAnalysis.pdf

When Big Data Meets Big Smog: A Big Spatio-temporal Data Framework for China Severe Smog Analysis Jiaoyan Chen College of Computer Science, Zhejiang University, Yuquan Campus, Yugu Road, Hangzhou, China jiaoyanchen@ Huajun Chen College of Computer Science, Zhejiang University, Yuquan Campus, Yugu Road, Hangzhou, China huajunsir@ Jeff Z. Pan Department of Computing Science, The University of Aberdeen, Aberdeen, UK jeff.z.pan@abdn.ac.uk Ming Wu College of Computer Science, East China University of Science and Technology Shanghai, China ecustwm@ Ningyu Zhang College of Computer Science, Zhejiang University, Yuquan Campus, Yugu Road, Hangzhou, China zhangningyu@ Guozhou Zheng College of Computer Science, Zhejiang University, Yuquan Campus, Yugu Road, Hangzhou, China zzzgz@ ABSTRACT Recently, the appearing disaster of severe smog has been attacking many cities in China such as the capital Beijing. The chief culprit of China smog, namely PM2.5, is affected by various factors including air pollutants, weather, climate, geographical location, urbanization, etc. To analyze the fac- tors, we collect about 35,000,000 air quality records and about 30,000,000 weather records from the sensors in 77 China’s cities in 2013. Moreover, two big data sets named Geoname and DBPedia are also combined for the data of cli- mate, geographical location and urbanization. To deal with big spatio-temporal data for big smog analysis, we propose a MapReduce-based framework named BigSmog. It mainly conducts parallel correlation analysis of the factors and scal- able training of artificial neural networks for spatio-temporal approximation of the concentration of PM2.5. In the ex- periments, BigSmog displays high scalability for big smog analysis with big spatio-temporal data. The analysis result shows that the air pollutants influence the short-term con- centration of PM2.5 more than the weather and the factors of geographical location and climate rather than urbanization play a major role in deter