Lightweight Distributed Execution strongEnginestrong for Large- Scale.pdfVIP

Lightweight Distributed Execution Engine for Large- Scale Spatial Join Query Processing Jianting Zhang Simin You Le Gruenwald Dept. of Computer Science Dept. of Computer Science Dept. of Computer Science The City College of New York CUNY Graduate Center The University of Oklahoma New York, NY, USA New York, NY, USA Norman, OK, USA jzhang@cs.ccny.cuny .edu syou@ ggruenwald@ Abstract— Existing Big Data systems are mostly designed for running on single computing nodes and, thus, require new relational data. They are either incapable or inefficient in systems to reduce processing times, from days or hours to processing large-scale semi-structured data efficiently due to the minutes or even seconds, in order to be practically useful for inherent limitations on data abstraction, indexing support and researchers and decision makers [3]. exposure to native parallel programming tools. In this study, we Our previous efforts in extending the leading in- report our work in developing a lightweight distributed execution engine for spatial join query processing on large-scale geospatial memory Big Data system Impala [4] to support spatial data data. By integrating data parallel designs for single computing processing with a focus on spatial joins on multi-core CPU nodes,