Inter-Block GPU Communication via Fast Barrier Synchronization.pdfVIP

Inter-Block GPU Communication via Fast Barrier Synchronization Shucai Xiao and Wu-chun Feng Department of Computer Science Virginia Tech shucai, wfeng@ Abstract Categories and Subject Descriptors D.1.3[Software] [Program- The graphics processing unit (GPU) has evolved from a fixed- ming Techniques]: Concurrent Programming function processor with programmable stages to a programmable General Terms Design, Performance, Model processor with many fixed-function components that deliver mas- sive parallelism. Consequently, GPUs increasingly take advantage Keywords Parallel computing, CUDA, kernel execution-time of the programmable processing power for general-purpose, non- model, GPU synchronization graphics tasks, i.e., general-purpose computation on graphics pro- cessing units (GPGPU). However, while the GPU can massively 1. Introduction accelerate data parallel (or task parallel) applications, the lack of explicit support for inter-block communication on the GPU ham- Today, improving the computational capability of a processor pers its broader adoption as a general-purpose computing device. comes from increasing its number of processing cores rather than Inter-block communication on the GPU occurs via global mem- increasing its clock speed. This is reflected in both traditional multi- ory and then requires a barrier synchronization acros