12.8 简单的并行编程.pdfVIP

12.8 简单的并⾏编程 问题 You have a program that performs a lot of CPU-intensive work , and you want to makeit run faster by having it take advantage of multiple CPUs. 解决⽅案 The concurrent .futures library provides a ProcessPool xecutor class that can beused to execute computationally intensive functions in a separately running instance ofthe Python interpreter . However , in order to use it , you first need to have some com-putationally intensive work . Let ’s illustrate with a simple yet practical example .Suppose you have an entire directory of gzip-compressed Apache web server logs: logs/20 12070 1.log .gz20 120702.log .gz20 120703.log .gz20 120704 .log .gz20 120705.log .gz20 120706. Further suppose each log file contains lines like this: 124 .115.6.12 - - [10/Jul/20 12:00:18:50 -0500] “G T /robots.txt ...” 200 7 12 10 .2 12.209.67 - - [10/Jul/20 12:00:18:5 1 -0500] “G T /ply/ ...” 200 118752 10 .2 12.209.67 - - [10/Jul/20 12:00:18:5 1 -0500] “G T /favicon .ico ...” 404 36961.135.2 16.105 - - [10/Jul/20 12:00:20:04 -0500] “G T /blog/atom .xml ...” 304 -... Here is a simple script that takes this data and identifies all hosts that have accessed therobots.txt file: findrobots.py import gzipimport ioimport glob def find_robots(filename): ‘Find all of the hosts that access robots.txt in a single log file‘robots = set()with gzip .open(filename) as f: for line in io .TextIOWrapper(f ,encoding=ascii): fields = line .split()if fields[6] == ‘/robots.txt: robots.add(fields[0]) return robots def find_all_robots(logdir): ‘Find all hosts across and entire sequence of files‘files = glob .glob(logdir+/*.log .gz)all_robots = set()for robots in map(find_robots, files): all_robots.update(robots) return all_robots if name == ‘main: robots = find_all_robots(‘logs)for ipaddr in robots: print(ipaddr) The preceding program is written in the