python并发与web

python并发与web

python并发主要方式有：

• Thread(线程)
• Process(进程)
• 协程
  python因为GIL的存在使得python的并发无法利用CPU多核的优势以至于性能比较差，下面我们将通过几个例子来介绍python的并发。

线程

我们通过一个简单web server程序来观察python的线程，首先写一个耗时的小函数

def fib(n):
    if n <= 2:
        return 1
    else:
        return fib(n - 1) + fib(n - 2)

然后写一个fib web server，程序比较简单就不解释了。

from socket import *
from fib import fib

def fib_server(address):
    sock = socket(AF_INET, SOCK_STREAM)
    sock.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1)
    sock.bind(address)
    sock.listen(5)
    while True:
        client, addr = sock.accept()
        print('Connection', addr)
        fib_handle(client)
    
def fib_handler(client):
    while True:
        req = client.recv(100)
        if not req:
            break
        n = int(req)
        result = fib(n)
        resp = str(result).encode('ascii') + b'\n'
        client.send(resp)
    print('Closed')

fib_server(('', 25002))

运行shell命令可以看到计算结果

nc localhost 25002

10

55

由于服务段是单线程的，如果另外启动一个连接将得不到计算结果

nc localhost 25002

10

为了能让我们的server支持多个请求，我们对服务端代码加入多线程支持

#sever.py
#服务端代码
from socket import *
from fib import fib
from threading import Thread

def fib_server(address):
    sock = socket(AF_INET, SOCK_STREAM)
    sock.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1)
    sock.bind(address)
    sock.listen(5)
    while True:
        client, addr = sock.accept()
        print('Connection', addr)
        #fib_handler(client)
        Thread(target=fib_handler, args=(client,), daemon=True).start() #需要在python3下运行

def fib_handler(client):
    while True:
        req = client.recv(100)
        if not req:
            break
        n = int(req)
        result = fib(n)
        resp = str(result).encode('ascii') + b'\n'
        client.send(resp)
    print('Closed')
    
fib_server(('', 25002)) #在25002端口启动程序

运行shell命令可以看到计算结果

nc localhost 25002

10

55

由于服务端是多线程的，启动一个新连接将得到计算结果

nc localhost 25002

10

55

性能测试

我们加入一段性能测试代码

#perf1.py
from socket import *
from threading import Thread
import time

sock = socket(AF_INET, SOCK_STREAM)
sock.connect(('localhost', 25002))

n = 0

def monitor():
    global n
    while True:
        time.sleep(1)
        print(n, 'reqs/sec')
        n = 0
Thread(target=monitor).start()


while True:
    start = time.time()
    sock.send(b'1')
    resp = sock.recv(100)
    end = time.time()
    n += 1

#代码非常简单，通过全局变量n来统计qps(req/sec 每秒请求数)

在shell中运行perf1.py可以看到结果如下:

• 106025 reqs/sec
• 109382 reqs/sec
• 98211 reqs/sec
• 105391 reqs/sec
• 108875 reqs/sec

平均每秒请求数大概是10w左右

如果我们另外启动一个进程来进行性能测试就会发现python的GIL对线程造成的影响

python3 perf1.py

• 74677 reqs/sec
• 78284 reqs/sec
• 72029 reqs/sec
• 81719 reqs/sec
• 82392 reqs/sec
• 84261 reqs/sec

并且原来的shell中的qps也是类似结果

• 96488 reqs/sec
• 99380 reqs/sec
• 84918 reqs/sec
• 87485 reqs/sec
• 85118 reqs/sec
• 78211 reqs/sec

如果我们再运行

nc localhost 25002

40

来完全占用服务器资源一段时间，就可以看到shell窗口内的rqs迅速下降到

• 99 reqs/sec
• 99 reqs/sec

这也反映了Python的GIL的一个特点，会优先处理占用CPU资源大的任务

具体原因我也不知道，可能需要阅读GIL实现源码才能知道。

线程池在web编程的应用

python有个库叫做cherrypy，最近用到，大致浏览了一下其源代码，其内核使用的是python线程池技术。

cherrypy通过Python线程安全的队列来维护线程池，具体实现为:

class ThreadPool(object):

    """A Request Queue for an HTTPServer which pools threads.

    ThreadPool objects must provide min, get(), put(obj), start()
    and stop(timeout) attributes.
    """

    def __init__(self, server, min=10, max=-1,
        accepted_queue_size=-1, accepted_queue_timeout=10):
        self.server = server
        self.min = min
        self.max = max
        self._threads = []
        self._queue = queue.Queue(maxsize=accepted_queue_size)
        self._queue_put_timeout = accepted_queue_timeout
        self.get = self._queue.get

    def start(self):
        """Start the pool of threads."""
        for i in range(self.min):
            self._threads.append(WorkerThread(self.server))
        for worker in self._threads:
            worker.setName('CP Server ' + worker.getName())
            worker.start()
        for worker in self._threads:
            while not worker.ready:
                time.sleep(.1)
         ....
  
    def put(self, obj):
        self._queue.put(obj, block=True, timeout=self._queue_put_timeout)
        if obj is _SHUTDOWNREQUEST:
            return

    def grow(self, amount):
        """Spawn new worker threads (not above self.max)."""
        if self.max > 0:
            budget = max(self.max - len(self._threads), 0)
        else:
            # self.max <= 0 indicates no maximum
            budget = float('inf')

        n_new = min(amount, budget)

        workers = [self._spawn_worker() for i in range(n_new)]
        while not all(worker.ready for worker in workers):
            time.sleep(.1)
        self._threads.extend(workers)

        ....
        
    def shrink(self, amount):
        """Kill off worker threads (not below self.min)."""
        [...]

    def stop(self, timeout=5):
        # Must shut down threads here so the code that calls
        # this method can know when all threads are stopped.
        [...]
        

可以看出来，cherrypy的线程池将大小初始化为10，每当有一个httpconnect进来时就将其放入任务队列中，然后WorkerThread会不断从任务队列中取出任务执行，可以看到这是一个非常标准的线程池模型。

进程

由于Python的thread无法利用多核，为了充分利用多核CPU，Python可以使用了多进程来模拟线程以提高并发的性能。Python的进程代价比较高可以看做是另外再启动一个python进程。

#server_pool.py

from socket import *
from fib import fib
from threading import Thread
from concurrent.futures import ProcessPoolExecutor as Pool #这里用的python3的线程池，对应python2的threadpool

pool = Pool(4) #启动一个大小为4的进程池

def fib_server(address):
    sock = socket(AF_INET, SOCK_STREAM)
    sock.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1)
    sock.bind(address)
    sock.listen(5)
    while True:
        client, addr = sock.accept()
        print('Connection', addr)
        Thread(target=fib_handler, args=(client,), daemon=True).start()
    
def fib_handler(client):
    while True:
        req = client.recv(100)
        if not req:
            break
        n = int(req)
        future = pool.submit(fib, n)
        result = future.result()
        resp = str(result).encode('ascii') + b'\n'
        client.send(resp)
    print('Closed')

fib_server(('', 25002))

性能测试

可以看到新的server的qps为：

• 4613 reqs/sec
• 4764 reqs/sec
• 4619 reqs/sec
• 4393 reqs/sec
• 4768 reqs/sec
• 4846 reqs/sec

这个结果远低于前面的10w qps主要原因是进程启动速度较慢，进程池内部逻辑比较复杂，涉及到了数据传输，队列等问题。

但是通过多进程我们可以保证每一个链接相对独立，不会受其他请求太大的影响。

即使我们使用以下耗时的命令也不会影响到性能测试

nc localhost 25502

40

协程

协程简介

协程是一个古老的概念，最早出现在早期的os中，它出现的时间甚至比线程进程还要早。

协程也是一个比较难以理解和运用的并发方式，用协程写出来的代码比较难以理解。

python中使用yield和next来实现协程的控制。

def count(n):
    while(n > 0):
        yield n   #yield起到的作用是blocking，将代码阻塞在这里，生成一个generator，然后通过next调用。
        n -= 1
for i in count(5):
    print(i)
#可以看到运行结果：
5
4
3
2
1

下面我们通过例子来介绍如何书写协程代码。首先回到之前的代码。首先我们要想到我们为什么要用线程，当然是为了防止阻塞，
这里的阻塞来自socket的IO和cpu占用2个方面。协程的引入也是为了防止阻塞，因此我们先将代码中的阻塞点标记出来。

#sever.py
#服务端代码
from socket import *
from fib import fib

def fib_server(address):
    sock = socket(AF_INET, SOCK_STREAM)
    sock.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1)
    sock.bind(address)
    sock.listen(5)
    while True:
        client, addr = sock.accept()  #blocking
        print('Connection', addr)
        fib_handler(client)

def fib_handler(client):
    while True:
        req = client.recv(100)    #blocking
        if not req:
            break
        n = int(req)
        result = fib(n)
        resp = str(result).encode('ascii') + b'\n'
        client.send(resp)    #blocking
    print('Closed')
    
fib_server(('', 25002)) #在25002端口启动程序

上面标记了3个socket IO阻塞点，我们先忽略CPU占用。

• 首先我们在blocking点插入yield语句，这样做的原因就是，通过yield标记出blocking点以及blocking的原因，这样我们就可以在调度的时候实现noblocking，我们调度的时候遇到yield语句并且block之后就可以直接去执行其他的请求而不用阻塞在这里，这里我们也将实现一个简单的noblocking调度方法。

#sever.py
#服务端代码
from socket import *
from fib import fib

def fib_server(address):
    sock = socket(AF_INET, SOCK_STREAM)
    sock.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1)
    sock.bind(address)
    sock.listen(5)
    while True:
        yield 'recv', sock
        client, addr = sock.accept()  #blocking
        print('Connection', addr)
        fib_handler(client)

def fib_handler(client):
    while True:
        yield 'recv', client
        req = client.recv(100)    #blocking
        if not req:
            break
        n = int(req)
        result = fib(n)
        resp = str(result).encode('ascii') + b'\n'
        yield 'send', client
        client.send(resp)    #blocking
    print('Closed')
    
fib_server(('', 25002)) #在25002端口启动程序

• 上述程序无法运行，因为我们还没有一个yield的调度器，程序只是单纯的阻塞在了yield所标记的地方，这也是协程的一个好处，可以人为来调度，不像thread一样乱序执行。下面是包含了调度器的代码。

from socket import *
from fib import fib
from threading import Thread
from collections import deque
from concurrent.futures import ProcessPoolExecutor as Pool
from select import select

tasks = deque()  
recv_wait = {}
send_wait = {}
def run():
    while any([tasks, recv_wait, send_wait]):
        while not tasks:
            can_recv, can_send, _ = select(recv_wait, send_wait, [])
            for s in can_recv:
                tasks.append(recv_wait.pop(s))
            for s in can_send:
                tasks.append(send_wait.pop(s))         
        task = tasks.popleft()
        try:
            why, what = next(task)
            if why == 'recv':
                recv_wait[what] = task
            elif why == 'send':
                send_wait[what] = task
            else:
                raise RuntimeError("ARG!")
        except StopIteration:
            print("task done")

def fib_server(address):
    sock = socket(AF_INET, SOCK_STREAM)
    sock.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1)
    sock.bind(address)
    sock.listen(5)
    while True:
        yield 'recv', sock
        client, addr = sock.accept()
        print('Connection', addr)
        tasks.append(fib_handler(client))
    
def fib_handler(client):
    while True:
        yield 'recv', client
        req = client.recv(100)
        if not req:
            break
        n = int(req)
        result = fib(n)
        resp = str(result).encode('ascii') + b'\n'
        yield 'send', client
        client.send(resp)
    print('Closed')

tasks.append(fib_server(('', 25003)))
run()

• 我们通过轮询+select来控制协程，核心是用一个task queue来维护程序运行的流水线，用recv_wait和send_wait两个字典来实现任务的分发。

性能测试

可以看到新的server的qps为：

• (82262, 'reqs/sec')
• (82915, 'reqs/sec')
• (82128, 'reqs/sec')
• (82867, 'reqs/sec')
• (82284, 'reqs/sec')
• (82363, 'reqs/sec')
• (82954, 'reqs/sec')

与之前的thread模型性能比较接近，协程的好处是异步的，但是协程 仍然只能使用到一个CPU

当我们让服务器计算40的fib从而占满cpu时，qps迅速下降到了0。

tornado 基于协程的 python web框架

tornado是facebook出品的异步web框架,tornado中协程的使用比较简单，利用coroutine.gen装饰器可以将自己的异步函数注册进tornado的ioloop中，tornado异步方法一般的书写方式为:

@gen.coroutime
def post(self):
    resp = yield GetUser()
    self.write(resp)

tornado异步原理

def start(self):
    """Starts the I/O loop.
    The loop will run until one of the I/O handlers calls stop(), which
    will make the loop stop after the current event iteration completes.
    """
    self._running = True
    while True:
    [ ... ]
        if not self._running:
            break
        [ ... ]
        try:
            event_pairs = self._impl.poll(poll_timeout)
        except Exception, e:
            if e.args == (4, "Interrupted system call"):
                logging.warning("Interrupted system call", exc_info=1)
                continue
            else:
                raise
        # Pop one fd at a time from the set of pending fds and run
        # its handler. Since that handler may perform actions on
        # other file descriptors, there may be reentrant calls to
        # this IOLoop that update self._events
        self._events.update(event_pairs)
        while self._events:
            fd, events = self._events.popitem()
            try:
                self._handlers[fd](fd, events)
            except KeyboardInterrupt:
                raise
            except OSError, e:
                if e[0] == errno.EPIPE:
                    # Happens when the client closes the connection
                    pass
                else:
                    logging.error("Exception in I/O handler for fd %d",
                                  fd, exc_info=True)
            except:
                logging.error("Exception in I/O handler for fd %d",fd, exc_info=True)

这是tornado异步调度的核心主循环，poll()方法返回一个形如(fd: events)的键值对，并赋值给event_pairs变量，在内部的while循环中，event_pairs中的内容被一个一个的取出，然后相应的处理器会被调用，tornado通过下面的函数讲socket注册进epoll中。tornado在linux默认选择epoll，在windows下默认选择select(只能选择select)。

def add_handler(self, fd, handler, events):
    """Registers the given handler to receive the given events for fd."""
    self._handlers[fd] = handler
    self._impl.register(fd, events | self.ERROR)

cherrypy线程池与tornado协程的比较

我们通过最简单程序运行在单机上进行性能比较

测试的语句为：

ab -c 100 -n 1000 -k localhost:8080/ | grep "Time taken for tests:"

其中cherrypy的表现为：

• Completed 100 requests
• Completed 200 requests
• Completed 300 requests
• Completed 400 requests
• Completed 500 requests
• Completed 600 requests
• Completed 700 requests
• Completed 800 requests
• Completed 900 requests
• Completed 1000 requests
• Finished 1000 requests

Time taken for tests: 10.773 seconds

tornado的表现为：

• Completed 100 requests
• Completed 200 requests
• Completed 300 requests
• Completed 400 requests
• Completed 500 requests
• Completed 600 requests
• Completed 700 requests
• Completed 800 requests
• Completed 900 requests
• Completed 1000 requests
• Finished 1000 requests

Time taken for tests: 0.377 seconds

可以看出tornado的性能还是非常惊人的，当应用程序涉及到异步IO还是要尽量使用tornado

总结

本文主要介绍了python的线程、进程和协程以及其应用，并对这几种模型进行了简单的性能分析，python由于GIL的存在，不管是线程还是协程都不能利用到多核。

• 对于计算密集型的web app线程模型与协程模型的性能大致一样，线程由于调度受操作系统管理，其性能略好。
• 对于IO密集型的web app协程模型性能会有很大的优势。

参考文献

略