Python(协程)

day32

协程，又称微线程，纤程。英文名Coroutine。一句话说明什么是线程：协程是一种用户态的轻量级线程。

协程拥有自己的寄存器上下文和栈。协程调度切换时，将寄存器上下文和栈保存到其他地方，在切回来的时候，恢复先前保存的寄存器上下文和栈。因此：

协程能保留上一次调用时的状态（即所有局部状态的一个特定组合），每次过程重入时，就相当于进入上一次调用的状态，换种说法：进入上一次离开时所处逻辑流的位置。

 

复习yield

def f():
    print('ok1')
    count = yield 5
    print('count:', count)
    print('ok2')
    yield 67

gen = f()#生成器对象

ret1 = next(gen)#返回5    等同ret1 = gen.send(None)
print(ret1)
#从count = yield处开始执行
ret2 = gen.send(10)#返回67
print(ret2)

第十行执行到yield = 5，返回一个5。

下一次从count = yeild开始执行。

ok1
5
count: 10
ok2
67

Process finished with exit code 0

 

yield协程并发

 

def consumer(name):
    print("--->starting eating baozi...")
    while True:
        new_baozi = yield
        print("[%s] is eating baozi %s" % (name, new_baozi))
        # time.sleep(1)

def producer():
    next(con)
    next(con2)
    n = 0
    while n < 5:
        n += 1
        con.send(n)
        con2.send(n)
        print("\033[32;1m[producer]\033[0m is making baozi %s" % n)

if __name__ == '__main__':
    con = consumer("c1")#创建一个生成器对象， 其中有yield
    con2 = consumer("c2")
    p = producer()

 

可以在19行处设断点调试。

从con，con2中选择con分析程序的执行过程。

next（con）进入到consumer函数，执行到new_baozi = yield阻断,new_baozi = yeild并未执行，且无返回值。

到while循环的con.send(n)，使new_baozi = n，并且print，即第9行。

由于是while循环，循环到new_baozi = yield处继续阻断，等下一次for循环的con.send(n)。

producer中send完，consumer便会立即输出，与协程相类似。

 

我们先给协程一个标准定义，即符合什么条件就能称之为协程：

1. 必须在只有一个单线程里实现并发
2. 修改共享数据不需加锁
3. 用户程序里自己保存多个控制流的上下文栈
4. 一个协程遇到IO操作自动切换到其它协程

基于上面这4点定义，我们刚才用yield实现的程并不能算是合格的线程，因为它有一点功能没实现，哪一点呢？

 

执行结果：

--->starting eating baozi...
--->starting eating baozi...
[c1] is eating baozi 1
[c2] is eating baozi 1
[producer] is making baozi 1
[c1] is eating baozi 2
[c2] is eating baozi 2
[producer] is making baozi 2
[c1] is eating baozi 3
[c2] is eating baozi 3
[producer] is making baozi 3
[c1] is eating baozi 4
[c2] is eating baozi 4
[producer] is making baozi 4
[c1] is eating baozi 5
[c2] is eating baozi 5
[producer] is making baozi 5

Process finished with exit code 0

 

greenlet

完成不同任务间的切换。

# -*- coding:utf-8 -*-
from greenlet import greenlet

def test1():
    print(12)
    gr2.switch()
    print(34)
    gr2.switch()

def test2():
    print(56)
    gr1.switch()
    print(78)

gr1 = greenlet(test1)
gr2 = greenlet(test2)
gr1.switch()#先执行test1函数，遇到gr2.switch()，到test2中，从之前停止处开始执行
#完成不同任务间的切换

执行结果：

12
56
34
78

Process finished with exit code 0

 

gevent

import gevent
import time

def func1():
    print('\033[31;1m a\033[0m', time.ctime())
    gevent.sleep(2)#模拟IO阻塞的情况，阻塞时执行func2中
    print('\033[31;1m b\033[0m', time.ctime())


def func2():
    print('\033[32;1m c\033[0m', time.ctime())
    gevent.sleep(1)#不是time.sleep()
    print('\033[32;1m d\033[0m', time.ctime())


gevent.joinall([
    gevent.spawn(func1),
    gevent.spawn(func2),
    # gevent.spawn(func3),
])

先执行func1，到gevent.sleep(2)时，模拟了IO阻塞，马上执行func2，当func2中的阻塞1秒后，func1还处于阻塞状态，func2继续执行。

执行结果：

 a Tue Nov  6 16:37:04 2018
 c Tue Nov  6 16:37:04 2018
 d Tue Nov  6 16:37:05 2018
 b Tue Nov  6 16:37:06 2018

Process finished with exit code 0

 

协程实战

 

from gevent import monkey
monkey.patch_all()#可以检测到阻塞

import gevent
from urllib.request import urlopen
import time

def f(url):
    print('GET: %s' % url)
    resp = urlopen(url)
    data = resp.read()

    with open('xiaohua.html', 'wb') as f:
        f.write(data)

    print('%d bytes received from %s.' % (len(data), url))

                                   # f('http://www.xiaohuar.com/')
a = time.time()
gevent.joinall([
    gevent.spawn(f, 'https://www.python.org/'),
    gevent.spawn(f, 'https://www.yahoo.com/'),
    gevent.spawn(f, 'https://github.com/'),
])

b = time.time()

# l = ['https://www.python.org/', 'https://www.yahoo.com/', 'https://github.com/']
# for url in l:
#     f(url)
# c = time.time()

# print('普通方法所需时间：', c - b)#10s

print('使用协程所需时间：', b - a)# 2s

 

爬虫使用协程，爬取网络更快。遇到阻塞会更换任务。

而且monkey可以检测到是否阻塞。

执行结果：

GET: https://www.python.org/
GET: https://www.yahoo.com/
GET: https://github.com/
65038 bytes received from https://github.com/.
48925 bytes received from https://www.python.org/.
517942 bytes received from https://www.yahoo.com/.
使用协程所需时间： 5.216813802719116

Process finished with exit code 0