jug 的屏障机制

很多时候我们对于任务依赖的数据进行控制，比如任务依赖上一个计算，但是实际计算比较慢

如下代码,实际执行就会有问题

from jug import Task
inputs = load_data()

def keep(datum):
    # A long running computation which decides whether datum should be kept
    ...

keeps = [Task(keep, i) for i in inputs]

# Now I want to throw out data
# This will NOT work:
inputs = [i for i,k in zip(inputs,keeps) if k]
results = [Task(long_computation, i) for i in inputs]

解决方法

from jug import Task, barrier, value
inputs = load_data()

def keep(datum):
    # A long running computation which decides whether datum should be kept
    ...

keeps = [Task(keep, i) for i in inputs]

barrier() # <-------- this will divide the jugfile in two!
inputs = [i for i,k in zip(inputs, value(keeps)) if k]
results = [Task(long_computation, i) for i in inputs]

通过bvalue 的解决方法

from jug import Task, bvalue
inputs = load_data()

def keep(datum):
    # A long running computation which decides whether datum should be kept
    ...

keeps = [Task(keep, i) for i in inputs]

inputs = [i for i,k in zip(inputs, bvalue(keeps)) if k]
results = [Task(long_computation, i) for i in inputs]

参考资料

https://jug.readthedocs.io/en/latest/barrier.html