python之GIL官方文档 global interpreter lock 全局解释器锁


2. 术语 global interpreter lock 全局解释器锁
3. C-API 还有更多没有仔细看
4. 定期切换线程
5. wiki.python
6. python.doc FAQ


2. 术语 global interpreter lock 全局解释器锁

The mechanism used by the CPython interpreter to assure that only one thread executes Python bytecode at a time.

This simplifies the CPython implementation by making the object model (including critical built-in types such as dict)
implicitly safe against concurrent access.

Locking the entire interpreter makes it easier for the interpreter to be multi-threaded,
at the expense of much of the parallelism afforded by multi-processor machines.

However, some extension modules, either standard or third-party, are designed so as to release the GIL
when doing computationally-intensive tasks such as compression or hashing.
Also, the GIL is always released when doing I/O.

Past efforts to create a “free-threaded” interpreter (one which locks shared data at a much finer granularity)
have not been successful because performance suffered in the common single-processor case.
It is believed that overcoming this performance issue would make the implementation much more complicated
and therefore costlier to maintain.

3. C-API 还有更多没有仔细看

The Python interpreter is not fully thread-safe.
In order to support multi-threaded Python programs, there’s a global lock, called the global interpreter lock or GIL,
that must be held by the current thread before it can safely access Python objects.

Without the lock, even the simplest operations could cause problems in a multi-threaded program:
for example, when two threads simultaneously increment the reference count of the same object,
the reference count could end up being incremented only once instead of twice.

Therefore, the rule exists that only the thread that has acquired the GIL may operate on Python objects or call Python/C API functions.
In order to emulate concurrency of execution, the interpreter regularly tries to switch threads (see sys.setswitchinterval()).
The lock is also released around potentially blocking I/O operations like reading or writing a file,
so that other Python threads can run in the meantime.
因此,设定了这样一个规则,只有获取GIL的线程可以对Python对象操作或调用Python/C API函数。
为了模拟执行的并发性,解释器定期尝试切换线程(参见后续章节的sys.setswitchinterval() Python 2.x为sys.setcheckinterval())


Note Calling system I/O functions is the most common use case for releasing the GIL,
but it can also be useful before calling long-running computations which don’t need access to Python objects,
such as compression or cryptographic functions operating over memory buffers.
For example, the standard zlib and hashlib modules release the GIL when compressing or hashing data.

Note When only the main thread exists, no GIL operations are needed.
This is a common situation (most Python programs do not use threads), and the lock operations slow the interpreter down a bit.
Therefore, the lock is not created initially. This situation is equivalent to having acquired the lock:
when there is only a single thread, all object accesses are safe.

4. 定期切换线程

(1)Python 2.x为sys.setcheckinterval()

Set the interpreter’s “check interval”.
This integer value determines how often the interpreter checks for periodic things such as thread switches and signal handlers.
The default is 100, meaning the check is performed every 100 Python virtual instructions.
Setting it to a larger value may increase performance for programs using threads.
Setting it to a value <= 0 checks every virtual instruction, maximizing responsiveness as well as overhead.
将其设置为<= 0的一个值将检查每个虚拟机指令,这会最大限度地提高响应能力,同时会增加开销。
In [86]: sys.getcheckinterval()
Out[86]: 100

(2)Python 3.2开始为sys.setswitchinterval(interval)   

Set the interpreter’s thread switch interval (in seconds).
This floating-point value determines the ideal duration of the “timeslices” allocated to concurrently running Python threads.
Please note that the actual value can be higher, especially if long-running internal functions or methods are used.
Also, which thread becomes scheduled at the end of the interval is the operating system’s decision.
The interpreter doesn’t have its own scheduler.
New in version 3.2.
>>> sys.getswitchinterval()

5. wiki.python

In CPython, the global interpreter lock, or GIL, is a mutex that protects access to Python objects, preventing multiple threads from executing Python bytecodes at once. This lock is necessary mainly because CPython's memory management is not thread-safe. (However, since the GIL exists, other features have grown to depend on the guarantees that it enforces.)

CPython extensions must be GIL-aware in order to avoid defeating threads. For an explanation, see Global interpreter lock.

The GIL is controversial because it prevents multithreaded CPython programs from taking full advantage of multiprocessor systems in certain situations. Note that potentially blocking or long-running operations, such as I/O, image processing, and NumPy number crunching, happen outside the GIL. Therefore it is only in multithreaded programs that spend a lot of time inside the GIL, interpreting CPython bytecode, that the GIL becomes a bottleneck.

However the GIL can degrade performance even when it is not a bottleneck. Summarizing those slides: The system call overhead is significant, especially on multicore hardware. Two threads calling a function may take twice as much time as a single thread calling the function twice. The GIL can cause I/O-bound threads to be scheduled ahead of CPU-bound threads. And it prevents signals from being delivered.

Non-CPython implementations

  • Jython and IronPython have no GIL and can fully exploit multiprocessor systems

  • PyPy currently has a GIL like CPython

  • in Cython the GIL exists, but can be released temporarily using a "with" statement

Understanding the Python GIL: David Beazley at PyCon 2010   


6. python.doc FAQ




Aahz has a set of slides from his threading tutorial that are helpful; see

posted @ 2017-08-18 13:14 my8100 阅读(...) 评论(...) 编辑 收藏