/
githubmirror
/
cpython
Обзор
Документация
Войти
/
githubmirror
/
cpython
Код
Запросы
0
Пакеты
0
Релизы
0
Аналитика
Безопасность
3.9
Misc/SpecialBuilds.txt
179 строк
8 KB
Victor Stinner
bpo-39489: Remove COUNT_ALLOCS special build (GH-18259)
03 фев 2020, 17:17
Не верифицирован
03 фев 2020, 17:17
c6e5c11
Код
Авторство
О чём код?
This file describes some special Python build types enabled via compile-time preprocessor directives. IMPORTANT: if you want to build a debug-enabled Python, it is recommended that you use ``./configure --with-pydebug``, rather than the options listed here. However, if you wish to define some of these options individually, it is best to define them in the EXTRA_CFLAGS make variable; ``make EXTRA_CFLAGS="-DPy_REF_DEBUG"``. Py_REF_DEBUG ------------ Turn on aggregate reference counting. This arranges that extern _Py_RefTotal hold a count of all references, the sum of ob_refcnt across all objects. Passing ``-X showrefcount`` on the command line causes the interactive interpreter to print the reference count total as well the number of memory blocks allocated after each statement: >>> 23 23 [8288 refs, 14332 blocks] >>> Note that if this count increases when you're not storing away new objects, there's probably a leak. Remember, though, that in interactive mode the special name "_" holds a reference to the last result displayed! Py_REF_DEBUG also checks after every decref to verify that the refcount hasn't gone negative, and causes an immediate fatal error if it has. Py_DEBUG implies Py_REF_DEBUG. Special gimmicks: sys.gettotalrefcount() Return current total of all refcounts. Py_TRACE_REFS ------------- Build option: ``./configure --with-trace-refs``. Turn on heavy reference debugging. This is major surgery. Every PyObject grows two more pointers, to maintain a doubly-linked list of all live heap-allocated objects. Most built-in type objects are not in this list, as they're statically allocated. Note that because the fundamental PyObject layout changes, Python modules compiled with Py_TRACE_REFS are incompatible with modules compiled without it. Special gimmicks: sys.getobjects(max[, type]) Return list of the (no more than) max most-recently allocated objects, most recently allocated first in the list, least-recently allocated last in the list. max=0 means no limit on list length. If an optional type object is passed, the list is also restricted to objects of that type. The return list itself, and some temp objects created just to call sys.getobjects(), are excluded from the return list. Note that the list returned is just another object, though, so may appear in the return list the next time you call getobjects(); note that every object in the list is kept alive too, simply by virtue of being in the list. envvar PYTHONDUMPREFS If this envvar exists, Py_FinalizeEx() arranges to print a list of all still-live heap objects. This is printed twice, in different formats, before and after Py_FinalizeEx has cleaned up everything it can clean up. The first output block produces the repr() of each object so is more informative; however, a lot of stuff destined to die is still alive then. The second output block is much harder to work with (repr() can't be invoked anymore -- the interpreter has been torn down too far), but doesn't list any objects that will die. The tool script combinerefs.py can be run over this to combine the info from both output blocks. The second output block, and combinerefs.py, were new in Python 2.3b1. PYMALLOC_DEBUG -------------- When pymalloc is enabled (WITH_PYMALLOC is defined), calls to the PyObject_ memory routines are handled by Python's own small-object allocator, while calls to the PyMem_ memory routines are directed to the system malloc/ realloc/free. If PYMALLOC_DEBUG is also defined, calls to both PyObject_ and PyMem_ memory routines are directed to a special debugging mode of Python's small-object allocator. This mode fills dynamically allocated memory blocks with special, recognizable bit patterns, and adds debugging info on each end of dynamically allocated memory blocks. The special bit patterns are: #define CLEANBYTE 0xCB /* clean (newly allocated) memory */ #define DEADBYTE 0xDB /* dead (newly freed) memory */ #define FORBIDDENBYTE 0xFB /* forbidden -- untouchable bytes */ Strings of these bytes are unlikely to be valid addresses, floats, or 7-bit ASCII strings. Let S = sizeof(size_t). 2*S bytes are added at each end of each block of N bytes requested. The memory layout is like so, where p represents the address returned by a malloc-like or realloc-like function (p[i:j] means the slice of bytes from *(p+i) inclusive up to *(p+j) exclusive; note that the treatment of negative indices differs from a Python slice): p[-2*S:-S] Number of bytes originally asked for. This is a size_t, big-endian (easier to read in a memory dump). p[-S] API ID. See PEP 445. This is a character, but seems undocumented. p[-S+1:0] Copies of FORBIDDENBYTE. Used to catch under- writes and reads. p[0:N] The requested memory, filled with copies of CLEANBYTE, used to catch reference to uninitialized memory. When a realloc-like function is called requesting a larger memory block, the new excess bytes are also filled with CLEANBYTE. When a free-like function is called, these are overwritten with DEADBYTE, to catch reference to freed memory. When a realloc- like function is called requesting a smaller memory block, the excess old bytes are also filled with DEADBYTE. p[N:N+S] Copies of FORBIDDENBYTE. Used to catch over- writes and reads. p[N+S:N+2*S] A serial number, incremented by 1 on each call to a malloc-like or realloc-like function. Big-endian size_t. If "bad memory" is detected later, the serial number gives an excellent way to set a breakpoint on the next run, to capture the instant at which this block was passed out. The static function bumpserialno() in obmalloc.c is the only place the serial number is incremented, and exists so you can set such a breakpoint easily. A realloc-like or free-like function first checks that the FORBIDDENBYTEs at each end are intact. If they've been altered, diagnostic output is written to stderr, and the program is aborted via Py_FatalError(). The other main failure mode is provoking a memory error when a program reads up one of the special bit patterns and tries to use it as an address. If you get in a debugger then and look at the object, you're likely to see that it's entirely filled with 0xDB (meaning freed memory is getting used) or 0xCB (meaning uninitialized memory is getting used). Note that PYMALLOC_DEBUG requires WITH_PYMALLOC. Py_DEBUG implies PYMALLOC_DEBUG (if WITH_PYMALLOC is enabled). Special gimmicks: envvar PYTHONMALLOCSTATS If this envvar exists, a report of pymalloc summary statistics is printed to stderr whenever a new arena is allocated, and also by Py_FinalizeEx(). Changed in 2.5: The number of extra bytes allocated is 4*sizeof(size_t). Before it was 16 on all boxes, reflecting that Python couldn't make use of allocations >= 2**32 bytes even on 64-bit boxes before 2.5. Py_DEBUG -------- This is what is generally meant by "a debug build" of Python. Py_DEBUG implies LLTRACE, Py_REF_DEBUG, and PYMALLOC_DEBUG (if WITH_PYMALLOC is enabled). In addition, C assert()s are enabled (via the C way: by not defining NDEBUG), and some routines do additional sanity checks inside "#ifdef Py_DEBUG" blocks. LLTRACE ------- Compile in support for Low Level TRACE-ing of the main interpreter loop. When this preprocessor symbol is defined, before PyEval_EvalFrame executes a frame's code it checks the frame's global namespace for a variable "__ltrace__". If such a variable is found, mounds of information about what the interpreter is doing are sprayed to stdout, such as every opcode and opcode argument and values pushed onto and popped off the value stack. Not useful very often, but very useful when needed. Py_DEBUG implies LLTRACE.