| :file:`{exec_prefix}/include/pythonversion/`, where :envvar:`prefix` and |
| :envvar:`exec_prefix` are defined by the corresponding parameters to Python's |
| :program:`configure` script and *version* is ``sys.version[:3]``. On Windows, |
| the headers are installed in :file:`{prefix}/include`, where :envvar:`prefix` is |
| the installation directory specified to the installer. |
| |
| To include the headers, place both directories (if different) on your compiler's |
| search path for includes. Do *not* place the parent directories on the search |
n | path and then use ``#include <python|version|/Python.h>``; this will break on |
n | path and then use ``#include <pythonX.Y/Python.h>``; this will break on |
| multi-platform builds since the platform independent headers under |
| :envvar:`prefix` include the platform specific headers from |
| :envvar:`exec_prefix`. |
| |
| C++ users should note that though the API is defined entirely using C, the |
| header files do properly declare the entry points to be ``extern "C"``, so there |
| is no need to do anything special to use the API from C++. |
| |
| |
n | .. _objects: |
n | .. _api-objects: |
| |
| Objects, Types and Reference Counts |
| =================================== |
| |
| .. index:: object: type |
| |
| Most Python/C API functions have one or more arguments as well as a return value |
| of type :ctype:`PyObject\*`. This type is a pointer to an opaque data type |
| assignments, scope rules, and argument passing), it is only fitting that they |
| should be represented by a single C type. Almost all Python objects live on the |
| heap: you never declare an automatic or static variable of type |
| :ctype:`PyObject`, only pointer variables of type :ctype:`PyObject\*` can be |
| declared. The sole exception are the type objects; since these must never be |
| deallocated, they are typically static :ctype:`PyTypeObject` objects. |
| |
| All Python objects (even Python integers) have a :dfn:`type` and a |
n | :dfn:`reference count`. An object's type determines what kind of object it is |
n | :dfn:`reference count`. An object's type determines what kind of object it is |
| (e.g., an integer, a list, or a user-defined function; there are many more as |
| (e.g., an integer, a list, or a user-defined function; there are many more as |
| explained in the Python Reference Manual (XXX reference: ../ref/ref.html)). For |
| explained in :ref:`types`). For each of the well-known types there is a macro |
| each of the well-known types there is a macro to check whether an object is of |
| to check whether an object is of that type; for instance, ``PyList_Check(a)`` is |
| that type; for instance, ``PyList_Check(a)`` is true if (and only if) the object |
| true if (and only if) the object pointed to by *a* is a Python list. |
| pointed to by *a* is a Python list. |
| |
| |
n | .. _refcounts: |
n | .. _api-refcounts: |
| |
| Reference Counts |
| ---------------- |
| |
| The reference count is important because today's computers have a finite (and |
| often severely limited) memory size; it counts how many different places there |
| are that have a reference to an object. Such a place could be another object, |
| or a global (or static) C variable, or a local variable in some C function. |
| is considerably more complex than the incref one, since it must check whether |
| the reference count becomes zero and then cause the object's deallocator to be |
| called. The deallocator is a function pointer contained in the object's type |
| structure. The type-specific deallocator takes care of decrementing the |
| reference counts for other objects contained in the object if this is a compound |
| object type, such as a list, as well as performing any additional finalization |
| that's needed. There's no chance that the reference count can overflow; at |
| least as many bits are used to hold the reference count as there are distinct |
n | memory locations in virtual memory (assuming ``sizeof(long) >= sizeof(char*)``). |
n | memory locations in virtual memory (assuming ``sizeof(Py_ssize_t) >= sizeof(void*)``). |
| Thus, the reference count increment is a simple operation. |
| |
| It is not necessary to increment an object's reference count for every local |
| variable that contains a pointer to an object. In theory, the object's |
| reference count goes up by one when the variable is made to point to it and it |
| goes down by one when the variable goes out of scope. However, these two |
| cancel each other out, so at the end the reference count hasn't changed. The |
| only real reason to use the reference count is to prevent the object from being |
| |
| A safe approach is to always use the generic operations (functions whose name |
| begins with ``PyObject_``, ``PyNumber_``, ``PySequence_`` or ``PyMapping_``). |
| These operations always increment the reference count of the object they return. |
| This leaves the caller with the responsibility to call :cfunc:`Py_DECREF` when |
| they are done with the result; this soon becomes second nature. |
| |
| |
n | .. _refcountdetails: |
n | .. _api-refcountdetails: |
| |
| Reference Count Details |
| ^^^^^^^^^^^^^^^^^^^^^^^ |
| |
| The reference count behavior of functions in the Python/C API is best explained |
| in terms of *ownership of references*. Ownership pertains to references, never |
| to objects (objects are not owned: they are always shared). "Owning a |
| reference" means being responsible for calling Py_DECREF on it when the |
| the code that receives ownership of the reference then becomes responsible for |
| eventually decref'ing it by calling :cfunc:`Py_DECREF` or :cfunc:`Py_XDECREF` |
| when it's no longer needed---or passing on this responsibility (usually to its |
| caller). When a function passes ownership of a reference on to its caller, the |
| caller is said to receive a *new* reference. When no ownership is transferred, |
| the caller is said to *borrow* the reference. Nothing needs to be done for a |
| borrowed reference. |
| |
n | Conversely, when a calling function passes it a reference to an object, there |
n | Conversely, when a calling function passes in a reference to an object, there |
| are two possibilities: the function *steals* a reference to the object, or it |
| does not. *Stealing a reference* means that when you pass a reference to a |
| function, that function assumes that it now owns that reference, and you are not |
| responsible for it any longer. |
| |
| .. index:: |
| single: PyList_SetItem() |
| single: PyTuple_SetItem() |
| Py_DECREF(item); /* Discard reference ownership */ |
| } |
| return total; |
| } |
| |
| .. index:: single: sum_sequence() |
| |
| |
n | .. _types: |
n | .. _api-types: |
| |
| Types |
| ----- |
| |
| There are few other data types that play a significant role in the Python/C |
| API; most are simple C types such as :ctype:`int`, :ctype:`long`, |
| :ctype:`double` and :ctype:`char\*`. A few structure types are used to |
| describe static tables used to list the functions exported by a module or the |
| data attributes of a new object type, and another is used to describe the value |
| of a complex number. These will be discussed together with the functions that |
| use them. |
| |
| |
n | .. _exceptions: |
n | .. _api-exceptions: |
| |
| Exceptions |
| ========== |
| |
| The Python programmer only needs to deal with exceptions if specific error |
| handling is required; unhandled exceptions are automatically propagated to the |
| caller, then to the caller's caller, and so on, until they reach the top-level |
| interpreter, where they are reported to the user accompanied by a stack |
| are not the same: the Python objects represent the last exception being handled |
| by a Python :keyword:`try` ... :keyword:`except` statement, while the C level |
| exception state only exists while an exception is being passed on between C |
| functions until it reaches the Python bytecode interpreter's main loop, which |
| takes care of transferring it to ``sys.exc_type`` and friends. |
| |
| .. index:: single: exc_info() (in module sys) |
| |
n | Note that starting with Python 1.5, the preferred, thread-safe way to access |
n | Note that starting with Python 1.5, the preferred, thread-safe way to access the |
| the exception state from Python code is to call the function |
| exception state from Python code is to call the function :func:`sys.exc_info`, |
| :func:`sys.exc_info`, which returns the per-thread exception state for Python |
| which returns the per-thread exception state for Python code. Also, the |
| code. Also, the semantics of both ways to access the exception state have |
| semantics of both ways to access the exception state have changed so that a |
| changed so that a function which catches an exception will save and restore its |
| function which catches an exception will save and restore its thread's exception |
| thread's exception state so as to preserve the exception state of its caller. |
| state so as to preserve the exception state of its caller. This prevents common |
| This prevents common bugs in exception handling code caused by an innocent- |
| bugs in exception handling code caused by an innocent-looking function |
| looking function overwriting the exception being handled; it also reduces the |
| overwriting the exception being handled; it also reduces the often unwanted |
| often unwanted lifetime extension for objects that are referenced by the stack |
| lifetime extension for objects that are referenced by the stack frames in the |
| frames in the traceback. |
| traceback. |
| |
| As a general principle, a function that calls another function to perform some |
| task should check whether the called function raised an exception, and if so, |
| pass the exception state on to its caller. It should discard any object |
| references that it owns, and return an error indicator, but it should *not* set |
| another exception --- that would overwrite the exception that was just raised, |
| and lose important information about the exact cause of the error. |
| |
| |
| .. index:: single: incr_item() |
| |
| .. index:: |
| single: PyErr_ExceptionMatches() |
| single: PyErr_Clear() |
| single: Py_XDECREF() |
| |
n | This example represents an endorsed use of the :keyword:`goto` statement in C! |
n | This example represents an endorsed use of the ``goto`` statement in C! |
| It illustrates the use of :cfunc:`PyErr_ExceptionMatches` and |
| :cfunc:`PyErr_Clear` to handle specific exceptions, and the use of |
| :cfunc:`Py_XDECREF` to dispose of owned references that may be *NULL* (note the |
| ``'X'`` in the name; :cfunc:`Py_DECREF` would crash when confronted with a |
| *NULL* reference). It is important that the variables used to hold owned |
| references are initialized to *NULL* for this to work; likewise, the proposed |
| return value is initialized to ``-1`` (failure) and only set to success after |
| the final call made is successful. |
| |
| |
n | .. _embedding: |
n | .. _api-embedding: |
| |
| Embedding Python |
| ================ |
| |
| The one important task that only embedders (as opposed to extension writers) of |
| the Python interpreter have to worry about is the initialization, and possibly |
| the finalization, of the Python interpreter. Most functionality of the |
| interpreter can only be used after the interpreter has been initialized. |
| triple: module; search; path |
| single: path (in module sys) |
| |
| The basic initialization function is :cfunc:`Py_Initialize`. This initializes |
| the table of loaded modules, and creates the fundamental modules |
| :mod:`__builtin__`, :mod:`__main__`, :mod:`sys`, and :mod:`exceptions`. It also |
| initializes the module search path (``sys.path``). |
| |
n | .. % |
| |
| .. index:: single: PySys_SetArgv() |
| |
| :cfunc:`Py_Initialize` does not set the "script argument list" (``sys.argv``). |
| If this variable is needed by Python code that will be executed later, it must |
| be set explicitly with a call to ``PySys_SetArgv(argc, argv)`` subsequent to |
| the call to :cfunc:`Py_Initialize`. |
| |
| On most systems (in particular, on Unix and Windows, although the details are |
| slightly different), :cfunc:`Py_Initialize` calculates the module search path |
| based upon its best guess for the location of the standard Python interpreter |
| executable, assuming that the Python library is found in a fixed location |
| relative to the Python interpreter executable. In particular, it looks for a |
n | directory named :file:`lib/python|version|` relative to the parent directory |
n | directory named :file:`lib/python{X.Y}` relative to the parent directory |
| where the executable named :file:`python` is found on the shell command search |
| path (the environment variable :envvar:`PATH`). |
| |
| For instance, if the Python executable is found in |
| :file:`/usr/local/bin/python`, it will assume that the libraries are in |
n | :file:`/usr/local/lib/python|version|`. (In fact, this particular path is also |
n | :file:`/usr/local/lib/python{X.Y}`. (In fact, this particular path is also |
| the "fallback" location, used when no executable file named :file:`python` is |
| found along :envvar:`PATH`.) The user can override this behavior by setting the |
| environment variable :envvar:`PYTHONHOME`, or insert additional directories in |
| front of the standard path by setting :envvar:`PYTHONPATH`. |
| |
| .. index:: |
| single: Py_SetProgramName() |
| single: Py_GetPath() |
| Python and wants to free memory allocated by Python. This can be accomplished |
| by calling :cfunc:`Py_Finalize`. The function :cfunc:`Py_IsInitialized` returns |
| true if Python is currently in the initialized state. More information about |
| these functions is given in a later chapter. Notice that :cfunc:`Py_Finalize` |
| does *not* free all memory allocated by the Python interpreter, e.g. memory |
| allocated by extension modules currently cannot be released. |
| |
| |
n | .. _debugging: |
n | .. _api-debugging: |
| |
| Debugging Builds |
| ================ |
| |
| Python can be built with several macros to enable extra checks of the |
| interpreter and extension modules. These checks tend to add a large amount of |
| overhead to the runtime so they are not enabled by default. |
| |
| :file:`Misc/SpecialBuilds.txt` in the Python source distribution. Builds are |
| available that support tracing of reference counts, debugging the memory |
| allocator, or low-level profiling of the main interpreter loop. Only the most |
| frequently-used builds will be described in the remainder of this section. |
| |
| Compiling the interpreter with the :cmacro:`Py_DEBUG` macro defined produces |
| what is generally meant by "a debug build" of Python. :cmacro:`Py_DEBUG` is |
| enabled in the Unix build by adding :option:`--with-pydebug` to the |
t | :file:`configure` command. It is also implied by the presence of the not- |
t | :file:`configure` command. It is also implied by the presence of the |
| Python-specific :cmacro:`_DEBUG` macro. When :cmacro:`Py_DEBUG` is enabled in |
| not-Python-specific :cmacro:`_DEBUG` macro. When :cmacro:`Py_DEBUG` is enabled |
| the Unix build, compiler optimization is disabled. |
| in the Unix build, compiler optimization is disabled. |
| |
| In addition to the reference count debugging described below, the following |
| extra checks are performed: |
| |
| * Extra checks are added to the object allocator. |
| |
| * Extra checks are added to the parser and compiler. |
| |