rest25/extending/extending.rst => rest262/extending/extending.rst | ||
---|---|---|
f | 1 | .. highlightlang:: c |
2 | ||
3 | ||
n | 4- | .. _intro: |
n | 4+ | .. _extending-intro: |
5 | ||
6 | ****************************** | |
7 | Extending Python with C or C++ | |
8 | ****************************** | |
9 | ||
10 | It is quite easy to add new built-in modules to Python, if you know how to | |
11 | program in C. Such :dfn:`extension modules` can do two things that can't be | |
12 | done directly in Python: they can implement new built-in object types, and they | |
16 | defines a set of functions, macros and variables that provide access to most | |
17 | aspects of the Python run-time system. The Python API is incorporated in a C | |
18 | source file by including the header ``"Python.h"``. | |
19 | ||
20 | The compilation of an extension module depends on its intended use as well as on | |
21 | your system setup; details are given in later chapters. | |
22 | ||
23 | ||
n | 24- | .. _simpleexample: |
n | 24+ | .. _extending-simpleexample: |
25 | ||
26 | A Simple Example | |
27 | ================ | |
28 | ||
29 | Let's create an extension module called ``spam`` (the favorite food of Monty | |
30 | Python fans...) and let's say we want to create a Python interface to the C | |
31 | library function :cfunc:`system`. [#]_ This function takes a null-terminated | |
32 | character string as argument and returns an integer. We want this function to | |
98 | ||
99 | :cfunc:`PyArg_ParseTuple` returns true (nonzero) if all arguments have the right | |
100 | type and its components have been stored in the variables whose addresses are | |
101 | passed. It returns false (zero) if an invalid argument list was passed. In the | |
102 | latter case it also raises an appropriate exception so the calling function can | |
103 | return *NULL* immediately (as we saw in the example). | |
104 | ||
105 | ||
n | 106- | .. _errors: |
n | 106+ | .. _extending-errors: |
107 | ||
108 | Intermezzo: Errors and Exceptions | |
109 | ================================= | |
110 | ||
111 | An important convention throughout the Python interpreter is the following: when | |
112 | a function fails, it should set an exception condition and return an error value | |
113 | (usually a *NULL* pointer). Exceptions are stored in a static global variable | |
114 | inside the interpreter; if this variable is *NULL* no exception has occurred. A | |
115 | second global variable stores the "associated value" of the exception (the | |
116 | second argument to :keyword:`raise`). A third variable contains the stack | |
117 | traceback in case the error originated in Python code. These three variables | |
118 | are the C equivalents of the Python variables ``sys.exc_type``, | |
119 | ``sys.exc_value`` and ``sys.exc_traceback`` (see the section on module | |
n | 120- | :mod:`sys` in the Python Library Reference (XXX reference: ../lib/lib.html)). |
n | 120+ | :mod:`sys` in the Python Library Reference). It is important to know about them |
121- | It is important to know about them to understand how errors are passed around. | |
121+ | to understand how errors are passed around. | |
122 | ||
123 | The Python API defines a number of functions to set various types of exceptions. | |
124 | ||
125 | The most common one is :cfunc:`PyErr_SetString`. Its arguments are an exception | |
126 | object and a C string. The exception object is usually a predefined object like | |
127 | :cdata:`PyExc_ZeroDivisionError`. The C string indicates the cause of the error | |
128 | and is converted to a Python string object and stored as the "associated value" | |
129 | of the exception. | |
160 | To ignore an exception set by a function call that failed, the exception | |
161 | condition must be cleared explicitly by calling :cfunc:`PyErr_Clear`. The only | |
162 | time C code should call :cfunc:`PyErr_Clear` is if it doesn't want to pass the | |
163 | error on to the interpreter but wants to handle it completely by itself | |
164 | (possibly by trying something else, or pretending nothing went wrong). | |
165 | ||
166 | Every failing :cfunc:`malloc` call must be turned into an exception --- the | |
167 | direct caller of :cfunc:`malloc` (or :cfunc:`realloc`) must call | |
n | 168- | :cfunc:`PyErr_NoMemory` and return a failure indicator itself. All the object- |
n | 168+ | :cfunc:`PyErr_NoMemory` and return a failure indicator itself. All the |
169- | creating functions (for example, :cfunc:`PyInt_FromLong`) already do this, so | |
169+ | object-creating functions (for example, :cfunc:`PyInt_FromLong`) already do | |
170- | this note is only relevant to those who call :cfunc:`malloc` directly. | |
170+ | this, so this note is only relevant to those who call :cfunc:`malloc` directly. | |
171 | ||
172 | Also note that, with the important exception of :cfunc:`PyArg_ParseTuple` and | |
173 | friends, functions that return an integer status usually return a positive value | |
174 | or zero for success and ``-1`` for failure, like Unix system calls. | |
175 | ||
176 | Finally, be careful to clean up garbage (by making :cfunc:`Py_XDECREF` or | |
177 | :cfunc:`Py_DECREF` calls for objects you have already created) when you return | |
178 | an error indicator! | |
196 | with an exception object (leaving out the error checking for now):: | |
197 | ||
198 | PyMODINIT_FUNC | |
199 | initspam(void) | |
200 | { | |
201 | PyObject *m; | |
202 | ||
203 | m = Py_InitModule("spam", SpamMethods); | |
n | 204+ | if (m == NULL) |
205+ | return; | |
204 | ||
205 | SpamError = PyErr_NewException("spam.error", NULL, NULL); | |
206 | Py_INCREF(SpamError); | |
207 | PyModule_AddObject(m, "error", SpamError); | |
208 | } | |
209 | ||
210 | Note that the Python name for the exception object is :exc:`spam.error`. The | |
211 | :cfunc:`PyErr_NewException` function may create a class with the base class | |
212 | being :exc:`Exception` (unless another class is passed in instead of *NULL*), | |
n | 213- | described in the Python Library Reference (XXX reference: ../lib/lib.html) under |
n | 215+ | described in :ref:`bltin-exceptions`. |
214- | "Built-in Exceptions." | |
215 | ||
216 | Note also that the :cdata:`SpamError` variable retains a reference to the newly | |
217 | created exception class; this is intentional! Since the exception could be | |
218 | removed from the module by external code, an owned reference to the class is | |
219 | needed to ensure that it will not be discarded, causing :cdata:`SpamError` to | |
220 | become a dangling pointer. Should it become a dangling pointer, C code which | |
221 | raises the exception could cause a core dump or other unintended side effects. | |
222 | ||
301 | arguments should be passed to the function. In this case, the C function should | |
302 | accept a third ``PyObject *`` parameter which will be a dictionary of keywords. | |
303 | Use :cfunc:`PyArg_ParseTupleAndKeywords` to parse the arguments to such a | |
304 | function. | |
305 | ||
306 | The method table must be passed to the interpreter in the module's | |
307 | initialization function. The initialization function must be named | |
308 | :cfunc:`initname`, where *name* is the name of the module, and should be the | |
n | 309- | only non-\ :keyword:`static` item defined in the module file:: |
n | 310+ | only non-\ ``static`` item defined in the module file:: |
310 | ||
311 | PyMODINIT_FUNC | |
312 | initspam(void) | |
313 | { | |
314 | (void) Py_InitModule("spam", SpamMethods); | |
315 | } | |
316 | ||
317 | Note that PyMODINIT_FUNC declares the function as ``void`` return type, | |
320 | ||
321 | When the Python program imports module :mod:`spam` for the first time, | |
322 | :cfunc:`initspam` is called. (See below for comments about embedding Python.) | |
323 | It calls :cfunc:`Py_InitModule`, which creates a "module object" (which is | |
324 | inserted in the dictionary ``sys.modules`` under the key ``"spam"``), and | |
325 | inserts built-in function objects into the newly created module based upon the | |
326 | table (an array of :ctype:`PyMethodDef` structures) that was passed as its | |
327 | second argument. :cfunc:`Py_InitModule` returns a pointer to the module object | |
n | 328- | that it creates (which is unused here). It aborts with a fatal error if the |
n | 329+ | that it creates (which is unused here). It may abort with a fatal error for |
329- | module could not be initialized satisfactorily, so the caller doesn't need to | |
330+ | certain errors, or return *NULL* if the module could not be initialized | |
330- | check for errors. | |
331+ | satisfactorily. | |
331 | ||
332 | When embedding Python, the :cfunc:`initspam` function is not called | |
333 | automatically unless there's an entry in the :cdata:`_PyImport_Inittab` table. | |
n | 334- | The easiest way to handle this is to statically initialize your statically- |
n | 335+ | The easiest way to handle this is to statically initialize your |
335- | linked modules by directly calling :cfunc:`initspam` after the call to | |
336+ | statically-linked modules by directly calling :cfunc:`initspam` after the call | |
336- | :cfunc:`Py_Initialize`:: | |
337+ | to :cfunc:`Py_Initialize`:: | |
337 | ||
338 | int | |
339 | main(int argc, char *argv[]) | |
340 | { | |
341 | /* Pass argv[0] to the Python interpreter */ | |
342 | Py_SetProgramName(argv[0]); | |
343 | ||
344 | /* Initialize the Python interpreter. Required. */ | |
417 | callback mechanism to the Python programmer; the implementation will require | |
418 | calling the Python callback functions from a C callback. Other uses are also | |
419 | imaginable. | |
420 | ||
421 | Fortunately, the Python interpreter is easily called recursively, and there is a | |
422 | standard interface to call a Python function. (I won't dwell on how to call the | |
423 | Python parser with a particular string as input --- if you're interested, have a | |
424 | look at the implementation of the :option:`-c` command line option in | |
n | 425- | :file:`Python/pythonmain.c` from the Python source code.) |
n | 426+ | :file:`Modules/main.c` from the Python source code.) |
426 | ||
427 | Calling a Python function is easy. First, the Python program must somehow pass | |
428 | you the Python function object. You should provide a function (or some other | |
429 | interface) to do this. When this function is called, save a pointer to the | |
430 | Python function object (be careful to :cfunc:`Py_INCREF` it!) in a global | |
431 | variable --- or wherever you see fit. For example, the following function might | |
432 | be part of a module definition:: | |
433 | ||
450 | /* Boilerplate to return "None" */ | |
451 | Py_INCREF(Py_None); | |
452 | result = Py_None; | |
453 | } | |
454 | return result; | |
455 | } | |
456 | ||
457 | This function must be registered with the interpreter using the | |
n | 458- | :const:`METH_VARARGS` flag; this is described in section :ref:`methodtable`, |
n | 459+ | :const:`METH_VARARGS` flag; this is described in section :ref:`methodtable`. The |
459- | "The Module's Method Table and Initialization Function." The | |
460 | :cfunc:`PyArg_ParseTuple` function and its arguments are documented in section | |
n | 461- | :ref:`parsetuple`, "Extracting Parameters in Extension Functions." |
n | 461+ | :ref:`parsetuple`. |
462 | ||
463 | The macros :cfunc:`Py_XINCREF` and :cfunc:`Py_XDECREF` increment/decrement the | |
464 | reference count of an object and are safe in the presence of *NULL* pointers | |
465 | (but note that *temp* will not be *NULL* in this context). More info on them | |
n | 466- | in section :ref:`refcounts`, "Reference Counts." |
n | 466+ | in section :ref:`refcounts`. |
467 | ||
n | 468- | .. index:: single: PyEval_CallObject() |
n | 468+ | .. index:: single: PyObject_CallObject() |
469 | ||
470 | Later, when it is time to call the function, you call the C function | |
n | 471- | :cfunc:`PyEval_CallObject`. This function has two arguments, both pointers to |
n | 471+ | :cfunc:`PyObject_CallObject`. This function has two arguments, both pointers to |
472 | arbitrary Python objects: the Python function, and the argument list. The | |
473 | argument list must always be a tuple object, whose length is the number of | |
n | 474- | arguments. To call the Python function with no arguments, pass an empty tuple; |
n | 474+ | arguments. To call the Python function with no arguments, pass in NULL, or |
475- | to call it with one argument, pass a singleton tuple. :cfunc:`Py_BuildValue` | |
475+ | an empty tuple; to call it with one argument, pass a singleton tuple. | |
476- | returns a tuple when its format string consists of zero or more format codes | |
476+ | :cfunc:`Py_BuildValue` returns a tuple when its format string consists of zero | |
477- | between parentheses. For example:: | |
477+ | or more format codes between parentheses. For example:: | |
478 | ||
479 | int arg; | |
480 | PyObject *arglist; | |
481 | PyObject *result; | |
482 | ... | |
483 | arg = 123; | |
484 | ... | |
485 | /* Time to call the callback */ | |
486 | arglist = Py_BuildValue("(i)", arg); | |
n | 487- | result = PyEval_CallObject(my_callback, arglist); |
n | 487+ | result = PyObject_CallObject(my_callback, arglist); |
488 | Py_DECREF(arglist); | |
489 | ||
n | 490- | :cfunc:`PyEval_CallObject` returns a Python object pointer: this is the return |
n | 490+ | :cfunc:`PyObject_CallObject` returns a Python object pointer: this is the return |
491- | value of the Python function. :cfunc:`PyEval_CallObject` is "reference-count- | |
491+ | value of the Python function. :cfunc:`PyObject_CallObject` is | |
492- | neutral" with respect to its arguments. In the example a new tuple was created | |
492+ | "reference-count-neutral" with respect to its arguments. In the example a new | |
493- | to serve as the argument list, which is :cfunc:`Py_DECREF`\ -ed immediately | |
493+ | tuple was created to serve as the argument list, which is :cfunc:`Py_DECREF`\ | |
494- | after the call. | |
494+ | -ed immediately after the call. | |
495 | ||
n | 496- | The return value of :cfunc:`PyEval_CallObject` is "new": either it is a brand |
n | 496+ | The return value of :cfunc:`PyObject_CallObject` is "new": either it is a brand |
497 | new object, or it is an existing object whose reference count has been | |
498 | incremented. So, unless you want to save it in a global variable, you should | |
499 | somehow :cfunc:`Py_DECREF` the result, even (especially!) if you are not | |
500 | interested in its value. | |
501 | ||
502 | Before you do this, however, it is important to check that the return value | |
503 | isn't *NULL*. If it is, the Python function terminated by raising an exception. | |
n | 504- | If the C code that called :cfunc:`PyEval_CallObject` is called from Python, it |
n | 504+ | If the C code that called :cfunc:`PyObject_CallObject` is called from Python, it |
505 | should now return an error indication to its Python caller, so the interpreter | |
506 | can print a stack trace, or the calling Python code can handle the exception. | |
507 | If this is not possible or desirable, the exception should be cleared by calling | |
508 | :cfunc:`PyErr_Clear`. For example:: | |
509 | ||
510 | if (result == NULL) | |
511 | return NULL; /* Pass error back */ | |
512 | ...use result... | |
n | 513- | Py_DECREF(result); |
n | 513+ | Py_DECREF(result); |
514 | ||
515 | Depending on the desired interface to the Python callback function, you may also | |
n | 516- | have to provide an argument list to :cfunc:`PyEval_CallObject`. In some cases |
n | 516+ | have to provide an argument list to :cfunc:`PyObject_CallObject`. In some cases |
517 | the argument list is also provided by the Python program, through the same | |
518 | interface that specified the callback function. It can then be saved and used | |
519 | in the same manner as the function object. In other cases, you may have to | |
520 | construct a new tuple to pass as the argument list. The simplest way to do this | |
521 | is to call :cfunc:`Py_BuildValue`. For example, if you want to pass an integral | |
522 | event code, you might use the following code:: | |
523 | ||
524 | PyObject *arglist; | |
525 | ... | |
526 | arglist = Py_BuildValue("(l)", eventcode); | |
n | 527- | result = PyEval_CallObject(my_callback, arglist); |
n | 527+ | result = PyObject_CallObject(my_callback, arglist); |
528 | Py_DECREF(arglist); | |
529 | if (result == NULL) | |
530 | return NULL; /* Pass error back */ | |
531 | /* Here maybe use the result */ | |
532 | Py_DECREF(result); | |
533 | ||
534 | Note the placement of ``Py_DECREF(arglist)`` immediately after the call, before | |
n | 535- | the error check! Also note that strictly spoken this code is not complete: |
n | 535+ | the error check! Also note that strictly speaking this code is not complete: |
536 | :cfunc:`Py_BuildValue` may run out of memory, and this should be checked. | |
n | 537+ | |
538+ | You may also call a function with keyword arguments by using | |
539+ | :cfunc:`PyObject_Call`, which supports arguments and keyword arguments. As in | |
540+ | the above example, we use :cfunc:`Py_BuildValue` to construct the dictionary. :: | |
541+ | ||
542+ | PyObject *dict; | |
543+ | ... | |
544+ | dict = Py_BuildValue("{s:i}", "name", val); | |
545+ | result = PyObject_Call(my_callback, NULL, dict); | |
546+ | Py_DECREF(dict); | |
547+ | if (result == NULL) | |
548+ | return NULL; /* Pass error back */ | |
549+ | /* Here maybe use the result */ | |
550+ | Py_DECREF(result); | |
537 | ||
538 | ||
539 | .. _parsetuple: | |
540 | ||
541 | Extracting Parameters in Extension Functions | |
542 | ============================================ | |
543 | ||
544 | .. index:: single: PyArg_ParseTuple() | |
545 | ||
546 | The :cfunc:`PyArg_ParseTuple` function is declared as follows:: | |
547 | ||
548 | int PyArg_ParseTuple(PyObject *arg, char *format, ...); | |
549 | ||
550 | The *arg* argument must be a tuple object containing an argument list passed | |
551 | from Python to a C function. The *format* argument must be a format string, | |
n | 552- | whose syntax is explained in "Parsing arguments and building values (XXX |
n | 566+ | whose syntax is explained in :ref:`arg-parsing` in the Python/C API Reference |
553- | reference: ../api/arg-parsing.html)" in the Python/C API Reference Manual (XXX | |
567+ | Manual. The remaining arguments must be addresses of variables whose type is | |
554- | reference: ../api/api.html). The remaining arguments must be addresses of | |
568+ | determined by the format string. | |
555- | variables whose type is determined by the format string. | |
556 | ||
557 | Note that while :cfunc:`PyArg_ParseTuple` checks that the Python arguments have | |
558 | the required types, it cannot check the validity of the addresses of C variables | |
559 | passed to the call: if you make mistakes there, your code will probably crash or | |
560 | at least overwrite random bits in memory. So be careful! | |
561 | ||
562 | Note that any Python object references which are provided to the caller are | |
563 | *borrowed* references; do not decrement their reference count! | |
648 | ||
649 | Nested tuples cannot be parsed when using keyword arguments! Keyword parameters | |
650 | passed in which are not present in the *kwlist* will cause :exc:`TypeError` to | |
651 | be raised. | |
652 | ||
653 | .. index:: single: Philbrick, Geoff | |
654 | ||
655 | Here is an example module which uses keywords, based on an example by Geoff | |
n | 656- | Philbrick (philbrick@hks.com): |
n | 669+ | Philbrick (philbrick@hks.com):: |
657- | ||
658- | .. % | |
659- | ||
660- | :: | |
661 | ||
662 | #include "Python.h" | |
663 | ||
664 | static PyObject * | |
665 | keywdarg_parrot(PyObject *self, PyObject *args, PyObject *keywds) | |
n | 666- | { |
n | 675+ | { |
667 | int voltage; | |
668 | char *state = "a stiff"; | |
669 | char *action = "voom"; | |
670 | char *type = "Norwegian Blue"; | |
671 | ||
672 | static char *kwlist[] = {"voltage", "state", "action", "type", NULL}; | |
673 | ||
n | 674- | if (!PyArg_ParseTupleAndKeywords(args, keywds, "i|sss", kwlist, |
n | 683+ | if (!PyArg_ParseTupleAndKeywords(args, keywds, "i|sss", kwlist, |
675 | &voltage, &state, &action, &type)) | |
n | 676- | return NULL; |
n | 685+ | return NULL; |
677 | ||
n | 678- | printf("-- This parrot wouldn't %s if you put %i Volts through it.\n", |
n | 687+ | printf("-- This parrot wouldn't %s if you put %i Volts through it.\n", |
679 | action, voltage); | |
680 | printf("-- Lovely plumage, the %s -- It's %s!\n", type, state); | |
681 | ||
682 | Py_INCREF(Py_None); | |
683 | ||
684 | return Py_None; | |
685 | } | |
686 | ||
748 | ||
749 | .. _refcounts: | |
750 | ||
751 | Reference Counts | |
752 | ================ | |
753 | ||
754 | In languages like C or C++, the programmer is responsible for dynamic allocation | |
755 | and deallocation of memory on the heap. In C, this is done using the functions | |
n | 756- | :cfunc:`malloc` and :cfunc:`free`. In C++, the operators :keyword:`new` and |
n | 765+ | :cfunc:`malloc` and :cfunc:`free`. In C++, the operators ``new`` and |
757- | :keyword:`delete` are used with essentially the same meaning and we'll restrict | |
766+ | ``delete`` are used with essentially the same meaning and we'll restrict | |
758 | the following discussion to the C case. | |
759 | ||
760 | Every block of memory allocated with :cfunc:`malloc` should eventually be | |
761 | returned to the pool of available memory by exactly one call to :cfunc:`free`. | |
762 | It is important to call :cfunc:`free` at the right time. If a block's address | |
763 | is forgotten but :cfunc:`free` is not called for it, the memory it occupies | |
764 | cannot be reused until the program terminates. This is called a :dfn:`memory | |
765 | leak`. On the other hand, if a program calls :cfunc:`free` for a block and then | |
810 | which is non-zero. Typical reference counting implementations are not able to | |
811 | reclaim the memory belonging to any objects in a reference cycle, or referenced | |
812 | from the objects in the cycle, even though there are no further references to | |
813 | the cycle itself. | |
814 | ||
815 | The cycle detector is able to detect garbage cycles and can reclaim them so long | |
816 | as there are no finalizers implemented in Python (:meth:`__del__` methods). | |
817 | When there are such finalizers, the detector exposes the cycles through the | |
n | 818- | :mod:`gc` module (XXX reference: ../lib/module-gc.html) (specifically, the |
n | 827+ | :mod:`gc` module (specifically, the |
819 | ``garbage`` variable in that module). The :mod:`gc` module also exposes a way | |
820 | to run the detector (the :func:`collect` function), as well as configuration | |
821 | interfaces and the ability to disable the detector at runtime. The cycle | |
822 | detector is considered an optional component; though it is included by default, | |
823 | it can be disabled at build time using the :option:`--without-cycle-gc` option | |
824 | to the :program:`configure` script on Unix platforms (including Mac OS X) or by | |
825 | removing the definition of ``WITH_CYCLE_GC`` in the :file:`pyconfig.h` header on | |
826 | other platforms. If the cycle detector is disabled in this way, the :mod:`gc` | |
852 | borrower of a reference should not call :cfunc:`Py_DECREF`. The borrower must | |
853 | not hold on to the object longer than the owner from which it was borrowed. | |
854 | Using a borrowed reference after the owner has disposed of it risks using freed | |
855 | memory and should be avoided completely. [#]_ | |
856 | ||
857 | The advantage of borrowing over owning a reference is that you don't need to | |
858 | take care of disposing of the reference on all possible paths through the code | |
859 | --- in other words, with a borrowed reference you don't run the risk of leaking | |
n | 860- | when a premature exit is taken. The disadvantage of borrowing over leaking is |
n | 869+ | when a premature exit is taken. The disadvantage of borrowing over owning is |
861 | that there are some subtle situations where in seemingly correct code a borrowed | |
862 | reference can be used after the owner from which it was borrowed has in fact | |
863 | disposed of it. | |
864 | ||
865 | A borrowed reference can be changed into an owned reference by calling | |
866 | :cfunc:`Py_INCREF`. This does not affect the status of the owner from which the | |
867 | reference was borrowed --- it creates a new owned reference, and gives full | |
868 | owner responsibilities (the new owner must dispose of the reference properly, as | |
1022 | checking. | |
1023 | ||
1024 | The C function calling mechanism guarantees that the argument list passed to C | |
1025 | functions (``args`` in the examples) is never *NULL* --- in fact it guarantees | |
1026 | that it is always a tuple. [#]_ | |
1027 | ||
1028 | It is a severe error to ever let a *NULL* pointer "escape" to the Python user. | |
1029 | ||
n | 1030- | .. % Frank Stajano: |
n | 1039+ | .. Frank Stajano: |
1031- | .. % A pedagogically buggy example, along the lines of the previous listing, | |
1040+ | A pedagogically buggy example, along the lines of the previous listing, would | |
1032- | .. % would be helpful here -- showing in more concrete terms what sort of | |
1041+ | be helpful here -- showing in more concrete terms what sort of actions could | |
1033- | .. % actions could cause the problem. I can't very well imagine it from the | |
1042+ | cause the problem. I can't very well imagine it from the description. | |
1034- | .. % description. | |
1035 | ||
1036 | ||
1037 | .. _cplusplus: | |
1038 | ||
1039 | Writing Extensions in C++ | |
1040 | ========================= | |
1041 | ||
1042 | It is possible to write extension modules in C++. Some restrictions apply. If | |
1062 | Python, but sometimes the code in an extension module can be useful for other | |
1063 | extension modules. For example, an extension module could implement a type | |
1064 | "collection" which works like lists without order. Just like the standard Python | |
1065 | list type has a C API which permits extension modules to create and manipulate | |
1066 | lists, this new collection type should have a set of C functions for direct | |
1067 | manipulation from other extension modules. | |
1068 | ||
1069 | At first sight this seems easy: just write the functions (without declaring them | |
n | 1070- | :keyword:`static`, of course), provide an appropriate header file, and document |
n | 1078+ | ``static``, of course), provide an appropriate header file, and document |
1071 | the C API. And in fact this would work if all extension modules were always | |
1072 | linked statically with the Python interpreter. When modules are used as shared | |
1073 | libraries, however, the symbols defined in one module may not be visible to | |
1074 | another module. The details of visibility depend on the operating system; some | |
1075 | systems use one global namespace for the Python interpreter and all extension | |
1076 | modules (Windows, for example), whereas others require an explicit list of | |
1077 | imported symbols at module link time (AIX is one example), or offer a choice of | |
1078 | different strategies (most Unices). And even if symbols are globally visible, | |
1079 | the module whose functions one wishes to call might not have been loaded yet! | |
1080 | ||
1081 | Portability therefore requires not to make any assumptions about symbol | |
1082 | visibility. This means that all symbols in extension modules should be declared | |
n | 1083- | :keyword:`static`, except for the module's initialization function, in order to |
n | 1091+ | ``static``, except for the module's initialization function, in order to |
1084 | avoid name clashes with other extension modules (as discussed in section | |
1085 | :ref:`methodtable`). And it means that symbols that *should* be accessible from | |
1086 | other extension modules must be exported in a different way. | |
1087 | ||
1088 | Python provides a special mechanism to pass C-level information (pointers) from | |
1089 | one extension module to another one: CObjects. A CObject is a Python data type | |
1090 | which stores a pointer (:ctype:`void \*`). CObjects can only be created and | |
1091 | accessed via their C API, but they can be passed around like any other Python | |
1103 | the writer of the exporting module, which is appropriate for commonly used | |
1104 | library modules. It stores all C API pointers (just one in the example!) in an | |
1105 | array of :ctype:`void` pointers which becomes the value of a CObject. The header | |
1106 | file corresponding to the module provides a macro that takes care of importing | |
1107 | the module and retrieving its C API pointers; client modules only have to call | |
1108 | this macro before accessing the C API. | |
1109 | ||
1110 | The exporting module is a modification of the :mod:`spam` module from section | |
n | 1111- | :ref:`simpleexample`. The function :func:`spam.system` does not call the C |
n | 1119+ | :ref:`extending-simpleexample`. The function :func:`spam.system` does not call |
1112- | library function :cfunc:`system` directly, but a function | |
1120+ | the C library function :cfunc:`system` directly, but a function | |
1113 | :cfunc:`PySpam_System`, which would of course do something more complicated in | |
1114 | reality (such as adding "spam" to every command). This function | |
1115 | :cfunc:`PySpam_System` is also exported to other extension modules. | |
1116 | ||
1117 | The function :cfunc:`PySpam_System` is a plain C function, declared | |
n | 1118- | :keyword:`static` like everything else:: |
n | 1126+ | ``static`` like everything else:: |
1119 | ||
1120 | static int | |
1121 | PySpam_System(const char *command) | |
1122 | { | |
1123 | return system(command); | |
1124 | } | |
1125 | ||
1126 | The function :cfunc:`spam_system` is modified in a trivial way:: | |
1153 | PyMODINIT_FUNC | |
1154 | initspam(void) | |
1155 | { | |
1156 | PyObject *m; | |
1157 | static void *PySpam_API[PySpam_API_pointers]; | |
1158 | PyObject *c_api_object; | |
1159 | ||
1160 | m = Py_InitModule("spam", SpamMethods); | |
n | 1169+ | if (m == NULL) |
1170+ | return; | |
1161 | ||
1162 | /* Initialize the C API pointer array */ | |
1163 | PySpam_API[PySpam_System_NUM] = (void *)PySpam_System; | |
1164 | ||
1165 | /* Create a CObject containing the API pointer array's address */ | |
1166 | c_api_object = PyCObject_FromVoidPtr((void *)PySpam_API, NULL); | |
1167 | ||
1168 | if (c_api_object != NULL) | |
1169 | PyModule_AddObject(m, "_C_API", c_api_object); | |
1170 | } | |
1171 | ||
n | 1172- | Note that ``PySpam_API`` is declared :keyword:`static`; otherwise the pointer |
n | 1182+ | Note that ``PySpam_API`` is declared ``static``; otherwise the pointer |
1173 | array would disappear when :func:`initspam` terminates! | |
1174 | ||
1175 | The bulk of the work is in the header file :file:`spammodule.h`, which looks | |
1176 | like this:: | |
1177 | ||
1178 | #ifndef Py_SPAMMODULE_H | |
1179 | #define Py_SPAMMODULE_H | |
1180 | #ifdef __cplusplus | |
1234 | :cfunc:`PySpam_System` is to call the function (or rather macro) | |
1235 | :cfunc:`import_spam` in its initialization function:: | |
1236 | ||
1237 | PyMODINIT_FUNC | |
1238 | initclient(void) | |
1239 | { | |
1240 | PyObject *m; | |
1241 | ||
n | 1242- | Py_InitModule("client", ClientMethods); |
n | 1252+ | m = Py_InitModule("client", ClientMethods); |
1253+ | if (m == NULL) | |
1254+ | return; | |
1243 | if (import_spam() < 0) | |
1244 | return; | |
1245 | /* additional initialization can happen here */ | |
1246 | } | |
1247 | ||
1248 | The main disadvantage of this approach is that the file :file:`spammodule.h` is | |
1249 | rather complicated. However, the basic structure is the same for each function | |
1250 | that is exported, so it has to be learned only once. | |
1251 | ||
1252 | Finally it should be mentioned that CObjects offer additional functionality, | |
1253 | which is especially useful for memory allocation and deallocation of the pointer | |
1254 | stored in a CObject. The details are described in the Python/C API Reference | |
t | 1255- | Manual (XXX reference: ../api/api.html) in the section "CObjects (XXX reference: |
t | 1267+ | Manual in the section :ref:`cobjects` and in the implementation of CObjects (files |
1256- | ../api/cObjects.html)" and in the implementation of CObjects (files | |
1257 | :file:`Include/cobject.h` and :file:`Objects/cobject.c` in the Python source | |
1258 | code distribution). | |
1259 | ||
1260 | .. rubric:: Footnotes | |
1261 | ||
1262 | .. [#] An interface for this function already exists in the standard module :mod:`os` | |
1263 | --- it was chosen as a simple and straightforward example. | |
1264 |
Legends | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|
|
|