rest25/library/pickle.rst => rest262/library/pickle.rst | ||
---|---|---|
n | 1- | |
2 | :mod:`pickle` --- Python object serialization | |
3 | ============================================= | |
4 | ||
5 | .. index:: | |
6 | single: persistence | |
7 | pair: persistent; objects | |
8 | pair: serializing; objects | |
9 | pair: marshalling; objects | |
10 | pair: flattening; objects | |
11 | pair: pickling; objects | |
12 | ||
13 | .. module:: pickle | |
14 | :synopsis: Convert Python objects to streams of bytes and back. | |
n | 15- | |
n | 14+ | .. sectionauthor:: Jim Kerr <jbkerr@sr.hp.com>. |
16- | ||
15+ | .. sectionauthor:: Barry Warsaw <barry@zope.com> | |
17- | .. % Substantial improvements by Jim Kerr <jbkerr@sr.hp.com>. | |
18- | .. % Rewritten by Barry Warsaw <barry@zope.com> | |
19 | ||
20 | The :mod:`pickle` module implements a fundamental, but powerful algorithm for | |
21 | serializing and de-serializing a Python object structure. "Pickling" is the | |
22 | process whereby a Python object hierarchy is converted into a byte stream, and | |
23 | "unpickling" is the inverse operation, whereby a byte stream is converted back | |
24 | into an object hierarchy. Pickling (and unpickling) is alternatively known as | |
25 | "serialization", "marshalling," [#]_ or "flattening", however, to avoid | |
26 | confusion, the terms used here are "pickling" and "unpickling". | |
86 | Note that serialization is a more primitive notion than persistence; although | |
87 | :mod:`pickle` reads and writes file objects, it does not handle the issue of | |
88 | naming persistent objects, nor the (even more complicated) issue of concurrent | |
89 | access to persistent objects. The :mod:`pickle` module can transform a complex | |
90 | object into a byte stream and it can transform the byte stream into an object | |
91 | with the same internal structure. Perhaps the most obvious thing to do with | |
92 | these byte streams is to write them onto a file, but it is also conceivable to | |
93 | send them across a network or store them in a database. The module | |
n | 94- | :mod:`shelve` provides a simple interface to pickle and unpickle objects on DBM- |
n | 91+ | :mod:`shelve` provides a simple interface to pickle and unpickle objects on |
95- | style database files. | |
92+ | DBM-style database files. | |
96 | ||
97 | ||
98 | Data stream format | |
99 | ------------------ | |
100 | ||
101 | .. index:: | |
102 | single: XDR | |
103 | single: External Data Representation | |
117 | ||
118 | * Protocol version 0 is the original ASCII protocol and is backwards compatible | |
119 | with earlier versions of Python. | |
120 | ||
121 | * Protocol version 1 is the old binary format which is also compatible with | |
122 | earlier versions of Python. | |
123 | ||
124 | * Protocol version 2 was introduced in Python 2.3. It provides much more | |
n | 125- | efficient pickling of new-style classes. |
n | 122+ | efficient pickling of :term:`new-style class`\es. |
126 | ||
n | 127- | Refer to PEP 307 for more information. |
n | 124+ | Refer to :pep:`307` for more information. |
128 | ||
129 | If a *protocol* is not specified, protocol 0 is used. If *protocol* is specified | |
130 | as a negative value or :const:`HIGHEST_PROTOCOL`, the highest protocol version | |
131 | available will be used. | |
132 | ||
133 | .. versionchanged:: 2.3 | |
134 | Introduced the *protocol* parameter. | |
135 | ||
244 | :class:`Unpickler`: | |
245 | ||
246 | ||
247 | .. class:: Pickler(file[, protocol]) | |
248 | ||
249 | This takes a file-like object to which it will write a pickle data stream. | |
250 | ||
251 | If the *protocol* parameter is omitted, protocol 0 is used. If *protocol* is | |
n | 252- | specified as a negative value, the highest protocol version will be used. |
n | 249+ | specified as a negative value or :const:`HIGHEST_PROTOCOL`, the highest |
250+ | protocol version will be used. | |
253 | ||
254 | .. versionchanged:: 2.3 | |
255 | Introduced the *protocol* parameter. | |
256 | ||
257 | *file* must have a :meth:`write` method that accepts a single string argument. | |
258 | It can thus be an open file object, a :mod:`StringIO` object, or any other | |
259 | custom object that meets this interface. | |
260 | ||
n | 261- | :class:`Pickler` objects define one (or two) public methods: |
n | 259+ | :class:`Pickler` objects define one (or two) public methods: |
262 | ||
263 | ||
n | 264- | .. method:: Pickler.dump(obj) |
n | 262+ | .. method:: dump(obj) |
265 | ||
n | 266- | Write a pickled representation of *obj* to the open file object given in the |
n | 264+ | Write a pickled representation of *obj* to the open file object given in the |
267- | constructor. Either the binary or ASCII format will be used, depending on the | |
265+ | constructor. Either the binary or ASCII format will be used, depending on the | |
268- | value of the *protocol* argument passed to the constructor. | |
266+ | value of the *protocol* argument passed to the constructor. | |
269 | ||
270 | ||
n | 271- | .. method:: Pickler.clear_memo() |
n | 269+ | .. method:: clear_memo() |
272 | ||
n | 273- | Clears the pickler's "memo". The memo is the data structure that remembers |
n | 271+ | Clears the pickler's "memo". The memo is the data structure that remembers |
274- | which objects the pickler has already seen, so that shared or recursive objects | |
272+ | which objects the pickler has already seen, so that shared or recursive objects | |
275- | pickled by reference and not by value. This method is useful when re-using | |
273+ | pickled by reference and not by value. This method is useful when re-using | |
276- | picklers. | |
274+ | picklers. | |
277 | ||
n | 278- | .. note:: |
n | 276+ | .. note:: |
279 | ||
n | 280- | Prior to Python 2.3, :meth:`clear_memo` was only available on the picklers |
n | 278+ | Prior to Python 2.3, :meth:`clear_memo` was only available on the picklers |
281- | created by :mod:`cPickle`. In the :mod:`pickle` module, picklers have an | |
279+ | created by :mod:`cPickle`. In the :mod:`pickle` module, picklers have an | |
282- | instance variable called :attr:`memo` which is a Python dictionary. So to clear | |
280+ | instance variable called :attr:`memo` which is a Python dictionary. So to clear | |
283- | the memo for a :mod:`pickle` module pickler, you could do the following:: | |
281+ | the memo for a :mod:`pickle` module pickler, you could do the following:: | |
284 | ||
n | 285- | mypickler.memo.clear() |
n | 283+ | mypickler.memo.clear() |
286 | ||
n | 287- | Code that does not need to support older versions of Python should simply use |
n | 285+ | Code that does not need to support older versions of Python should simply use |
288- | :meth:`clear_memo`. | |
286+ | :meth:`clear_memo`. | |
289 | ||
290 | It is possible to make multiple calls to the :meth:`dump` method of the same | |
291 | :class:`Pickler` instance. These must then be matched to the same number of | |
292 | calls to the :meth:`load` method of the corresponding :class:`Unpickler` | |
293 | instance. If the same object is pickled by multiple :meth:`dump` calls, the | |
294 | :meth:`load` will all yield references to the same object. [#]_ | |
295 | ||
296 | :class:`Unpickler` objects are defined as: | |
304 | factory. | |
305 | ||
306 | *file* must have two methods, a :meth:`read` method that takes an integer | |
307 | argument, and a :meth:`readline` method that requires no arguments. Both | |
308 | methods should return a string. Thus *file* can be a file object opened for | |
309 | reading, a :mod:`StringIO` object, or any other custom object that meets this | |
310 | interface. | |
311 | ||
n | 312- | :class:`Unpickler` objects have one (or two) public methods: |
n | 310+ | :class:`Unpickler` objects have one (or two) public methods: |
313 | ||
314 | ||
n | 315- | .. method:: Unpickler.load() |
n | 313+ | .. method:: load() |
316 | ||
n | 317- | Read a pickled object representation from the open file object given in the |
n | 315+ | Read a pickled object representation from the open file object given in |
318- | constructor, and return the reconstituted object hierarchy specified therein. | |
316+ | the constructor, and return the reconstituted object hierarchy specified | |
317+ | therein. | |
319 | ||
n | 319+ | This method automatically determines whether the data stream was written |
320+ | in binary mode or not. | |
320 | ||
n | 322+ | |
321- | .. method:: Unpickler.noload() | |
323+ | .. method:: noload() | |
322 | ||
n | 323- | This is just like :meth:`load` except that it doesn't actually create any |
n | 325+ | This is just like :meth:`load` except that it doesn't actually create any |
324- | objects. This is useful primarily for finding what's called "persistent ids" | |
326+ | objects. This is useful primarily for finding what's called "persistent | |
325- | that may be referenced in a pickle data stream. See section | |
327+ | ids" that may be referenced in a pickle data stream. See section | |
326- | :ref:`pickle-protocol` below for more details. | |
328+ | :ref:`pickle-protocol` below for more details. | |
327 | ||
n | 328- | **Note:** the :meth:`noload` method is currently only available on |
n | 330+ | **Note:** the :meth:`noload` method is currently only available on |
329- | :class:`Unpickler` objects created with the :mod:`cPickle` module. | |
331+ | :class:`Unpickler` objects created with the :mod:`cPickle` module. | |
330- | :mod:`pickle` module :class:`Unpickler`\ s do not have the :meth:`noload` | |
332+ | :mod:`pickle` module :class:`Unpickler`\ s do not have the :meth:`noload` | |
331- | method. | |
333+ | method. | |
332 | ||
333 | ||
334 | What can be pickled and unpickled? | |
335 | ---------------------------------- | |
336 | ||
337 | The following types can be pickled: | |
338 | ||
339 | * ``None``, ``True``, and ``False`` | |
389 | conversions can be made by the class's :meth:`__setstate__` method. | |
390 | ||
391 | ||
392 | .. _pickle-protocol: | |
393 | ||
394 | The pickle protocol | |
395 | ------------------- | |
396 | ||
n | 399+ | .. currentmodule:: None |
400+ | ||
397 | This section describes the "pickling protocol" that defines the interface | |
398 | between the pickler/unpickler and the objects that are being serialized. This | |
399 | protocol provides a standard way for you to define, customize, and control how | |
400 | your objects are serialized and de-serialized. The description in this section | |
401 | doesn't cover specific customizations that you can employ to make the unpickling | |
402 | environment slightly safer from untrusted pickle data streams; see section | |
403 | :ref:`pickle-sub` for more details. | |
404 | ||
405 | ||
406 | .. _pickle-inst: | |
407 | ||
408 | Pickling and unpickling normal class instances | |
409 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
410 | ||
n | 411- | .. index:: |
n | 415+ | .. method:: object.__getinitargs__() |
412- | single: __getinitargs__() (copy protocol) | |
413- | single: __init__() (instance constructor) | |
414 | ||
n | 415- | When a pickled class instance is unpickled, its :meth:`__init__` method is |
n | 417+ | When a pickled class instance is unpickled, its :meth:`__init__` method is |
416- | normally *not* invoked. If it is desirable that the :meth:`__init__` method be | |
418+ | normally *not* invoked. If it is desirable that the :meth:`__init__` method | |
417- | called on unpickling, an old-style class can define a method | |
419+ | be called on unpickling, an old-style class can define a method | |
418- | :meth:`__getinitargs__`, which should return a *tuple* containing the arguments | |
420+ | :meth:`__getinitargs__`, which should return a *tuple* containing the | |
419- | to be passed to the class constructor (:meth:`__init__` for example). The | |
421+ | arguments to be passed to the class constructor (:meth:`__init__` for | |
420- | :meth:`__getinitargs__` method is called at pickle time; the tuple it returns is | |
422+ | example). The :meth:`__getinitargs__` method is called at pickle time; the | |
421- | incorporated in the pickle for the instance. | |
423+ | tuple it returns is incorporated in the pickle for the instance. | |
422 | ||
n | 423- | .. index:: single: __getnewargs__() (copy protocol) |
n | 425+ | .. method:: object.__getnewargs__() |
424 | ||
n | 425- | New-style types can provide a :meth:`__getnewargs__` method that is used for |
n | 427+ | New-style types can provide a :meth:`__getnewargs__` method that is used for |
426- | protocol 2. Implementing this method is needed if the type establishes some | |
428+ | protocol 2. Implementing this method is needed if the type establishes some | |
427- | internal invariants when the instance is created, or if the memory allocation is | |
429+ | internal invariants when the instance is created, or if the memory allocation | |
428- | affected by the values passed to the :meth:`__new__` method for the type (as it | |
430+ | is affected by the values passed to the :meth:`__new__` method for the type | |
429- | is for tuples and strings). Instances of a new-style type :class:`C` are | |
431+ | (as it is for tuples and strings). Instances of a :term:`new-style class` | |
430- | created using :: | |
432+ | ``C`` are created using :: | |
431 | ||
n | 432- | obj = C.__new__(C, \*args) |
n | 434+ | obj = C.__new__(C, *args) |
433 | ||
n | 434- | |
435- | where *args* is the result of calling :meth:`__getnewargs__` on the original | |
436+ | where *args* is the result of calling :meth:`__getnewargs__` on the original | |
436- | object; if there is no :meth:`__getnewargs__`, an empty tuple is assumed. | |
437+ | object; if there is no :meth:`__getnewargs__`, an empty tuple is assumed. | |
437 | ||
n | 438- | .. index:: |
n | 439+ | .. method:: object.__getstate__() |
439- | single: __getstate__() (copy protocol) | |
440- | single: __setstate__() (copy protocol) | |
441- | single: __dict__ (instance attribute) | |
442 | ||
n | 443- | Classes can further influence how their instances are pickled; if the class |
n | 441+ | Classes can further influence how their instances are pickled; if the class |
444- | defines the method :meth:`__getstate__`, it is called and the return state is | |
442+ | defines the method :meth:`__getstate__`, it is called and the return state is | |
445- | pickled as the contents for the instance, instead of the contents of the | |
443+ | pickled as the contents for the instance, instead of the contents of the | |
446- | instance's dictionary. If there is no :meth:`__getstate__` method, the | |
444+ | instance's dictionary. If there is no :meth:`__getstate__` method, the | |
447- | instance's :attr:`__dict__` is pickled. | |
445+ | instance's :attr:`__dict__` is pickled. | |
448 | ||
n | 447+ | .. method:: object.__setstate__() |
448+ | ||
449- | Upon unpickling, if the class also defines the method :meth:`__setstate__`, it | |
449+ | Upon unpickling, if the class also defines the method :meth:`__setstate__`, | |
450- | is called with the unpickled state. [#]_ If there is no :meth:`__setstate__` | |
450+ | it is called with the unpickled state. [#]_ If there is no | |
451- | method, the pickled state must be a dictionary and its items are assigned to the | |
451+ | :meth:`__setstate__` method, the pickled state must be a dictionary and its | |
452- | new instance's dictionary. If a class defines both :meth:`__getstate__` and | |
452+ | items are assigned to the new instance's dictionary. If a class defines both | |
453- | :meth:`__setstate__`, the state object needn't be a dictionary and these methods | |
453+ | :meth:`__getstate__` and :meth:`__setstate__`, the state object needn't be a | |
454- | can do what they want. [#]_ | |
454+ | dictionary and these methods can do what they want. [#]_ | |
455 | ||
n | 456- | .. warning:: |
n | 456+ | .. warning:: |
457 | ||
n | 458- | For new-style classes, if :meth:`__getstate__` returns a false value, the |
n | 458+ | For :term:`new-style class`\es, if :meth:`__getstate__` returns a false |
459- | :meth:`__setstate__` method will not be called. | |
459+ | value, the :meth:`__setstate__` method will not be called. | |
460+ | ||
461+ | .. note:: | |
462+ | ||
463+ | At unpickling time, some methods like :meth:`__getattr__`, | |
464+ | :meth:`__getattribute__`, or :meth:`__setattr__` may be called upon the | |
465+ | instance. In case those methods rely on some internal invariant being | |
466+ | true, the type should implement either :meth:`__getinitargs__` or | |
467+ | :meth:`__getnewargs__` to establish such an invariant; otherwise, neither | |
468+ | :meth:`__new__` nor :meth:`__init__` will be called. | |
460 | ||
461 | ||
462 | Pickling and unpickling extension types | |
463 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
464 | ||
n | 474+ | .. method:: object.__reduce__() |
475+ | ||
465- | When the :class:`Pickler` encounters an object of a type it knows nothing about | |
476+ | When the :class:`Pickler` encounters an object of a type it knows nothing | |
466- | --- such as an extension type --- it looks in two places for a hint of how to | |
477+ | about --- such as an extension type --- it looks in two places for a hint of | |
467- | pickle it. One alternative is for the object to implement a :meth:`__reduce__` | |
478+ | how to pickle it. One alternative is for the object to implement a | |
468- | method. If provided, at pickling time :meth:`__reduce__` will be called with no | |
479+ | :meth:`__reduce__` method. If provided, at pickling time :meth:`__reduce__` | |
469- | arguments, and it must return either a string or a tuple. | |
480+ | will be called with no arguments, and it must return either a string or a | |
481+ | tuple. | |
470 | ||
n | 471- | If a string is returned, it names a global variable whose contents are pickled |
n | 483+ | If a string is returned, it names a global variable whose contents are |
472- | as normal. The string returned by :meth:`__reduce__` should be the object's | |
484+ | pickled as normal. The string returned by :meth:`__reduce__` should be the | |
473- | local name relative to its module; the pickle module searches the module | |
485+ | object's local name relative to its module; the pickle module searches the | |
474- | namespace to determine the object's module. | |
486+ | module namespace to determine the object's module. | |
475 | ||
n | 476- | When a tuple is returned, it must be between two and five elements long. |
n | 488+ | When a tuple is returned, it must be between two and five elements long. |
477- | Optional elements can either be omitted, or ``None`` can be provided as their | |
489+ | Optional elements can either be omitted, or ``None`` can be provided as their | |
478- | value. The semantics of each element are: | |
490+ | value. The contents of this tuple are pickled as normal and used to | |
491+ | reconstruct the object at unpickling time. The semantics of each element | |
492+ | are: | |
479 | ||
n | 480- | * A callable object that will be called to create the initial version of the |
n | 494+ | * A callable object that will be called to create the initial version of the |
481- | object. The next element of the tuple will provide arguments for this callable, | |
495+ | object. The next element of the tuple will provide arguments for this | |
482- | and later elements provide additional state information that will subsequently | |
496+ | callable, and later elements provide additional state information that will | |
483- | be used to fully reconstruct the pickled date. | |
497+ | subsequently be used to fully reconstruct the pickled data. | |
484 | ||
n | 485- | In the unpickling environment this object must be either a class, a callable |
n | 499+ | In the unpickling environment this object must be either a class, a |
486- | registered as a "safe constructor" (see below), or it must have an attribute | |
500+ | callable registered as a "safe constructor" (see below), or it must have an | |
487- | :attr:`__safe_for_unpickling__` with a true value. Otherwise, an | |
501+ | attribute :attr:`__safe_for_unpickling__` with a true value. Otherwise, an | |
488- | :exc:`UnpicklingError` will be raised in the unpickling environment. Note that | |
502+ | :exc:`UnpicklingError` will be raised in the unpickling environment. Note | |
489- | as usual, the callable itself is pickled by name. | |
503+ | that as usual, the callable itself is pickled by name. | |
490 | ||
n | 491- | * A tuple of arguments for the callable object. |
n | 505+ | * A tuple of arguments for the callable object. |
492 | ||
n | 493- | .. versionchanged:: 2.5 |
n | 507+ | .. versionchanged:: 2.5 |
494- | Formerly, this argument could also be ``None``. | |
508+ | Formerly, this argument could also be ``None``. | |
495 | ||
n | 496- | * Optionally, the object's state, which will be passed to the object's |
n | 510+ | * Optionally, the object's state, which will be passed to the object's |
497- | :meth:`__setstate__` method as described in section :ref:`pickle-inst`. If the | |
511+ | :meth:`__setstate__` method as described in section :ref:`pickle-inst`. If | |
498- | object has no :meth:`__setstate__` method, then, as above, the value must be a | |
512+ | the object has no :meth:`__setstate__` method, then, as above, the value | |
499- | dictionary and it will be added to the object's :attr:`__dict__`. | |
513+ | must be a dictionary and it will be added to the object's :attr:`__dict__`. | |
500 | ||
n | 501- | * Optionally, an iterator (and not a sequence) yielding successive list items. |
n | 515+ | * Optionally, an iterator (and not a sequence) yielding successive list |
502- | These list items will be pickled, and appended to the object using either | |
516+ | items. These list items will be pickled, and appended to the object using | |
503- | ``obj.append(item)`` or ``obj.extend(list_of_items)``. This is primarily used | |
517+ | either ``obj.append(item)`` or ``obj.extend(list_of_items)``. This is | |
504- | for list subclasses, but may be used by other classes as long as they have | |
518+ | primarily used for list subclasses, but may be used by other classes as | |
505- | :meth:`append` and :meth:`extend` methods with the appropriate signature. | |
519+ | long as they have :meth:`append` and :meth:`extend` methods with the | |
506- | (Whether :meth:`append` or :meth:`extend` is used depends on which pickle | |
520+ | appropriate signature. (Whether :meth:`append` or :meth:`extend` is used | |
507- | protocol version is used as well as the number of items to append, so both must | |
521+ | depends on which pickle protocol version is used as well as the number of | |
508- | be supported.) | |
522+ | items to append, so both must be supported.) | |
509 | ||
n | 510- | * Optionally, an iterator (not a sequence) yielding successive dictionary items, |
n | 524+ | * Optionally, an iterator (not a sequence) yielding successive dictionary |
511- | which should be tuples of the form ``(key, value)``. These items will be | |
525+ | items, which should be tuples of the form ``(key, value)``. These items | |
512- | pickled and stored to the object using ``obj[key] = value``. This is primarily | |
526+ | will be pickled and stored to the object using ``obj[key] = value``. This | |
513- | used for dictionary subclasses, but may be used by other classes as long as they | |
527+ | is primarily used for dictionary subclasses, but may be used by other | |
514- | implement :meth:`__setitem__`. | |
528+ | classes as long as they implement :meth:`__setitem__`. | |
515 | ||
n | 530+ | .. method:: object.__reduce_ex__(protocol) |
531+ | ||
516- | It is sometimes useful to know the protocol version when implementing | |
532+ | It is sometimes useful to know the protocol version when implementing | |
517- | :meth:`__reduce__`. This can be done by implementing a method named | |
533+ | :meth:`__reduce__`. This can be done by implementing a method named | |
518- | :meth:`__reduce_ex__` instead of :meth:`__reduce__`. :meth:`__reduce_ex__`, when | |
534+ | :meth:`__reduce_ex__` instead of :meth:`__reduce__`. :meth:`__reduce_ex__`, | |
519- | it exists, is called in preference over :meth:`__reduce__` (you may still | |
535+ | when it exists, is called in preference over :meth:`__reduce__` (you may | |
520- | provide :meth:`__reduce__` for backwards compatibility). The | |
536+ | still provide :meth:`__reduce__` for backwards compatibility). The | |
521- | :meth:`__reduce_ex__` method will be called with a single integer argument, the | |
537+ | :meth:`__reduce_ex__` method will be called with a single integer argument, | |
522- | protocol version. | |
538+ | the protocol version. | |
523 | ||
n | 524- | The :class:`object` class implements both :meth:`__reduce__` and |
n | 540+ | The :class:`object` class implements both :meth:`__reduce__` and |
525- | :meth:`__reduce_ex__`; however, if a subclass overrides :meth:`__reduce__` but | |
541+ | :meth:`__reduce_ex__`; however, if a subclass overrides :meth:`__reduce__` | |
526- | not :meth:`__reduce_ex__`, the :meth:`__reduce_ex__` implementation detects this | |
542+ | but not :meth:`__reduce_ex__`, the :meth:`__reduce_ex__` implementation | |
527- | and calls :meth:`__reduce__`. | |
543+ | detects this and calls :meth:`__reduce__`. | |
528 | ||
529 | An alternative to implementing a :meth:`__reduce__` method on the object to be | |
530 | pickled, is to register the callable with the :mod:`copy_reg` module. This | |
531 | module provides a way for programs to register "reduction functions" and | |
532 | constructors for user-defined types. Reduction functions have the same | |
533 | semantics and interface as the :meth:`__reduce__` method described above, except | |
534 | that they are called with a single argument, the object to be pickled. | |
535 | ||
536 | The registered constructor is deemed a "safe constructor" for purposes of | |
537 | unpickling as described above. | |
538 | ||
539 | ||
540 | Pickling and unpickling external objects | |
541 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
n | 558+ | |
559+ | .. index:: | |
560+ | single: persistent_id (pickle protocol) | |
561+ | single: persistent_load (pickle protocol) | |
542 | ||
543 | For the benefit of object persistence, the :mod:`pickle` module supports the | |
544 | notion of a reference to an object outside the pickled data stream. Such | |
545 | objects are referenced by a "persistent id", which is just an arbitrary string | |
546 | of printable ASCII characters. The resolution of such names is not defined by | |
547 | the :mod:`pickle` module; it will delegate this resolution to user defined | |
548 | functions on the pickler and unpickler. [#]_ | |
549 | ||
613 | In the :mod:`cPickle` module, the unpickler's :attr:`persistent_load` attribute | |
614 | can also be set to a Python list, in which case, when the unpickler reaches a | |
615 | persistent id, the persistent id string will simply be appended to this list. | |
616 | This functionality exists so that a pickle data stream can be "sniffed" for | |
617 | object references without actually instantiating all the objects in a pickle. | |
618 | [#]_ Setting :attr:`persistent_load` to a list is usually used in conjunction | |
619 | with the :meth:`noload` method on the Unpickler. | |
620 | ||
n | 621- | .. % BAW: Both pickle and cPickle support something called |
n | 641+ | .. BAW: Both pickle and cPickle support something called inst_persistent_id() |
622- | .. % inst_persistent_id() which appears to give unknown types a second | |
642+ | which appears to give unknown types a second shot at producing a persistent | |
623- | .. % shot at producing a persistent id. Since Jim Fulton can't remember | |
643+ | id. Since Jim Fulton can't remember why it was added or what it's for, I'm | |
624- | .. % why it was added or what it's for, I'm leaving it undocumented. | |
644+ | leaving it undocumented. | |
625 | ||
626 | ||
627 | .. _pickle-sub: | |
628 | ||
629 | Subclassing Unpicklers | |
630 | ---------------------- | |
n | 651+ | |
652+ | .. index:: | |
653+ | single: load_global() (pickle protocol) | |
654+ | single: find_global() (pickle protocol) | |
631 | ||
632 | By default, unpickling will import any class that it finds in the pickle data. | |
633 | You can control exactly what gets unpickled and what gets called by customizing | |
634 | your unpickler. Unfortunately, exactly how you do this is different depending | |
635 | on whether you're using :mod:`pickle` or :mod:`cPickle`. [#]_ | |
636 | ||
637 | In the :mod:`pickle` module, you need to derive a subclass from | |
638 | :class:`Unpickler`, overriding the :meth:`load_global` method. | |
684 | # Pickle dictionary using protocol 0. | |
685 | pickle.dump(data1, output) | |
686 | ||
687 | # Pickle the list using the highest protocol available. | |
688 | pickle.dump(selfref_list, output, -1) | |
689 | ||
690 | output.close() | |
691 | ||
n | 692- | The following example reads the resulting pickled data. When reading a pickle- |
n | 716+ | The following example reads the resulting pickled data. When reading a |
693- | containing file, you should open the file in binary mode because you can't be | |
717+ | pickle-containing file, you should open the file in binary mode because you | |
694- | sure if the ASCII or binary format was used. :: | |
718+ | can't be sure if the ASCII or binary format was used. :: | |
695 | ||
696 | import pprint, pickle | |
697 | ||
698 | pkl_file = open('data.pkl', 'rb') | |
699 | ||
700 | data1 = pickle.load(pkl_file) | |
701 | pprint.pprint(data1) | |
702 | ||
707 | ||
708 | Here's a larger example that shows how to modify pickling behavior for a class. | |
709 | The :class:`TextReader` class opens a text file, and returns the line number and | |
710 | line contents each time its :meth:`readline` method is called. If a | |
711 | :class:`TextReader` instance is pickled, all attributes *except* the file object | |
712 | member are saved. When the instance is unpickled, the file is reopened, and | |
713 | reading resumes from the last location. The :meth:`__setstate__` and | |
714 | :meth:`__getstate__` methods are used to implement this behavior. :: | |
n | 739+ | |
740+ | #!/usr/local/bin/python | |
715 | ||
716 | class TextReader: | |
717 | """Print and number lines in a text file.""" | |
718 | def __init__(self, file): | |
719 | self.file = file | |
720 | self.fh = open(file) | |
721 | self.lineno = 0 | |
722 | ||
729 | line = line[:-1] | |
730 | return "%d: %s" % (self.lineno, line) | |
731 | ||
732 | def __getstate__(self): | |
733 | odict = self.__dict__.copy() # copy the dict since we change it | |
734 | del odict['fh'] # remove filehandle entry | |
735 | return odict | |
736 | ||
n | 737- | def __setstate__(self,dict): |
n | 763+ | def __setstate__(self, dict): |
738 | fh = open(dict['file']) # reopen file | |
739 | count = dict['lineno'] # read from file... | |
740 | while count: # until line count is restored | |
741 | fh.readline() | |
742 | count = count - 1 | |
743 | self.__dict__.update(dict) # update attributes | |
744 | self.fh = fh # save the file object | |
745 | ||
746 | A sample usage might be something like this:: | |
747 | ||
748 | >>> import TextReader | |
749 | >>> obj = TextReader.TextReader("TextReader.py") | |
750 | >>> obj.readline() | |
751 | '1: #!/usr/local/bin/python' | |
n | 752- | >>> # (more invocations of obj.readline() here) |
753- | ... obj.readline() | |
778+ | >>> obj.readline() | |
779+ | '2: ' | |
780+ | >>> obj.readline() | |
754- | '7: class TextReader:' | |
781+ | '3: class TextReader:' | |
755 | >>> import pickle | |
n | 756- | >>> pickle.dump(obj,open('save.p','w')) |
n | 783+ | >>> pickle.dump(obj, open('save.p', 'wb')) |
757 | ||
758 | If you want to see that :mod:`pickle` works across Python processes, start | |
759 | another Python session, before continuing. What follows can happen from either | |
760 | the same process or a new process. :: | |
761 | ||
762 | >>> import pickle | |
n | 763- | >>> reader = pickle.load(open('save.p')) |
n | 790+ | >>> reader = pickle.load(open('save.p', 'rb')) |
764 | >>> reader.readline() | |
t | 765- | '8: "Print and number lines in a text file."' |
t | 792+ | '4: """Print and number lines in a text file."""' |
766 | ||
767 | ||
768 | .. seealso:: | |
769 | ||
770 | Module :mod:`copy_reg` | |
771 | Pickle interface constructor registration for extension types. | |
772 | ||
773 | Module :mod:`shelve` |
Legends | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|
|
|