f | **************************** |
n | What's New in Python 2.5 |
n | What's New in Python 2.5 |
| **************************** |
| |
| :Author: A.M. Kuchling |
| |
n | .. |release| replace:: 1.0 |
n | .. |release| replace:: 1.01 |
| |
n | .. % $Id: whatsnew25.tex 51743 2006-09-05 13:11:33Z andrew.kuchling $ |
n | .. $Id: whatsnew25.tex 56611 2007-07-29 08:26:10Z georg.brandl $ |
| .. % Fix XXX comments |
| .. Fix XXX comments |
| |
| This article explains the new features in Python 2.5. The final release of |
| Python 2.5 is scheduled for August 2006; :pep:`356` describes the planned |
| release schedule. |
| |
| The changes in Python 2.5 are an interesting mix of language and library |
| improvements. The library enhancements will be more important to Python's user |
| community, I think, because several widely-useful packages were added. New |
n | modules include ElementTree for XML processing (section :ref:`module-etree`), |
n | modules include ElementTree for XML processing (:mod:`xml.etree`), |
| the SQLite database module (section :ref:`module-sqlite`), and the :mod:`ctypes` |
| the SQLite database module (:mod:`sqlite`), and the :mod:`ctypes` |
| module for calling C functions (section :ref:`module-ctypes`). |
| module for calling C functions. |
| |
| The language changes are of middling significance. Some pleasant new features |
| were added, but most of them aren't features that you'll use every day. |
| Conditional expressions were finally added to the language using a novel syntax; |
| see section :ref:`pep-308`. The new ':keyword:`with`' statement will make |
| writing cleanup code easier (section :ref:`pep-343`). Values can now be passed |
| into generators (section :ref:`pep-342`). Imports are now visible as either |
| absolute or relative (section :ref:`pep-328`). Some corner cases of exception |
| As well as the language and library additions, other improvements and bugfixes |
| were made throughout the source tree. A search through the SVN change logs |
| finds there were 353 patches applied and 458 bugs fixed between Python 2.4 and |
| 2.5. (Both figures are likely to be underestimates.) |
| |
| This article doesn't try to be a complete specification of the new features; |
| instead changes are briefly introduced using helpful examples. For full |
| details, you should always refer to the documentation for Python 2.5 at |
n | `<http://docs.python.org>`_. If you want to understand the complete |
n | http://docs.python.org. If you want to understand the complete implementation |
| implementation and design rationale, refer to the PEP for a particular new |
| and design rationale, refer to the PEP for a particular new feature. |
| feature. |
| |
| Comments, suggestions, and error reports for this document are welcome; please |
| e-mail them to the author or open a bug in the Python bug tracker. |
| |
n | .. % ====================================================================== |
n | .. ====================================================================== |
| |
| |
| .. _pep-308: |
| |
| PEP 308: Conditional Expressions |
| ================================ |
| |
| For a long time, people have been requesting a way to write conditional |
| the dependency information will be recorded in the :file:`PKG-INFO` file. |
| |
| Another new keyword parameter is ``download_url``, which should be set to a URL |
| for the package's source code. This means it's now possible to look up an entry |
| in the package index, determine the dependencies for a package, and download the |
| required packages. :: |
| |
| VERSION = '1.0' |
n | setup(name='PyPackage', |
n | setup(name='PyPackage', |
| version=VERSION, |
| requires=['numarray', 'zlib (>=1.1.4)'], |
| obsoletes=['OldPackage'] |
| download_url=('http://www.example.com/pypackage/dist/pkg-%s.tar.gz' |
| % VERSION), |
| ) |
| |
| Another new enhancement to the Python package index at |
n | `<http://cheeseshop.python.org>`_ is storing source and binary archives for a |
n | http://cheeseshop.python.org is storing source and binary archives for a |
| package. The new :command:`upload` Distutils command will upload a package to |
| the repository. |
| |
| Before a package can be uploaded, you must be able to build a distribution using |
| the :command:`sdist` Distutils command. Once that works, you can run ``python |
| setup.py upload`` to add your package to the PyPI archive. Optionally you can |
| GPG-sign the package by supplying the :option:`--sign` and :option:`--identity` |
| options. |
| archive. |
| |
| |
| .. seealso:: |
| |
| :pep:`338` - Executing modules as scripts |
| PEP written and implemented by Nick Coghlan. |
| |
n | .. % ====================================================================== |
n | .. ====================================================================== |
| |
| |
| .. _pep-341: |
| |
| PEP 341: Unified try/except/finally |
| =================================== |
| |
| Until Python 2.5, the :keyword:`try` statement came in two flavours. You could |
| use a :keyword:`finally` block to ensure that code is always executed, or one or |
| more :keyword:`except` blocks to catch specific exceptions. You couldn't |
| combine both :keyword:`except` blocks and a :keyword:`finally` block, because |
| generating the right bytecode for the combined version was complicated and it |
n | wasn't clear what the semantics of the combined should be. |
n | wasn't clear what the semantics of the combined statement should be. |
| |
| Guido van Rossum spent some time working with Java, which does support the |
| equivalent of combining :keyword:`except` blocks and a :keyword:`finally` block, |
| and this clarified what the statement should mean. In Python 2.5, you can now |
| write:: |
| |
| try: |
| block-1 ... |
| except Exception1: |
| handler-1 ... |
| except Exception2: |
| handler-2 ... |
| else: |
| else-block |
| finally: |
n | final-block |
n | final-block |
| |
| The code in *block-1* is executed. If the code raises an exception, the various |
| :keyword:`except` blocks are tested: if the exception is of class |
| :class:`Exception1`, *handler-1* is executed; otherwise if it's of class |
| :class:`Exception2`, *handler-2* is executed, and so forth. If no exception is |
| raised, the *else-block* is executed. |
| |
| No matter what happened previously, the *final-block* is executed once the code |
| >>> print it.next() |
| 1 |
| >>> print it.send(8) |
| 8 |
| >>> print it.next() |
| 9 |
| >>> print it.next() |
| Traceback (most recent call last): |
n | File ``t.py'', line 15, in ? |
n | File "t.py", line 15, in ? |
| print it.next() |
| StopIteration |
| |
n | Because :keyword:`yield` will often be returning :const:`None`, you should |
n | :keyword:`yield` will usually return :const:`None`, so you should always check |
| always check for this case. Don't just use its value in expressions unless |
| for this case. Don't just use its value in expressions unless you're sure that |
| you're sure that the :meth:`send` method will be the only method used resume |
| the :meth:`send` method will be the only method used to resume your generator |
| your generator function. |
| function. |
| |
| In addition to :meth:`send`, there are two other new methods on generators: |
| |
| * :meth:`throw(type, value=None, traceback=None)` is used to raise an exception |
| inside the generator; the exception is raised by the :keyword:`yield` expression |
| where the generator's execution is paused. |
| |
| * :meth:`close` raises a new :exc:`GeneratorExit` exception inside the generator |
n | to terminate the iteration. On receiving this exception, the generator's code |
n | to terminate the iteration. On receiving this exception, the generator's code |
| must either raise :exc:`GeneratorExit` or :exc:`StopIteration`; catching the |
| must either raise :exc:`GeneratorExit` or :exc:`StopIteration`. Catching the |
| exception and doing anything else is illegal and will trigger a |
| :exc:`GeneratorExit` exception and returning a value is illegal and will trigger |
| :exc:`RuntimeError`. :meth:`close` will also be called by Python's garbage |
| a :exc:`RuntimeError`; if the function raises some other exception, that |
| exception is propagated to the caller. :meth:`close` will also be called by |
| collector when the generator is garbage-collected. |
| Python's garbage collector when the generator is garbage-collected. |
| |
| If you need to run cleanup code when a :exc:`GeneratorExit` occurs, I suggest |
| using a ``try: ... finally:`` suite instead of catching :exc:`GeneratorExit`. |
| |
| The cumulative effect of these changes is to turn generators from one-way |
| producers of information into both producers and consumers. |
| |
| Generators also become *coroutines*, a more generalized form of subroutines. |
| |
| The ':keyword:`with`' statement is a new control-flow structure whose basic |
| structure is:: |
| |
| with expression [as variable]: |
| with-block |
| |
| The expression is evaluated, and it should result in an object that supports the |
n | context management protocol. This object may return a value that can optionally |
n | context management protocol (that is, has :meth:`__enter__` and :meth:`__exit__` |
| methods. |
| |
| The object's :meth:`__enter__` is called before *with-block* is executed and |
| therefore can run set-up code. It also may return a value that is bound to the |
| be bound to the name *variable*. (Note carefully that *variable* is *not* |
| name *variable*, if given. (Note carefully that *variable* is *not* assigned |
| assigned the result of *expression*.) The object can then run set-up code |
| the result of *expression*.) |
| before *with-block* is executed and some clean-up code is executed after the |
| block is done, even if the block raised an exception. |
| |
n | After execution of the *with-block* is finished, the object's :meth:`__exit__` |
| method is called, even if the block raised an exception, and can therefore run |
| clean-up code. |
| |
| To enable the statement in Python 2.5, you need to add the following directive |
| To enable the statement in Python 2.5, you need to add the following directive |
| to your module:: |
| |
| from __future__ import with_statement |
| |
| The statement will always be enabled in Python 2.6. |
| |
| Some standard Python objects now support the context management protocol and can |
| be used with the ':keyword:`with`' statement. File objects are one example:: |
| |
| with open('/etc/passwd', 'r') as f: |
| for line in f: |
| print line |
| ... more processing code ... |
| |
| After this statement has executed, the file object in *f* will have been |
n | automatically closed, even if the 'for' loop raised an exception part-way |
n | automatically closed, even if the :keyword:`for` loop raised an exception part- |
| through the block. |
| way through the block. |
| |
| .. note:: |
| |
| In this case, *f* is the same object created by :func:`open`, because |
| :meth:`file.__enter__` returns *self*. |
| |
| The :mod:`threading` module's locks and condition variables also support the |
| ':keyword:`with`' statement:: |
| |
| lock = threading.Lock() |
| with lock: |
| # Critical section of code |
| ... |
| .. seealso:: |
| |
| :pep:`343` - The "with" statement |
| PEP written by Guido van Rossum and Nick Coghlan; implemented by Mike Bland, |
| Guido van Rossum, and Neal Norwitz. The PEP shows the code generated for a |
| ':keyword:`with`' statement, which can be helpful in learning how the statement |
| works. |
| |
n | ../lib/module-contextlib.html |
| The documentation for the :mod:`contextlib` module. |
| The documentation for the :mod:`contextlib` module. |
| |
n | .. % ====================================================================== |
n | .. ====================================================================== |
| |
| |
| .. _pep-352: |
| |
| PEP 352: Exceptions as New-Style Classes |
| ======================================== |
| |
| Exception classes can now be new-style classes, not just classic classes, and |
| hitting Control-C or code calling :func:`sys.exit`. A bare ``except:`` will |
| catch all exceptions, so you commonly need to list :exc:`KeyboardInterrupt` and |
| :exc:`SystemExit` in order to re-raise them. The usual pattern is:: |
| |
| try: |
| ... |
| except (KeyboardInterrupt, SystemExit): |
| raise |
n | except: |
n | except: |
| # Log error... |
| # Log error... |
| # Continue running program... |
| |
| In Python 2.5, you can now write ``except Exception`` to achieve the same |
| result, catching all the exceptions that usually indicate errors but leaving |
| :exc:`KeyboardInterrupt` and :exc:`SystemExit` alone. As in previous versions, |
| a bare ``except:`` still catches all exceptions. |
| |
| The goal for Python 3.0 is to require any class raised as an exception to derive |
| platforms. |
| |
| Various pieces of the Python interpreter used C's :ctype:`int` type to store |
| sizes or counts; for example, the number of items in a list or tuple were stored |
| in an :ctype:`int`. The C compilers for most 64-bit platforms still define |
| :ctype:`int` as a 32-bit type, so that meant that lists could only hold up to |
| ``2**31 - 1`` = 2147483647 items. (There are actually a few different |
| programming models that 64-bit C compilers can use -- see |
n | `<http://www.unix.org/version2/whatsnew/lp64_wp.html>`_ for a discussion -- but |
n | http://www.unix.org/version2/whatsnew/lp64_wp.html for a discussion -- but the |
| the most commonly available model leaves :ctype:`int` as 32 bits.) |
| most commonly available model leaves :ctype:`int` as 32 bits.) |
| |
| A limit of 2147483647 items doesn't really matter on a 32-bit platform because |
| you'll run out of memory before hitting the length limit. Each list item |
| requires space for a pointer, which is 4 bytes, plus space for a |
| :ctype:`PyObject` representing the item. 2147483647\*4 is already more bytes |
| than a 32-bit address space can contain. |
| |
| It's possible to address that much memory on a 64-bit platform, however. The |
| :meth:`__int__`, floating-point numbers would also become legal slice indexes |
| and that's clearly an undesirable behaviour. |
| |
| Instead, a new special method called :meth:`__index__` was added. It takes no |
| arguments and returns an integer giving the slice index to use. For example:: |
| |
| class C: |
| def __index__ (self): |
n | return self.value |
n | return self.value |
| |
| The return value must be either a Python integer or long integer. The |
| interpreter will check that the type returned is correct, and raises a |
| :exc:`TypeError` if this requirement isn't met. |
| |
| A corresponding :attr:`nb_index` slot was added to the C-level |
| :ctype:`PyNumberMethods` structure to let C extensions implement this protocol. |
| :cfunc:`PyNumber_Index(obj)` can be used in extension code to call the |
| :meth:`__index__` function and retrieve its result. |
| |
| |
| .. seealso:: |
| |
| :pep:`357` - Allowing Any Object to be Used for Slicing |
| PEP written and implemented by Travis Oliphant. |
| |
n | .. % ====================================================================== |
n | .. ====================================================================== |
| |
| |
| .. _other-lang: |
| |
| Other Language Changes |
| ====================== |
| |
| Here are all of the changes that Python 2.5 makes to the core Python language. |
| * The :meth:`startswith` and :meth:`endswith` methods of string types now accept |
| tuples of strings to check for. :: |
| |
| def is_image_file (filename): |
| return filename.endswith(('.gif', '.jpg', '.tiff')) |
| |
| (Implemented by Georg Brandl following a suggestion by Tom Lynn.) |
| |
n | .. % RFE #1491485 |
n | .. RFE #1491485 |
| |
| * The :func:`min` and :func:`max` built-in functions gained a ``key`` keyword |
| parameter analogous to the ``key`` argument for :meth:`sort`. This parameter |
| supplies a function that takes a single argument and is called for every value |
| in the list; :func:`min`/:func:`max` will return the element with the |
| smallest/largest return value from this function. For example, to find the |
| longest string in a list, you can do:: |
| |
| L = ['medium', 'longest', 'short'] |
| # Prints 'longest' |
n | print max(L, key=len) |
n | print max(L, key=len) |
| # Prints 'short', because lexicographically 'short' has the largest value |
n | print max(L) |
n | print max(L) |
| |
| (Contributed by Steven Bethard and Raymond Hettinger.) |
| |
| * Two new built-in functions, :func:`any` and :func:`all`, evaluate whether an |
| iterator contains any true or false values. :func:`any` returns :const:`True` |
| if any value returned by the iterator is true; otherwise it will return |
| :const:`False`. :func:`all` returns :const:`True` only if all of the values |
| returned by the iterator evaluate as true. (Suggested by Guido van Rossum, and |
| implemented by Raymond Hettinger.) |
| |
n | * The result of a class's :meth:`__hash__` method can now be either a long |
n | * The result of a class's :meth:`__hash__` method can now be either a long |
| integer or a regular integer. If a long integer is returned, the hash of that |
n | value is taken. In earlier versions the hash value was required to be a regular |
n | value is taken. In earlier versions the hash value was required to be a |
| integer, but in 2.5 the :func:`id` built-in was changed to always return non- |
| regular integer, but in 2.5 the :func:`id` built-in was changed to always |
| negative numbers, and users often seem to use ``id(self)`` in :meth:`__hash__` |
| return non-negative numbers, and users often seem to use ``id(self)`` in |
| methods (though this is discouraged). |
| :meth:`__hash__` methods (though this is discouraged). |
| |
n | .. % Bug #1536021 |
n | .. Bug #1536021 |
| |
| * ASCII is now the default encoding for modules. It's now a syntax error if a |
| module contains string literals with 8-bit characters but doesn't have an |
| encoding declaration. In Python 2.4 this triggered a warning, not a syntax |
| error. See :pep:`263` for how to declare a module's encoding; for example, you |
| might add a line like this near the top of the source file:: |
| |
| # -*- coding: latin1 -*- |
| |
| * A new warning, :class:`UnicodeWarning`, is triggered when you attempt to |
| compare a Unicode string and an 8-bit string that can't be converted to Unicode |
| using the default ASCII encoding. The result of the comparison is false:: |
| |
| >>> chr(128) == unichr(128) # Can't convert chr(128) to Unicode |
n | __main__:1: UnicodeWarning: Unicode equal comparison failed |
n | __main__:1: UnicodeWarning: Unicode equal comparison failed |
| to convert both arguments to Unicode - interpreting them |
| to convert both arguments to Unicode - interpreting them |
| as being unequal |
| False |
| >>> chr(127) == unichr(127) # chr(127) can be converted |
| True |
| |
| Previously this would raise a :class:`UnicodeDecodeError` exception, but in 2.5 |
| this could result in puzzling problems when accessing a dictionary. If you |
| looked up ``unichr(128)`` and ``chr(128)`` was being used as a key, you'd get a |
| by Raymond Hettinger.) |
| |
| * The speed of some Unicode operations, such as finding substrings, string |
| splitting, and character map encoding and decoding, has been improved. |
| (Substring search and splitting improvements were added by Fredrik Lundh and |
| Andrew Dalke at the NeedForSpeed sprint. Character maps were improved by Walter |
| Dörwald and Martin von Löwis.) |
| |
n | .. % Patch 1313939, 1359618 |
n | .. Patch 1313939, 1359618 |
| |
| * The :func:`long(str, base)` function is now faster on long digit strings |
| because fewer intermediate results are calculated. The peak is for strings of |
| around 800--1000 digits where the function is 6 times faster. (Contributed by |
| Alan McIntyre and committed at the NeedForSpeed sprint.) |
| |
n | .. % Patch 1442927 |
n | .. Patch 1442927 |
| |
| * It's now illegal to mix iterating over a file with ``for line in file`` and |
| calling the file object's :meth:`read`/:meth:`readline`/:meth:`readlines` |
| methods. Iteration uses an internal buffer and the :meth:`read\*` methods |
| don't use that buffer. Instead they would return the data following the |
| buffer, causing the data to appear out of order. Mixing iteration and these |
| methods will now trigger a :exc:`ValueError` from the :meth:`read\*` method. |
| (Implemented by Thomas Wouters.) |
| |
| .. Patch 1397960 |
| |
| * The :mod:`struct` module now compiles structure format strings into an |
| internal representation and caches this representation, yielding a 20% speedup. |
| (Contributed by Bob Ippolito at the NeedForSpeed sprint.) |
| |
| * The :mod:`re` module got a 1 or 2% speedup by switching to Python's allocator |
| functions instead of the system's :cfunc:`malloc` and :cfunc:`free`. |
| (Contributed by Jack Diederich at the NeedForSpeed sprint.) |
| |
| * Function calls are now faster because code objects now keep the most recently |
| finished frame (a "zombie frame") in an internal field of the code object, |
| reusing it the next time the code object is invoked. (Original patch by Michael |
| Hudson, modified by Armin Rigo and Richard Jones; committed at the NeedForSpeed |
| sprint.) Frame objects are also slightly smaller, which may improve cache |
| locality and reduce memory usage a bit. (Contributed by Neal Norwitz.) |
| |
n | .. % Patch 876206 |
n | .. Patch 876206 |
| .. % Patch 1337051 |
| .. Patch 1337051 |
| |
| * Python's built-in exceptions are now new-style classes, a change that speeds |
| up instantiation considerably. Exception handling in Python 2.5 is therefore |
| about 30% faster than in 2.4. (Contributed by Richard Jones, Georg Brandl and |
| Sean Reifschneider at the NeedForSpeed sprint.) |
| |
| * Importing now caches the paths tried, recording whether they exist or not so |
| that the interpreter makes fewer :cfunc:`open` and :cfunc:`stat` calls on |
| startup. (Contributed by Martin von Löwis and Georg Brandl.) |
| |
n | .. % Patch 921466 |
n | .. Patch 921466 |
| |
n | .. % ====================================================================== |
n | .. ====================================================================== |
| |
| |
n | .. _modules: |
n | .. _25modules: |
| |
| New, Improved, and Removed Modules |
| ================================== |
| |
| The standard library received many enhancements and bug fixes in Python 2.5. |
| Here's a partial list of the most notable changes, sorted alphabetically by |
| module name. Consult the :file:`Misc/NEWS` file in the source tree for a more |
| complete list of changes, or look through the SVN logs for all the details. |
| of a tuple. :class:`CodecInfo` instances behave like a 4-tuple to preserve |
| backward compatibility but also have the attributes :attr:`encode`, |
| :attr:`decode`, :attr:`incrementalencoder`, :attr:`incrementaldecoder`, |
| :attr:`streamwriter`, and :attr:`streamreader`. Incremental codecs can receive |
| input and produce output in multiple chunks; the output is the same as if the |
| entire input was fed to the non-incremental codec. See the :mod:`codecs` module |
| documentation for details. (Designed and implemented by Walter Dörwald.) |
| |
n | .. % Patch 1436130 |
n | .. Patch 1436130 |
| |
| * The :mod:`collections` module gained a new type, :class:`defaultdict`, that |
| subclasses the standard :class:`dict` type. The new type mostly behaves like a |
| dictionary but constructs a default value when a key isn't present, |
| automatically adding it to the dictionary for the requested key value. |
| |
| The first argument to :class:`defaultdict`'s constructor is a factory function |
| that gets called whenever a key is requested but not found. This factory |
| index = defaultdict(list) |
| |
| for w in words: |
| init_letter = w[0] |
| index[init_letter].append(w) |
| |
| Printing ``index`` results in the following output:: |
| |
n | defaultdict(<type 'list'>, {'c': ['cammin', 'che'], 'e': ['era'], |
n | defaultdict(<type 'list'>, {'c': ['cammin', 'che'], 'e': ['era'], |
| 'd': ['del', 'di', 'diritta'], 'm': ['mezzo', 'mi'], |
| 'd': ['del', 'di', 'diritta'], 'm': ['mezzo', 'mi'], |
| 'l': ['la'], 'o': ['oscura'], 'n': ['nel', 'nostra'], |
| 'l': ['la'], 'o': ['oscura'], 'n': ['nel', 'nostra'], |
| 'p': ['per'], 's': ['selva', 'smarrita'], |
| 'p': ['per'], 's': ['selva', 'smarrita'], |
| 'r': ['ritrovai'], 'u': ['una'], 'v': ['vita', 'via']} |
| |
| (Contributed by Guido van Rossum.) |
| |
| * The :class:`deque` double-ended queue type supplied by the :mod:`collections` |
| module now has a :meth:`remove(value)` method that removes the first occurrence |
| of *value* in the queue, raising :exc:`ValueError` if the value isn't found. |
| (Contributed by Raymond Hettinger.) |
| |
| * New module: The :mod:`contextlib` module contains helper functions for use |
n | with the new ':keyword:`with`' statement. See section :ref:`module-contextlib` |
n | with the new ':keyword:`with`' statement. See section :ref:`contextlibmod` |
| for more about this module. |
| |
| * New module: The :mod:`cProfile` module is a C implementation of the existing |
| :mod:`profile` module that has much lower overhead. The module's interface is |
| the same as :mod:`profile`: you run ``cProfile.run('main()')`` to profile a |
| function, can save profile data to a file, etc. It's not yet known if the |
| Hotshot profiler, which is also written in C but doesn't match the |
| :mod:`profile` module's interface, will continue to be maintained in future |
| being executed at all. This is intended for code snippets that are usage |
| examples intended for the reader and aren't actually test cases. |
| |
| An *encoding* parameter was added to the :func:`testfile` function and the |
| :class:`DocFileSuite` class to specify the file's encoding. This makes it |
| easier to use non-ASCII characters in tests contained within a docstring. |
| (Contributed by Bjorn Tillenius.) |
| |
n | .. % Patch 1080727 |
n | .. Patch 1080727 |
| |
| * The :mod:`email` package has been updated to version 4.0. (Contributed by |
| Barry Warsaw.) |
| |
n | .. % XXX need to provide some more detail here |
n | .. XXX need to provide some more detail here |
| |
| * The :mod:`fileinput` module was made more flexible. Unicode filenames are now |
| supported, and a *mode* parameter that defaults to ``"r"`` was added to the |
| :func:`input` function to allow opening files in binary or universal-newline |
| mode. Another new parameter, *openhook*, lets you use a function other than |
| :func:`open` to open the input files. Once you're iterating over the set of |
| files, the :class:`FileInput` object's new :meth:`fileno` returns the file |
| descriptor for the currently opened file. (Contributed by Georg Brandl.) |
| :func:`format_string` function that works like :func:`format` but also supports |
| mixing %char specifiers with arbitrary text. |
| |
| A new :func:`currency` function was also added that formats a number according |
| to the current locale's settings. |
| |
| (Contributed by Georg Brandl.) |
| |
n | .. % Patch 1180296 |
n | .. Patch 1180296 |
| |
| * The :mod:`mailbox` module underwent a massive rewrite to add the capability to |
| modify mailboxes in addition to reading them. A new set of classes that include |
| :class:`mbox`, :class:`MH`, and :class:`Maildir` are used to read mailboxes, and |
| have an :meth:`add(message)` method to add messages, :meth:`remove(key)` to |
| remove messages, and :meth:`lock`/:meth:`unlock` to lock/unlock the mailbox. |
| The following example converts a maildir-format mailbox into an mbox-format |
| one:: |
| :func:`resource.getrusage` function. :func:`wait4(pid)` does take a process ID. |
| (Contributed by Chad J. Schroeder.) |
| |
| On FreeBSD, the :func:`os.stat` function now returns times with nanosecond |
| resolution, and the returned object now has :attr:`st_gen` and |
| :attr:`st_birthtime`. The :attr:`st_flags` member is also available, if the |
| platform supports it. (Contributed by Antti Louko and Diego Pettenò.) |
| |
n | .. % (Patch 1180695, 1212117) |
n | .. (Patch 1180695, 1212117) |
| |
| * The Python debugger provided by the :mod:`pdb` module can now store lists of |
| commands to execute when a breakpoint is reached and execution stops. Once |
| breakpoint #1 has been created, enter ``commands 1`` and enter a series of |
| commands to be executed, finishing the list with ``end``. The command list can |
| include commands that resume execution, such as ``continue`` or ``next``. |
| (Contributed by Grégoire Dooms.) |
| |
n | .. % Patch 790710 |
n | .. Patch 790710 |
| |
| * The :mod:`pickle` and :mod:`cPickle` modules no longer accept a return value |
| of ``None`` from the :meth:`__reduce__` method; the method must return a tuple |
| of arguments instead. The ability to return ``None`` was deprecated in Python |
| 2.4, so this completes the removal of the feature. |
| |
| * The :mod:`pkgutil` module, containing various utility functions for finding |
| packages, was enhanced to support PEP 302's import hooks and now also works for |
| detailed measurement of the interpreter's speed. It times particular operations |
| such as function calls, tuple slicing, method lookups, and numeric operations, |
| instead of performing many different operations and reducing the result to a |
| single number as :file:`pystone.py` does. |
| |
| * The :mod:`pyexpat` module now uses version 2.0 of the Expat parser. |
| (Contributed by Trent Mick.) |
| |
n | * The :class:`Queue` class provided by the :mod:`Queue` module gained two new |
| methods. :meth:`join` blocks until all items in the queue have been retrieved |
| and all processing work on the items have been completed. Worker threads call |
| the other new method, :meth:`task_done`, to signal that processing for an item |
| has been completed. (Contributed by Raymond Hettinger.) |
| |
| * The old :mod:`regex` and :mod:`regsub` modules, which have been deprecated |
| ever since Python 2.0, have finally been deleted. Other deleted modules: |
| :mod:`statcache`, :mod:`tzparse`, :mod:`whrandom`. |
| |
| * Also deleted: the :file:`lib-old` directory, which includes ancient modules |
| such as :mod:`dircmp` and :mod:`ni`, was removed. :file:`lib-old` wasn't on the |
| default ``sys.path``, so unless your programs explicitly added the directory to |
| ``sys.path``, this removal shouldn't affect your code. |
| |
| * The :mod:`rlcompleter` module is no longer dependent on importing the |
| :mod:`readline` module and therefore now works on non-Unix platforms. (Patch |
| from Robert Kiendl.) |
| |
n | .. % Patch #1472854 |
n | .. Patch #1472854 |
| |
| * The :mod:`SimpleXMLRPCServer` and :mod:`DocXMLRPCServer` classes now have a |
| :attr:`rpc_paths` attribute that constrains XML-RPC operations to a limited set |
| of URL paths; the default is to allow only ``'/'`` and ``'/RPC2'``. Setting |
| :attr:`rpc_paths` to ``None`` or an empty tuple disables this path checking. |
| |
n | .. % Bug #1473048 |
n | .. Bug #1473048 |
| |
| * The :mod:`socket` module now supports :const:`AF_NETLINK` sockets on Linux, |
| thanks to a patch from Philippe Biondi. Netlink sockets are a Linux-specific |
| mechanism for communications between a user-space process and kernel code; an |
n | introductory article about them is at |
n | introductory article about them is at http://www.linuxjournal.com/article/7356. |
| `<http://www.linuxjournal.com/article/7356>`_. In Python code, netlink addresses |
| In Python code, netlink addresses are represented as a tuple of 2 integers, |
| are represented as a tuple of 2 integers, ``(pid, group_mask)``. |
| ``(pid, group_mask)``. |
| |
n | Two new methods on socket objects, :meth:`recv_buf(buffer)` and |
n | Two new methods on socket objects, :meth:`recv_into(buffer)` and |
| :meth:`recvfrom_buf(buffer)`, store the received data in an object that |
| :meth:`recvfrom_into(buffer)`, store the received data in an object that |
| supports the buffer protocol instead of returning the data as a string. This |
| means you can put the data directly into an array or a memory-mapped file. |
| |
| Socket objects also gained :meth:`getfamily`, :meth:`gettype`, and |
| :meth:`getproto` accessor methods to retrieve the family, type, and protocol |
| values for the socket. |
| |
| * New module: the :mod:`spwd` module provides functions for accessing the shadow |
| * The :class:`TarFile` class in the :mod:`tarfile` module now has an |
| :meth:`extractall` method that extracts all members from the archive into the |
| current working directory. It's also possible to set a different directory as |
| the extraction target, and to unpack only a subset of the archive's members. |
| |
| The compression used for a tarfile opened in stream mode can now be autodetected |
| using the mode ``'r|*'``. (Contributed by Lars Gustäbel.) |
| |
n | .. % patch 918101 |
n | .. patch 918101 |
| |
| * The :mod:`threading` module now lets you set the stack size used when new |
| threads are created. The :func:`stack_size([*size*])` function returns the |
| currently configured stack size, and supplying the optional *size* parameter |
| sets a new value. Not all platforms support changing the stack size, but |
| Windows, POSIX threading, and OS/2 all do. (Contributed by Andrew MacIntyre.) |
| |
n | .. % Patch 1454481 |
n | .. Patch 1454481 |
| |
| * The :mod:`unicodedata` module has been updated to use version 4.1.0 of the |
| Unicode character database. Version 3.2.0 is required by some specifications, |
| so it's still available as :attr:`unicodedata.ucd_3_2_0`. |
| |
| * New module: the :mod:`uuid` module generates universally unique identifiers |
| (UUIDs) according to :rfc:`4122`. The RFC defines several different UUID |
| versions that are generated from a starting string, from system properties, or |
| there are a number of switches to control the behaviour (:option:`-n` for a new |
| browser window, :option:`-t` for a new tab). New module-level functions, |
| :func:`open_new` and :func:`open_new_tab`, were added to support this. The |
| module's :func:`open` function supports an additional feature, an *autoraise* |
| parameter that signals whether to raise the open window when possible. A number |
| of additional browsers were added to the supported list such as Firefox, Opera, |
| Konqueror, and elinks. (Contributed by Oleg Broytmann and Georg Brandl.) |
| |
n | .. % Patch #754022 |
n | .. Patch #754022 |
| |
| * The :mod:`xmlrpclib` module now supports returning :class:`datetime` objects |
| for the XML-RPC date type. Supply ``use_datetime=True`` to the :func:`loads` |
| function or the :class:`Unmarshaller` class to enable this feature. (Contributed |
| by Skip Montanaro.) |
| |
n | .. % Patch 1120353 |
n | .. Patch 1120353 |
| |
| * The :mod:`zipfile` module now supports the ZIP64 version of the format, |
| meaning that a .zip archive can now be larger than 4 GiB and can contain |
| individual files larger than 4 GiB. (Contributed by Ronald Oussoren.) |
| |
n | .. % Patch 1446489 |
n | .. Patch 1446489 |
| |
| * The :mod:`zlib` module's :class:`Compress` and :class:`Decompress` objects now |
| support a :meth:`copy` method that makes a copy of the object's internal state |
| and returns a new :class:`Compress` or :class:`Decompress` object. |
| (Contributed by Chris AtLee.) |
| |
n | .. % Patch 1435422 |
n | .. Patch 1435422 |
| |
n | .. % ====================================================================== |
n | .. ====================================================================== |
| |
| |
| .. _module-ctypes: |
| |
| The ctypes package |
| ------------------ |
| |
| The :mod:`ctypes` package, written by Thomas Heller, has been added to the |
| modules, now that :mod:`ctypes` is included with core Python. |
| |
| |
| .. seealso:: |
| |
| http://starship.python.net/crew/theller/ctypes/ |
| The ctypes web page, with a tutorial, reference, and FAQ. |
| |
n | ../lib/module-ctypes.html |
| The documentation for the :mod:`ctypes` module. |
| The documentation for the :mod:`ctypes` module. |
| |
n | .. % ====================================================================== |
n | .. ====================================================================== |
| |
| |
| .. _module-etree: |
| |
| The ElementTree package |
| ----------------------- |
| |
| A subset of Fredrik Lundh's ElementTree library for processing XML has been |
| added to the standard library as :mod:`xml.etree`. The available modules are |
| :mod:`ElementTree`, :mod:`ElementPath`, and :mod:`ElementInclude` from |
| ElementTree 1.2.6. The :mod:`cElementTree` accelerator module is also |
| included. |
| |
| The rest of this section will provide a brief overview of using ElementTree. |
| Full documentation for ElementTree is available at |
n | `<http://effbot.org/zone/element-index.htm>`_. |
n | http://effbot.org/zone/element-index.htm. |
| |
| ElementTree represents an XML document as a tree of element nodes. The text |
| content of the document is stored as the :attr:`.text` and :attr:`.tail` |
| attributes of (This is one of the major differences between ElementTree and |
| the Document Object Model; in the DOM there are many different types of node, |
| including :class:`TextNode`.) |
| |
| The most commonly used parsing function is :func:`parse`, that takes either a |
| :meth:`update(string)` hashes the specified string into the current digest |
| state, :meth:`digest` and :meth:`hexdigest` return the digest value as a binary |
| string or a string of hex digits, and :meth:`copy` returns a new hashing object |
| with the same digest state. |
| |
| |
| .. seealso:: |
| |
n | ../lib/module-hashlib.html |
| The documentation for the :mod:`hashlib` module. |
| The documentation for the :mod:`hashlib` module. |
| |
n | .. % ====================================================================== |
n | .. ====================================================================== |
| |
| |
| .. _module-sqlite: |
| |
| The sqlite3 package |
| ------------------- |
| |
n | The pysqlite module (`<http://www.pysqlite.org>`_), a wrapper for the SQLite |
n | The pysqlite module (http://www.pysqlite.org), a wrapper for the SQLite embedded |
| embedded database, has been added to the standard library under the package name |
| database, has been added to the standard library under the package name |
| :mod:`sqlite3`. |
| |
n | SQLite is a C library that provides a SQL-language database that stores data in |
n | SQLite is a C library that provides a lightweight disk-based database that |
| disk files without requiring a separate server process. pysqlite was written by |
| doesn't require a separate server process and allows accessing the database |
| Gerhard Häring and provides a SQL interface compliant with the DB-API 2.0 |
| using a nonstandard variant of the SQL query language. Some applications can use |
| specification described by :pep:`249`. This means that it should be possible to |
| SQLite for internal data storage. It's also possible to prototype an |
| write the first version of your applications using SQLite for data storage. If |
| application using SQLite and then port the code to a larger database such as |
| switching to a larger database such as PostgreSQL or Oracle is later necessary, |
| PostgreSQL or Oracle. |
| the switch should be relatively easy. |
| |
| pysqlite was written by Gerhard Häring and provides a SQL interface compliant |
| with the DB-API 2.0 specification described by :pep:`249`. |
| |
| If you're compiling the Python source yourself, note that the source tree |
| doesn't include the SQLite code, only the wrapper module. You'll need to have |
| the SQLite libraries and headers installed before compiling Python, and the |
| build process will compile the module when the necessary headers are available. |
| |
| To use the module, you must first create a :class:`Connection` object that |
| represents the database. Here the data will be stored in the |
| |
| Once you have a :class:`Connection`, you can create a :class:`Cursor` object |
| and call its :meth:`execute` method to perform SQL commands:: |
| |
| c = conn.cursor() |
| |
| # Create table |
| c.execute('''create table stocks |
n | (date timestamp, trans varchar, symbol varchar, |
n | (date text, trans text, symbol text, |
| qty decimal, price decimal)''') |
| qty real, price real)''') |
| |
| # Insert a row of data |
| c.execute("""insert into stocks |
| values ('2006-01-05','BUY','RHAT',100,35.14)""") |
| |
| Usually your SQL operations will need to use values from Python variables. You |
| shouldn't assemble your query using Python's string operations because doing so |
| is insecure; it makes your program vulnerable to an SQL injection attack. |
| ... |
| (u'2006-01-05', u'BUY', u'RHAT', 100, 35.140000000000001) |
| (u'2006-03-28', u'BUY', u'IBM', 1000, 45.0) |
| (u'2006-04-06', u'SELL', u'IBM', 500, 53.0) |
| (u'2006-04-05', u'BUY', u'MSOFT', 1000, 72.0) |
| >>> |
| |
| For more information about the SQL dialect supported by SQLite, see |
n | `<http://www.sqlite.org>`_. |
n | http://www.sqlite.org. |
| |
| |
| .. seealso:: |
| |
| http://www.pysqlite.org |
| The pysqlite web page. |
| |
| http://www.sqlite.org |
| The SQLite web page; the documentation describes the syntax and the available |
| data types for the supported SQL dialect. |
| |
n | ../lib/module-sqlite3.html |
| The documentation for the :mod:`sqlite3` module. |
| The documentation for the :mod:`sqlite3` module. |
| |
| :pep:`249` - Database API Specification 2.0 |
| PEP written by Marc-André Lemburg. |
| |
n | .. % ====================================================================== |
n | .. ====================================================================== |
| |
| |
| .. _module-wsgiref: |
| |
| The wsgiref package |
| ------------------- |
| |
| The Web Server Gateway Interface (WSGI) v1.0 defines a standard interface |
| between web servers and Python web applications and is described in :pep:`333`. |
| The :mod:`wsgiref` package is a reference implementation of the WSGI |
| specification. |
| |
n | .. % XXX should this be in a PEP 333 section instead? |
n | .. XXX should this be in a PEP 333 section instead? |
| |
| The package includes a basic HTTP server that will run a WSGI application; this |
| server is useful for debugging but isn't intended for production use. Setting |
| up a server takes only a few lines of code:: |
| |
| from wsgiref import simple_server |
| |
| wsgi_app = ... |
| |
| host = '' |
| port = 8000 |
| httpd = simple_server.make_server(host, port, wsgi_app) |
| httpd.serve_forever() |
| |
n | .. % XXX discuss structure of WSGI applications? |
n | .. XXX discuss structure of WSGI applications? |
| .. % XXX provide an example using Django or some other framework? |
| .. XXX provide an example using Django or some other framework? |
| |
| |
| .. seealso:: |
| |
| http://www.wsgi.org |
| A central web site for WSGI-related resources. |
| |
| :pep:`333` - Python Web Server Gateway Interface v1.0 |
| PEP written by Phillip J. Eby. |
| |
n | .. % ====================================================================== |
n | .. ====================================================================== |
| |
| |
| .. _build-api: |
| |
| Build and C API Changes |
| ======================= |
| |
| Changes to Python's build process and to the C API include: |
| * The Python source tree was converted from CVS to Subversion, in a complex |
| migration procedure that was supervised and flawlessly carried out by Martin von |
| Löwis. The procedure was developed as :pep:`347`. |
| |
| * Coverity, a company that markets a source code analysis tool called Prevent, |
| provided the results of their examination of the Python source code. The |
| analysis found about 60 bugs that were quickly fixed. Many of the bugs were |
| refcounting problems, often occurring in error-handling code. See |
n | `<http://scan.coverity.com>`_ for the statistics. |
n | http://scan.coverity.com for the statistics. |
| |
| * The largest change to the C API came from :pep:`353`, which modifies the |
| interpreter to use a :ctype:`Py_ssize_t` type definition instead of |
| :ctype:`int`. See the earlier section :ref:`pep-353` for a discussion of this |
| change. |
| |
| * The design of the bytecode compiler has changed a great deal, no longer |
| generating bytecode by traversing the parse tree. Instead the parse tree is |
| more information, read the source code, and then ask questions on python-dev. |
| |
| The AST code was developed under Jeremy Hylton's management, and implemented by |
| (in alphabetical order) Brett Cannon, Nick Coghlan, Grant Edwards, John |
| Ehresman, Kurt Kaiser, Neal Norwitz, Tim Peters, Armin Rigo, and Neil |
| Schemenauer, plus the participants in a number of AST sprints at conferences |
| such as PyCon. |
| |
n | .. % List of names taken from Jeremy's python-dev post at |
n | .. List of names taken from Jeremy's python-dev post at |
| .. % http://mail.python.org/pipermail/python-dev/2005-October/057500.html |
| .. http://mail.python.org/pipermail/python-dev/2005-October/057500.html |
| |
| * Evan Jones's patch to obmalloc, first described in a talk at PyCon DC 2005, |
| was applied. Python 2.4 allocated small objects in 256K-sized arenas, but never |
| freed arenas. With this patch, Python will free arenas when they're empty. The |
| net effect is that on some platforms, when you allocate many objects, Python's |
| memory usage may actually drop when you delete them and the memory may be |
| returned to the operating system. (Implemented by Evan Jones, and reworked by |
| Tim Peters.) |
| compiled with a C++ compiler without errors. (Implemented by Anthony Baxter, |
| Martin von Löwis, Skip Montanaro.) |
| |
| * The :cfunc:`PyRange_New` function was removed. It was never documented, never |
| used in the core code, and had dangerously lax error checking. In the unlikely |
| case that your extensions were using it, you can replace it by something like |
| the following:: |
| |
n | range = PyObject_CallFunction((PyObject*) &PyRange_Type, "lll", |
n | range = PyObject_CallFunction((PyObject*) &PyRange_Type, "lll", |
| start, stop, step); |
| |
n | .. % ====================================================================== |
n | .. ====================================================================== |
| |
| |
| .. _ports: |
| |
| Port-Specific Changes |
| --------------------- |
| |
| * MacOS X (10.3 and higher): dynamic loading of modules now uses the |
| :cfunc:`dlopen` function instead of MacOS-specific functions. |
| |
n | * MacOS X: a :option:`--enable-universalsdk` switch was added to the |
n | * MacOS X: an :option:`--enable-universalsdk` switch was added to the |
| :program:`configure` script that compiles the interpreter as a universal binary |
| able to run on both PowerPC and Intel processors. (Contributed by Ronald |
n | Oussoren.) |
n | Oussoren; :issue:`2573`.) |
| |
| * Windows: :file:`.dll` is no longer supported as a filename extension for |
| extension modules. :file:`.pyd` is now the only filename extension that will be |
| searched for. |
| |
n | .. % ====================================================================== |
n | .. ====================================================================== |
| |
| |
| .. _porting: |
| |
| Porting to Python 2.5 |
| ===================== |
| |
| This section lists previously described changes that may require changes to your |
| the same change to avoid warnings and to support 64-bit machines. See the |
| earlier section :ref:`pep-353` for a discussion of this change. |
| |
| * C API: The obmalloc changes mean that you must be careful to not mix usage |
| of the :cfunc:`PyMem_\*` and :cfunc:`PyObject_\*` families of functions. Memory |
| allocated with one family's :cfunc:`\*_Malloc` must be freed with the |
| corresponding family's :cfunc:`\*_Free` function. |
| |
n | .. % ====================================================================== |
n | .. ====================================================================== |
| |
t | |
| .. _acks: |
| |
| Acknowledgements |
| ================ |
| |
| The author would like to thank the following people for offering suggestions, |
| corrections and assistance with various drafts of this article: Georg Brandl, |
| Nick Coghlan, Phillip J. Eby, Lars Gustäbel, Raymond Hettinger, Ralf W. Grosse- |
| Kunstleve, Kent Johnson, Iain Lowe, Martin von Löwis, Fredrik Lundh, Andrew |