| |
| INFOSEEK CORPORATION DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE, |
| INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT |
| SHALL INFOSEEK CORPORATION BE LIABLE FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL |
| DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, |
| WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING |
| OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. |
| |
n | The profiler was written after only programming in Python for 3 weeks. As a |
n | .. _profiler-introduction: |
| result, it is probably clumsy code, but I don't know for sure yet 'cause I'm a |
| beginner :-). I did work hard to make the code run fast, so that profiling |
| would be a reasonable thing to do. I tried not to repeat code fragments, but |
| I'm sure I did some stuff in really awkward ways at times. Please send |
| suggestions for improvements to: jar@netscape.com. I won't promise *any* |
| support. ...but I'd appreciate the feedback. |
| |
| |
| Introduction to the profilers |
| ============================= |
n | |
| .. _profiler introduction: |
| |
| .. index:: |
| single: deterministic profiling |
| single: profiling, deterministic |
| |
n | A :dfn:`profiler` is a program that describes the run time performance of a |
n | A :dfn:`profiler` is a program that describes the run time performance |
| program, providing a variety of statistics. This documentation describes the |
| of a program, providing a variety of statistics. This documentation |
| profiler functionality provided in the modules :mod:`profile` and :mod:`pstats`. |
| describes the profiler functionality provided in the modules |
| :mod:`cProfile`, :mod:`profile` and :mod:`pstats`. This profiler |
| This profiler provides :dfn:`deterministic profiling` of any Python programs. |
| provides :dfn:`deterministic profiling` of Python programs. It also |
| It also provides a series of report generation tools to allow users to rapidly |
| provides a series of report generation tools to allow users to rapidly |
| examine the results of a profile operation. |
| |
| The Python standard library provides three different profilers: |
| |
n | #. :mod:`profile`, a pure Python module, described in the sequel. Copyright © |
n | #. :mod:`cProfile` is recommended for most users; it's a C extension |
| with reasonable overhead |
| that makes it suitable for profiling long-running programs. |
| Based on :mod:`lsprof`, |
| contributed by Brett Rosen and Ted Czotter. |
| |
| .. versionadded:: 2.5 |
| |
| #. :mod:`profile`, a pure Python module whose interface is imitated by |
| :mod:`cProfile`. Adds significant overhead to profiled programs. |
| If you're trying to extend |
| the profiler in some way, the task might be easier with this module. |
| 1994, by InfoSeek Corporation. |
| Copyright © 1994, by InfoSeek Corporation. |
| |
| .. versionchanged:: 2.4 |
n | also reports the time spent in calls to built-in functions and methods. |
n | Now also reports the time spent in calls to built-in functions and methods. |
| |
n | #. :mod:`cProfile`, a module written in C, with a reasonable overhead that makes |
n | #. :mod:`hotshot` was an experimental C module that focused on minimizing |
| it suitable for profiling long-running programs. Based on :mod:`lsprof`, |
| the overhead of profiling, at the expense of longer data |
| contributed by Brett Rosen and Ted Czotter. |
| post-processing times. It is no longer maintained and may be |
| dropped in a future version of Python. |
| |
n | .. versionadded:: 2.5 |
| |
| #. :mod:`hotshot`, a C module focusing on minimizing the overhead while |
| profiling, at the expense of long data post-processing times. |
| |
| .. versionchanged:: 2.5 |
n | the results should be more meaningful than in the past: the timing core |
n | The results should be more meaningful than in the past: the timing core |
| contained a critical bug. |
| |
| The :mod:`profile` and :mod:`cProfile` modules export the same interface, so |
n | they are mostly interchangeables; :mod:`cProfile` has a much lower overhead but |
n | they are mostly interchangeable; :mod:`cProfile` has a much lower overhead but |
| is not so far as well-tested and might not be available on all systems. |
| is newer and might not be available on all systems. |
| :mod:`cProfile` is really a compatibility layer on top of the internal |
n | :mod:`_lsprof` module. The :mod:`hotshot` module is reserved to specialized |
n | :mod:`_lsprof` module. The :mod:`hotshot` module is reserved for specialized |
| usages. |
| usage. |
| |
| .. % \section{How Is This Profiler Different From The Old Profiler?} |
| .. % \nodename{Profiler Changes} |
| .. % |
| .. % (This section is of historical importance only; the old profiler |
| .. % discussed here was last seen in Python 1.1.) |
| .. % |
| .. % The big changes from old profiling module are that you get more |
| .. % information, and you pay less CPU time. It's not a trade-off, it's a |
| .. % trade-up. |
| .. % |
| .. % To be specific: |
| .. % |
| .. % \begin{description} |
| .. % |
| .. % \item[Bugs removed:] |
| .. % Local stack frame is no longer molested, execution time is now charged |
| .. % to correct functions. |
| .. % |
| .. % \item[Accuracy increased:] |
| .. % Profiler execution time is no longer charged to user's code, |
| .. % calibration for platform is supported, file reads are not done \emph{by} |
| .. % profiler \emph{during} profiling (and charged to user's code!). |
| .. % |
| .. % \item[Speed increased:] |
| .. % Overhead CPU cost was reduced by more than a factor of two (perhaps a |
| .. % factor of five), lightweight profiler module is all that must be |
| .. % loaded, and the report generating module (\module{pstats}) is not needed |
| .. % during profiling. |
| .. % |
| .. % \item[Recursive functions support:] |
| .. % Cumulative times in recursive functions are correctly calculated; |
| .. % recursive entries are counted. |
| .. % |
| .. % \item[Large growth in report generating UI:] |
| .. % Distinct profiles runs can be added together forming a comprehensive |
| .. % report; functions that import statistics take arbitrary lists of |
| .. % files; sorting criteria is now based on keywords (instead of 4 integer |
| .. % options); reports shows what functions were profiled as well as what |
| .. % profile file was referenced; output format has been improved. |
| .. % |
| .. % \end{description} |
| |
| |
| .. _profile-instant: |
| |
| Instant User's Manual |
| ===================== |
| |
| This section is provided for users that "don't want to read the manual." It |
| standard entry points and functions. For a more in-depth view of some of the |
| code, consider reading the later section on Profiler Extensions, which includes |
| discussion of how to derive "better" profilers from the classes presented, or |
| reading the source code for these modules. |
| |
| |
| .. function:: run(command[, filename]) |
| |
n | This function takes a single argument that has can be passed to the |
n | This function takes a single argument that can be passed to the |
| :keyword:`exec` statement, and an optional file name. In all cases this routine |
| :keyword:`exec` statement, and an optional file name. In all cases this |
| attempts to :keyword:`exec` its first argument, and gather profiling statistics |
| routine attempts to :keyword:`exec` its first argument, and gather profiling |
| from the execution. If no file name is present, then this function automatically |
| statistics from the execution. If no file name is present, then this function |
| prints a simple profiling report, sorted by the standard name string (file/line |
| automatically prints a simple profiling report, sorted by the standard name |
| /function-name) that is presented in each line. The following is a typical |
| string (file/line/function-name) that is presented in each line. The |
| output from such a call:: |
| following is a typical output from such a call:: |
| |
| 2706 function calls (2004 primitive calls) in 4.504 CPU seconds |
| |
| Ordered by: standard name |
| |
| ncalls tottime percall cumtime percall filename:lineno(function) |
| 2 0.006 0.003 0.953 0.477 pobject.py:75(save_objects) |
| 43/3 0.533 0.012 0.749 0.250 pobject.py:99(evaluate) |
| ... |
| |
| The first line indicates that 2706 calls were monitored. Of those calls, 2004 |
| were :dfn:`primitive`. We define :dfn:`primitive` to mean that the call was not |
| induced via recursion. The next line: ``Ordered by: standard name``, indicates |
| that the text string in the far right column was used to sort the output. The |
| column headings include: |
| |
n | ncalls |
n | ncalls |
| for the number of calls, |
| |
n | tottime |
n | tottime |
| for the total time spent in the given function (and excluding time made in calls |
| to sub-functions), |
| |
n | percall |
n | percall |
| is the quotient of ``tottime`` divided by ``ncalls`` |
| |
n | cumtime |
n | cumtime |
| is the total time spent in this and all subfunctions (from invocation till |
| exit). This figure is accurate *even* for recursive functions. |
| |
n | percall |
n | percall |
| is the quotient of ``cumtime`` divided by primitive calls |
| |
n | filename:lineno(function) |
n | filename:lineno(function) |
| provides the respective data of each function |
| |
| When there are two numbers in the first column (for example, ``43/3``), then the |
| latter is the number of primitive calls, and the former is the actual number of |
| calls. Note that when the function does not recurse, these two values are the |
| same, and only the single figure is printed. |
| |
| |
| 'file', 'line')``. |
| |
| For backward-compatibility reasons, the numeric arguments ``-1``, ``0``, ``1``, |
| and ``2`` are permitted. They are interpreted as ``'stdname'``, ``'calls'``, |
| ``'time'``, and ``'cumulative'`` respectively. If this old style format |
| (numeric) is used, only one sort key (the numeric key) will be used, and |
| additional arguments will be silently ignored. |
| |
n | .. % For compatibility with the old profiler, |
n | .. For compatibility with the old profiler, |
| |
| |
| .. method:: Stats.reverse_order() |
| |
| This method for the :class:`Stats` class reverses the ordering of the basic list |
| within the object. Note that by default ascending vs descending order is |
| properly selected based on the sort key of choice. |
| |
n | .. % This method is provided primarily for |
n | .. This method is provided primarily for compatibility with the old profiler. |
| .. % compatibility with the old profiler. |
| |
| |
| .. method:: Stats.print_stats([restriction, ...]) |
| |
| This method for the :class:`Stats` class prints out a report as described in the |
| :func:`profile.run` definition. |
| |
| The order of the printing is based on the last :meth:`sort_stats` operation done |
| .. method:: Stats.print_callers([restriction, ...]) |
| |
| This method for the :class:`Stats` class prints a list of all functions that |
| called each function in the profiled database. The ordering is identical to |
| that provided by :meth:`print_stats`, and the definition of the restricting |
| argument is also identical. Each caller is reported on its own line. The |
| format differs slightly depending on the profiler that produced the stats: |
| |
n | * With :mod:`profile`, a number is shown in parentheses after each caller to |
n | * With :mod:`profile`, a number is shown in parentheses after each caller to |
| show how many times this specific call was made. For convenience, a second non- |
| show how many times this specific call was made. For convenience, a second |
| parenthesized number repeats the cumulative time spent in the function at the |
| non-parenthesized number repeats the cumulative time spent in the function |
| right. |
| at the right. |
| |
n | * With :mod:`cProfile`, each caller is preceeded by three numbers: the number of |
n | * With :mod:`cProfile`, each caller is preceded by three numbers: the number of |
| times this specific call was made, and the total and cumulative times spent in |
| the current function while it was invoked by this specific caller. |
| |
| |
| .. method:: Stats.print_callees([restriction, ...]) |
| |
| This method for the :class:`Stats` class prints a list of all function that were |
| called by the indicated function. Aside from this reversal of direction of |