| |
| .. module:: asynchat |
| :synopsis: Support for asynchronous command/response protocols. |
| .. moduleauthor:: Sam Rushing <rushing@nightmare.com> |
| .. sectionauthor:: Steve Holden <sholden@holdenweb.com> |
| |
| |
| This module builds on the :mod:`asyncore` infrastructure, simplifying |
n | asynchronous clients and servers and making it easier to handle protocols whose |
n | asynchronous clients and servers and making it easier to handle protocols |
| elements are terminated by arbitrary strings, or are of variable length. |
| whose elements are terminated by arbitrary strings, or are of variable length. |
| :mod:`asynchat` defines the abstract class :class:`async_chat` that you |
| subclass, providing implementations of the :meth:`collect_incoming_data` and |
| :meth:`found_terminator` methods. It uses the same asynchronous loop as |
n | :mod:`asyncore`, and the two types of channel, :class:`asyncore.dispatcher` and |
n | :mod:`asyncore`, and the two types of channel, :class:`asyncore.dispatcher` |
| :class:`asynchat.async_chat`, can freely be mixed in the channel map. Typically |
| and :class:`asynchat.async_chat`, can freely be mixed in the channel map. |
| an :class:`asyncore.dispatcher` server channel generates new |
| Typically an :class:`asyncore.dispatcher` server channel generates new |
| :class:`asynchat.async_chat` channel objects as it receives incoming connection |
| :class:`asynchat.async_chat` channel objects as it receives incoming |
| requests. |
| connection requests. |
| |
| |
| .. class:: async_chat() |
| |
| This class is an abstract subclass of :class:`asyncore.dispatcher`. To make |
| practical use of the code you must subclass :class:`async_chat`, providing |
n | meaningful :meth:`collect_incoming_data` and :meth:`found_terminator` methods. |
n | meaningful :meth:`collect_incoming_data` and :meth:`found_terminator` |
| methods. |
| The :class:`asyncore.dispatcher` methods can be used, although not all make |
| sense in a message/response context. |
| |
n | Like :class:`asyncore.dispatcher`, :class:`async_chat` defines a set of events |
n | Like :class:`asyncore.dispatcher`, :class:`async_chat` defines a set of |
| that are generated by an analysis of socket conditions after a :cfunc:`select` |
| events that are generated by an analysis of socket conditions after a |
| call. Once the polling loop has been started the :class:`async_chat` object's |
| :cfunc:`select` call. Once the polling loop has been started the |
| methods are called by the event-processing framework with no action on the part |
| :class:`async_chat` object's methods are called by the event-processing |
| of the programmer. |
| framework with no action on the part of the programmer. |
| |
n | Two class attributes can be modified, to improve performance, or possibly |
| even to conserve memory. |
| |
| |
| .. data:: ac_in_buffer_size |
| |
| The asynchronous input buffer size (default ``4096``). |
| |
| |
| .. data:: ac_out_buffer_size |
| |
| The asynchronous output buffer size (default ``4096``). |
| |
| Unlike :class:`asyncore.dispatcher`, :class:`async_chat` allows you to define a |
| Unlike :class:`asyncore.dispatcher`, :class:`async_chat` allows you to |
| first-in-first-out queue (fifo) of *producers*. A producer need have only one |
| define a first-in-first-out queue (fifo) of *producers*. A producer need |
| method, :meth:`more`, which should return data to be transmitted on the channel. |
| have only one method, :meth:`more`, which should return data to be |
| transmitted on the channel. |
| The producer indicates exhaustion (*i.e.* that it contains no more data) by |
| having its :meth:`more` method return the empty string. At this point the |
n | :class:`async_chat` object removes the producer from the fifo and starts using |
n | :class:`async_chat` object removes the producer from the fifo and starts |
| the next producer, if any. When the producer fifo is empty the |
| using the next producer, if any. When the producer fifo is empty the |
| :meth:`handle_write` method does nothing. You use the channel object's |
n | :meth:`set_terminator` method to describe how to recognize the end of, or an |
n | :meth:`set_terminator` method to describe how to recognize the end of, or |
| important breakpoint in, an incoming transmission from the remote endpoint. |
| an important breakpoint in, an incoming transmission from the remote |
| endpoint. |
| |
| To build a functioning :class:`async_chat` subclass your input methods |
n | :meth:`collect_incoming_data` and :meth:`found_terminator` must handle the data |
n | :meth:`collect_incoming_data` and :meth:`found_terminator` must handle the |
| that the channel receives asynchronously. The methods are described below. |
| data that the channel receives asynchronously. The methods are described |
| below. |
| |
| |
| .. method:: async_chat.close_when_done() |
| |
n | Pushes a ``None`` on to the producer fifo. When this producer is popped off the |
n | Pushes a ``None`` on to the producer fifo. When this producer is popped off |
| fifo it causes the channel to be closed. |
| the fifo it causes the channel to be closed. |
| |
| |
| .. method:: async_chat.collect_incoming_data(data) |
| |
n | Called with *data* holding an arbitrary amount of received data. The default |
n | Called with *data* holding an arbitrary amount of received data. The |
| method, which must be overridden, raises a :exc:`NotImplementedError` exception. |
| default method, which must be overridden, raises a |
| :exc:`NotImplementedError` exception. |
| |
| |
| .. method:: async_chat._collect_incoming_data(data) |
| |
| Sample implementation of a data collection rutine to be used in conjunction |
| with :meth:`_get_data` in a user-specified :meth:`found_terminator`. |
| |
| |
| .. method:: async_chat.discard_buffers() |
| |
n | In emergencies this method will discard any data held in the input and/or output |
n | In emergencies this method will discard any data held in the input and/or |
| buffers and the producer fifo. |
| output buffers and the producer fifo. |
| |
| |
| .. method:: async_chat.found_terminator() |
| |
n | Called when the incoming data stream matches the termination condition set by |
n | Called when the incoming data stream matches the termination condition set |
| :meth:`set_terminator`. The default method, which must be overridden, raises a |
| by :meth:`set_terminator`. The default method, which must be overridden, |
| :exc:`NotImplementedError` exception. The buffered input data should be |
| raises a :exc:`NotImplementedError` exception. The buffered input data |
| available via an instance attribute. |
| should be available via an instance attribute. |
| |
| |
| .. method:: async_chat._get_data() |
| |
| Will return and clear the data received with the sample |
| :meth:`_collect_incoming_data` implementation. |
| |
| |
| .. method:: async_chat.get_terminator() |
| |
| Returns the current terminator for the channel. |
| |
| |
| .. method:: async_chat.handle_close() |
| |
| Called when the channel is closed. The default method silently closes the |
| channel's socket. |
| |
| |
| .. method:: async_chat.handle_read() |
| |
n | Called when a read event fires on the channel's socket in the asynchronous loop. |
n | Called when a read event fires on the channel's socket in the asynchronous |
| The default method checks for the termination condition established by |
| loop. The default method checks for the termination condition established |
| :meth:`set_terminator`, which can be either the appearance of a particular |
| by :meth:`set_terminator`, which can be either the appearance of a |
| string in the input stream or the receipt of a particular number of characters. |
| particular string in the input stream or the receipt of a particular number |
| When the terminator is found, :meth:`handle_read` calls the |
| of characters. When the terminator is found, :meth:`handle_read` calls the |
| :meth:`found_terminator` method after calling :meth:`collect_incoming_data` with |
| :meth:`found_terminator` method after calling :meth:`collect_incoming_data` |
| any data preceding the terminating condition. |
| with any data preceding the terminating condition. |
| |
| |
| .. method:: async_chat.handle_write() |
| |
n | Called when the application may write data to the channel. The default method |
n | Called when the application may write data to the channel. The default |
| calls the :meth:`initiate_send` method, which in turn will call |
| method calls the :meth:`initiate_send` method, which in turn will call |
| :meth:`refill_buffer` to collect data from the producer fifo associated with the |
| :meth:`refill_buffer` to collect data from the producer fifo associated |
| channel. |
| with the channel. |
| |
| |
| .. method:: async_chat.push(data) |
| |
n | Creates a :class:`simple_producer` object (*see below*) containing the data and |
n | Creates a :class:`simple_producer` object (*see below*) containing the data |
| pushes it on to the channel's ``producer_fifo`` to ensure its transmission. This |
| and pushes it on to the channel's ``producer_fifo`` to ensure its |
| is all you need to do to have the channel write the data out to the network, |
| transmission. This is all you need to do to have the channel write the |
| although it is possible to use your own producers in more complex schemes to |
| data out to the network, although it is possible to use your own producers |
| implement encryption and chunking, for example. |
| in more complex schemes to implement encryption and chunking, for example. |
| |
| |
| .. method:: async_chat.push_with_producer(producer) |
| |
n | Takes a producer object and adds it to the producer fifo associated with the |
n | Takes a producer object and adds it to the producer fifo associated with |
| channel. When all currently-pushed producers have been exhausted the channel |
| the channel. When all currently-pushed producers have been exhausted the |
| will consume this producer's data by calling its :meth:`more` method and send |
| channel will consume this producer's data by calling its :meth:`more` |
| the data to the remote endpoint. |
| method and send the data to the remote endpoint. |
| |
| |
| .. method:: async_chat.readable() |
| |
n | Should return ``True`` for the channel to be included in the set of channels |
n | Should return ``True`` for the channel to be included in the set of |
| tested by the :cfunc:`select` loop for readability. |
| channels tested by the :cfunc:`select` loop for readability. |
| |
| |
| .. method:: async_chat.refill_buffer() |
| |
n | Refills the output buffer by calling the :meth:`more` method of the producer at |
n | Refills the output buffer by calling the :meth:`more` method of the |
| the head of the fifo. If it is exhausted then the producer is popped off the |
| producer at the head of the fifo. If it is exhausted then the producer is |
| fifo and the next producer is activated. If the current producer is, or becomes, |
| popped off the fifo and the next producer is activated. If the current |
| ``None`` then the channel is closed. |
| producer is, or becomes, ``None`` then the channel is closed. |
| |
| |
| .. method:: async_chat.set_terminator(term) |
| |
n | Sets the terminating condition to be recognised on the channel. ``term`` may be |
n | Sets the terminating condition to be recognized on the channel. ``term`` |
| any of three types of value, corresponding to three different ways to handle |
| may be any of three types of value, corresponding to three different ways |
| incoming protocol data. |
| to handle incoming protocol data. |
| |
| +-----------+---------------------------------------------+ |
| | term | Description | |
| +===========+=============================================+ |
| | *string* | Will call :meth:`found_terminator` when the | |
| | | string is found in the input stream | |
| +-----------+---------------------------------------------+ |
| | *integer* | Will call :meth:`found_terminator` when the | |
| | | indicated number of characters have been | |
| | | received | |
| +-----------+---------------------------------------------+ |
| | ``None`` | The channel continues to collect data | |
| | | forever | |
| +-----------+---------------------------------------------+ |
| |
n | Note that any data following the terminator will be available for reading by the |
n | Note that any data following the terminator will be available for reading |
| channel after :meth:`found_terminator` is called. |
| by the channel after :meth:`found_terminator` is called. |
| |
| |
| .. method:: async_chat.writable() |
| |
| Should return ``True`` as long as items remain on the producer fifo, or the |
| channel is connected and the channel's output buffer is non-empty. |
| |
| |
| asynchat - Auxiliary Classes and Functions |
| ------------------------------------------ |
| |
| |
| .. class:: simple_producer(data[, buffer_size=512]) |
| |
n | A :class:`simple_producer` takes a chunk of data and an optional buffer size. |
n | A :class:`simple_producer` takes a chunk of data and an optional buffer |
| Repeated calls to its :meth:`more` method yield successive chunks of the data no |
| size. Repeated calls to its :meth:`more` method yield successive chunks of |
| larger than *buffer_size*. |
| the data no larger than *buffer_size*. |
| |
| |
n | .. method:: simple_producer.more() |
n | .. method:: more() |
| |
n | Produces the next chunk of information from the producer, or returns the empty |
n | Produces the next chunk of information from the producer, or returns the |
| string. |
| empty string. |
| |
| |
| .. class:: fifo([list=None]) |
| |
n | Each channel maintains a :class:`fifo` holding data which has been pushed by the |
n | Each channel maintains a :class:`fifo` holding data which has been pushed |
| application but not yet popped for writing to the channel. A :class:`fifo` is a |
| by the application but not yet popped for writing to the channel. A |
| list used to hold data and/or producers until they are required. If the *list* |
| :class:`fifo` is a list used to hold data and/or producers until they are |
| argument is provided then it should contain producers or data items to be |
| required. If the *list* argument is provided then it should contain |
| written to the channel. |
| producers or data items to be written to the channel. |
| |
| |
n | .. method:: fifo.is_empty() |
n | .. method:: is_empty() |
| |
n | Returns ``True`` iff the fifo is empty. |
n | Returns ``True`` if and only if the fifo is empty. |
| |
| |
n | .. method:: fifo.first() |
n | .. method:: first() |
| |
n | Returns the least-recently :meth:`push`\ ed item from the fifo. |
n | Returns the least-recently :meth:`push`\ ed item from the fifo. |
| |
| |
n | .. method:: fifo.push(data) |
n | .. method:: push(data) |
| |
n | Adds the given data (which may be a string or a producer object) to the producer |
n | Adds the given data (which may be a string or a producer object) to the |
| fifo. |
| producer fifo. |
| |
| |
n | .. method:: fifo.pop() |
n | .. method:: pop() |
| |
n | If the fifo is not empty, returns ``True, first()``, deleting the popped item. |
n | If the fifo is not empty, returns ``True, first()``, deleting the popped |
| Returns ``False, None`` for an empty fifo. |
| item. Returns ``False, None`` for an empty fifo. |
| |
| The :mod:`asynchat` module also defines one utility function, which may be of |
| use in network and textual analysis operations. |
| |
| |
| .. function:: find_prefix_at_end(haystack, needle) |
| |
n | Returns ``True`` if string *haystack* ends with any non-empty prefix of string |
n | Returns ``True`` if string *haystack* ends with any non-empty prefix of |
| *needle*. |
| string *needle*. |
| |
| |
| .. _asynchat-example: |
| |
| asynchat Example |
| ---------------- |
| |
| The following partial example shows how HTTP requests can be read with |
n | :class:`async_chat`. A web server might create an :class:`http_request_handler` |
n | :class:`async_chat`. A web server might create an |
| object for each incoming client connection. Notice that initially the channel |
| :class:`http_request_handler` object for each incoming client connection. |
| terminator is set to match the blank line at the end of the HTTP headers, and a |
| Notice that initially the channel terminator is set to match the blank line at |
| flag indicates that the headers are being read. |
| the end of the HTTP headers, and a flag indicates that the headers are being |
| read. |
| |
n | Once the headers have been read, if the request is of type POST (indicating that |
n | Once the headers have been read, if the request is of type POST (indicating |
| further data are present in the input stream) then the ``Content-Length:`` |
| that further data are present in the input stream) then the |
| header is used to set a numeric terminator to read the right amount of data from |
| ``Content-Length:`` header is used to set a numeric terminator to read the |
| the channel. |
| right amount of data from the channel. |
| |
| The :meth:`handle_request` method is called once all relevant input has been |
n | marshalled, after setting the channel terminator to ``None`` to ensure that any |
n | marshalled, after setting the channel terminator to ``None`` to ensure that |
| extraneous data sent by the web client are ignored. :: |
| any extraneous data sent by the web client are ignored. :: |
| |
| class http_request_handler(asynchat.async_chat): |
| |
n | def __init__(self, conn, addr, sessions, log): |
n | def __init__(self, sock, addr, sessions, log): |
| asynchat.async_chat.__init__(self, conn=conn) |
| asynchat.async_chat.__init__(self, sock=sock) |
| self.addr = addr |
| self.sessions = sessions |
| self.ibuffer = [] |
| self.obuffer = "" |
| self.set_terminator("\r\n\r\n") |
| self.reading_headers = True |
| self.handling = False |
| self.cgi_data = None |