pywebsocket3: A Standalone WebSocket Server for testing purposes

pywebsocket3

A Standalone WebSocket Server for testing purposes

pywebsocket3 is an API that provides WebSocket functionalities with a standalone WebSocket server. It is intended for testing or experimental purposes.

Installation

1. Follow standalone server documentation to start running the standalone server. It can be read by running the following command:

$ pydoc pywebsocket3.standalone

2. Once the standalone server is launched verify it by accessing http://localhost[:port]/console.html. Include the port number when specified on launch. If everything is working correctly, you will see a simple echo console.

Writing WebSocket handlers

When a WebSocket request comes in, the resource name specified in the handshake is considered as if it is a file path under <websock_handlers> and the handler defined in <websock_handlers>/<resource_name>_wsh.py is invoked.

For example, if the resource name is /example/chat, the handler defined in <websock_handlers>/example/chat_wsh.py is invoked.

A WebSocket handler is composed of the following three functions:

web_socket_do_extra_handshake(request) web_socket_transfer_data(request) web_socket_passive_closing_handshake(request)

where:

request: mod_python request.

web_socket_do_extra_handshake is called during the handshake after the headers are successfully parsed and WebSocket properties (ws_origin, and ws_resource) are added to request. A handler can reject the request by raising an exception.

A request object has the following properties that you can use during the extra handshake (web_socket_do_extra_handshake): - ws_resource - ws_origin - ws_version - ws_extensions - ws_deflate - ws_protocol - ws_requested_protocols

The last two are a bit tricky. See the next subsection.

Subprotocol Negotiation

ws_protocol is always set to None when web_socket_do_extra_handshake is called. If ws_requested_protocols is not None, you must choose one subprotocol from this list and set it to ws_protocol.

Data Transfer

web_socket_transfer_data is called after the handshake completed successfully. A handler can receive/send messages from/to the client using request. pywebsocket3.msgutil module provides utilities for data transfer.

You can receive a message by the following statement.

message = request.ws_stream.receive_message()

This call blocks until any complete text frame arrives, and the payload data of the incoming frame will be stored into message. When you’re using IETF HyBi 00 or later protocol, receive_message() will return None on receiving client-initiated closing handshake. When any error occurs, receive_message() will raise some exception.

You can send a message by the following statement.

request.ws_stream.send_message(message)

Closing Connection

Executing the following statement or just return-ing from web_socket_transfer_data cause connection close.

request.ws_stream.close_connection()

close_connection will wait for closing handshake acknowledgement coming from the client. When it couldn’t receive a valid acknowledgement, raises an exception.

web_socket_passive_closing_handshake is called after the server receives incoming closing frame from the client peer immediately. You can specify code and reason by return values. They are sent as a outgoing closing frame from the server. A request object has the following properties that you can use in web_socket_passive_closing_handshake. - ws_close_code - ws_close_reason

Threading

A WebSocket handler must be thread-safe. The standalone server uses threads by default.

Configuring WebSocket Extension Processors

See extensions.py for supported WebSocket extensions. Note that they are unstable and their APIs are subject to change substantially.

A request object has these extension processing related attributes.

  • ws_requested_extensions:

    A list of common.ExtensionParameter instances representing extension parameters received from the client in the client’s opening handshake. You shouldn’t modify it manually.

  • ws_extensions:

    A list of common.ExtensionParameter instances representing extension parameters to send back to the client in the server’s opening handshake. You shouldn’t touch it directly. Instead, call methods on extension processors.

  • ws_extension_processors:

    A list of loaded extension processors. Find the processor for the extension you want to configure from it, and call its methods.

pywebsocket3.common

This file must not depend on any module specific to the WebSocket protocol.

class pywebsocket3.common.ExtensionParameter(name)

This is exchanged on extension negotiation in opening handshake.

add_parameter(name, value)

Add a parameter.

get_parameter_names()

Return the names of the parameters.

get_parameter_value(name)

Get the value of a specific parameter.

get_parameters()

Return the parameters.

has_parameter(name)

Test if a parameter exists.

name()

Return the extension name.

exception pywebsocket3.common.ExtensionParsingException(name)

Exception to handle errors in extension parsing.

pywebsocket3.common.format_extension(extension)

Format an ExtensionParameter object.

pywebsocket3.common.format_extensions(extension_list)

Format a list of ExtensionParameter objects.

pywebsocket3.common.parse_extensions(data)

Parse Sec-WebSocket-Extensions header value.

Returns a list of ExtensionParameter objects. Leading LWSes must be trimmed.

pywebsocket3.dispatch

Dispatch WebSocket request.

exception pywebsocket3.dispatch.DispatchException(name, status=404)

Exception in dispatching WebSocket request.

class pywebsocket3.dispatch.Dispatcher(root_dir, scan_dir=None, allow_handlers_outside_root_dir=True, handler_encoding=None)

Dispatches WebSocket requests.

This class maintains a map from resource name to handlers.

add_resource_path_alias(alias_resource_path, existing_resource_path)

Add resource path alias.

Once added, request to alias_resource_path would be handled by handler registered for existing_resource_path.

Parameters:

  • alias_resource_path – alias resource path

  • existing_resource_path – existing resource path

do_extra_handshake(request)

Do extra checking in WebSocket handshake.

Select a handler based on request.uri and call its web_socket_do_extra_handshake function.

Parameters:

request – mod_python request.

Raises:
get_handler_suite(resource)

Retrieves two handlers (one for extra handshake processing, and one for data transfer) for the given request as a HandlerSuite object.

passive_closing_handshake(request)

Prepare code and reason for responding client initiated closing handshake.

source_warnings()

Return warnings in sourcing handlers.

transfer_data(request)

Let a handler transfer_data with a WebSocket client.

Select a handler based on request.ws_resource and call its web_socket_transfer_data function.

Parameters:

request – mod_python request.

Raises:

pywebsocket3.extensions

class pywebsocket3.extensions.PerMessageDeflateExtensionProcessor(request)

permessage-deflate extension processor.

Specification: http://tools.ietf.org/html/draft-ietf-hybi-permessage-compression-08

set_client_max_window_bits(value)

If this option is specified, this class adds the client_max_window_bits extension parameter to the handshake response, but doesn’t reduce the LZ77 sliding window size of its inflater. I.e., you can use this for testing client implementation but cannot reduce memory usage of this class.

If this method has been called with True and an offer without the client_max_window_bits extension parameter is received,

  • (When processing the permessage-deflate extension) this processor declines the request.

  • (When processing the permessage-compress extension) this processor accepts the request.

set_client_no_context_takeover(value)

If this option is specified, this class adds the client_no_context_takeover extension parameter to the handshake response, but doesn’t reset inflater for each message. I.e., you can use this for testing client implementation but cannot reduce memory usage of this class.

pywebsocket3.extensions.get_extension_processor(extension_request)

Given an ExtensionParameter representing an extension offer received from a client, configures and returns an instance of the corresponding extension processor class.

pywebsocket3.handshake

WebSocket opening handshake processor. This class try to apply available opening handshake processors for each protocol version until a connection is successfully established.

exception pywebsocket3.handshake.AbortedByUserException

Exception for aborting a connection intentionally.

If this exception is raised in do_extra_handshake handler, the connection will be abandoned. No other WebSocket or HTTP(S) handler will be invoked.

If this exception is raised in transfer_data_handler, the connection will be closed without closing handshake. No other WebSocket or HTTP(S) handler will be invoked.

exception pywebsocket3.handshake.HandshakeException(name, status=None)

This exception will be raised when an error occurred while processing WebSocket initial handshake.

exception pywebsocket3.handshake.VersionException(name, supported_versions='')

This exception will be raised when a version of client request does not match with version the server supports.

pywebsocket3.handshake.do_handshake(request, dispatcher)

Performs WebSocket handshake.

Parameters:

  • request – mod_python request.

  • dispatcher – Dispatcher (dispatch.Dispatcher).

Handshaker will add attributes such as ws_resource in performing handshake.

pywebsocket3.request_handler

Request Handler and Request/Connection classes for standalone server.

class pywebsocket3.request_handler.WebSocketRequestHandler(request, client_address, server)

CGIHTTPRequestHandler specialized for WebSocket.

MessageClass

alias of HTTPMessage

is_cgi()

Test whether self.path corresponds to a CGI script.

Add extra check that self.path doesn’t contains .. Also check if the file is a executable file or not. If the file is not executable, it is handled as static file or dir rather than a CGI script.

log_error(*args)

Override BaseHTTPServer.log_error.

log_request(code='-', size='-')

Override BaseHTTPServer.log_request.

parse_request()

Override BaseHTTPServer.BaseHTTPRequestHandler.parse_request.

Return True to continue processing for HTTP(S), False otherwise.

See BaseHTTPRequestHandler.handle_one_request method which calls this method to understand how the return value will be handled.

setup()

Override SocketServer.StreamRequestHandler.setup to wrap rfile with MemorizingFile.

This method will be called by BaseRequestHandler’s constructor before calling BaseHTTPRequestHandler.handle. BaseHTTPRequestHandler.handle will call BaseHTTPRequestHandler.handle_one_request and it will call WebSocketRequestHandler.parse_request.

pywebsocket3.stream

This file provides classes and helper functions for parsing/building frames of the WebSocket protocol (RFC 6455).

Specification: http://tools.ietf.org/html/rfc6455

exception pywebsocket3.stream.BadOperationException

This exception will be raised when send_message() is called on server-terminated connection or receive_message() is called on client-terminated connection.

exception pywebsocket3.stream.ConnectionTerminatedException

This exception will be raised when a connection is terminated unexpectedly.

class pywebsocket3.stream.FragmentedFrameBuilder(mask, frame_filters=[], encode_utf8=True)

A stateful class to send a message as fragments.

exception pywebsocket3.stream.InvalidFrameException

This exception will be raised when we received an invalid frame we cannot parse.

exception pywebsocket3.stream.InvalidUTF8Exception

This exception will be raised when we receive a text frame which contains invalid UTF-8 strings.

class pywebsocket3.stream.Stream(request, options)

A class for parsing/building frames of the WebSocket protocol (RFC 6455).

close_connection(code=1000, reason='', wait_response=True)

Closes a WebSocket connection. Note that this method blocks until it receives acknowledgement to the closing handshake.

Parameters:

  • code – Status code for close frame. If code is None, a close frame with empty body will be sent.

  • reason – string representing close reason.

  • wait_response – True when caller want to wait the response.

Raises:

BadOperationException – when reason is specified with code None or reason is not an instance of both str and unicode.

get_last_received_opcode()

Returns the opcode of the WebSocket message which the last received frame belongs to. The return value is valid iff immediately after receive_message call.

receive_bytes(length)

Receives multiple bytes. Retries read when we couldn’t receive the specified amount. This method returns byte strings.

Raises:

ConnectionTerminatedException – when read returns empty string.

receive_filtered_frame()

Receives a frame and applies frame filters and message filters. The frame to be received must satisfy following conditions: - The frame is not fragmented. - The opcode of the frame is TEXT or BINARY.

DO NOT USE this method except for testing purpose.

receive_message()

Receive a WebSocket frame and return its payload as a text in unicode or a binary in str.

Returns:

payload data of the frame - as unicode instance if received text frame - as str instance if received binary frame or None iff received closing handshake.

Raises:
send_message(message, end=True, binary=False)

Send message.

Parameters:

  • message – text in unicode or binary in str to send.

  • binary – send message as binary frame.

Raises:

BadOperationException – when called on a server-terminated connection or called with inconsistent message type or binary parameter.

class pywebsocket3.stream.StreamOptions

Holds option values to configure Stream objects.

exception pywebsocket3.stream.UnsupportedFrameException

This exception will be raised when we receive a frame with flag, opcode we cannot handle. Handlers can just catch and ignore this exception and call receive_message() again to continue processing the next frame.

pywebsocket3.stream.create_binary_frame(message, opcode=2, fin=1, mask=False, frame_filters=[])

Creates a simple binary frame with no extension, reserved bit.

pywebsocket3.stream.create_header(opcode, payload_length, fin, rsv1, rsv2, rsv3, mask)

Creates a frame header.

Raises:

Exception – when bad data is given.

pywebsocket3.stream.create_length_header(length, mask)

Creates a length header.

Parameters:

  • length – Frame length. Must be less than 2^63.

  • mask – Mask bit. Must be boolean.

Raises:

ValueError – when bad data is given.

pywebsocket3.stream.create_text_frame(message, opcode=1, fin=1, mask=False, frame_filters=[])

Creates a simple text frame with no extension, reserved bit.

class pywebsocket3.stream.deque

deque([iterable[, maxlen]]) –> deque object

A list-like sequence optimized for data accesses near its endpoints.

append()

Add an element to the right side of the deque.

appendleft()

Add an element to the left side of the deque.

clear()

Remove all elements from the deque.

copy()

Return a shallow copy of a deque.

count()

D.count(value) – return number of occurrences of value

extend()

Extend the right side of the deque with elements from the iterable

extendleft()

Extend the left side of the deque with elements from the iterable

index()

D.index(value, [start, [stop]]) – return first index of value. Raises ValueError if the value is not present.

insert()

D.insert(index, object) – insert object before index

maxlen

maximum size of a deque or None if unbounded

pop()

Remove and return the rightmost element.

popleft()

Remove and return the leftmost element.

remove()

D.remove(value) – remove first occurrence of value.

reverse()

D.reverse() – reverse IN PLACE

rotate()

Rotate the deque n steps to the right (default n=1). If n is negative, rotates left.

pywebsocket3.stream.parse_frame(receive_bytes, logger=None, ws_version=13, unmask_receive=True)

Parses a frame. Returns a tuple containing each header field and payload.

Parameters:

  • receive_bytes – a function that reads frame data from a stream or something similar. The function takes length of the bytes to be read. The function must raise ConnectionTerminatedException if there is not enough data to be read.

  • logger – a logging object.

  • ws_version – the version of WebSocket protocol.

  • unmask_receive – unmask received frames. When received unmasked frame, raises InvalidFrameException.

Raises:

pywebsocket3.http_header_util

Utilities for parsing and formatting headers that follow the grammar defined in HTTP RFC http://www.ietf.org/rfc/rfc2616.txt.

pywebsocket3.http_header_util.consume(state, amount=1)

Consumes specified amount of bytes from the head and returns the consumed bytes. If there’s not enough bytes to consume, returns None.

pywebsocket3.http_header_util.consume_lws(state)

Consumes a LWS from the head. Returns True if any LWS is consumed. Otherwise, returns False.

LWS = [CRLF] 1*( SP | HT )

pywebsocket3.http_header_util.consume_lwses(state)

Consumes *LWS from the head.

pywebsocket3.http_header_util.consume_string(state, expected)

Given a parsing state and a expected string, consumes the string from the head. Returns True if consumed successfully. Otherwise, returns False.

pywebsocket3.http_header_util.consume_token(state)

Consumes a token from the head. Returns the token or None if no token was found.

pywebsocket3.http_header_util.consume_token_or_quoted_string(state)

Consumes a token or a quoted-string, and returns the token or unquoted string. If no token or quoted-string was found, returns None.

pywebsocket3.http_header_util.parse_uri(uri)

Parse absolute URI then return host, port and resource.

pywebsocket3.http_header_util.peek(state, pos=0)

Peeks the character at pos from the head of data.

pywebsocket3.http_header_util.quote_if_necessary(s)

Quotes arbitrary string into quoted-string.

pywebsocket3.msgutil

Message related utilities.

Note: request.connection.write/read are used in this module, even though mod_python document says that they should be used only in connection handlers. Unfortunately, we have no other options. For example, request.write/read are not suitable because they don’t allow direct raw bytes writing/reading.

class pywebsocket3.msgutil.MessageReceiver(request, onmessage=None)

This class receives messages from the client.

This class provides three ways to receive messages: blocking, non-blocking, and via callback. Callback has the highest precedence.

Note: This class should not be used with the standalone server for wss because pyOpenSSL used by the server raises a fatal error if the socket is accessed from multiple threads.

receive()

Receive a message from the channel, blocking.

Returns:

message as a unicode string.

receive_nowait()

Receive a message from the channel, non-blocking.

Returns:

message as a unicode string if available. None otherwise.

run()

Method representing the thread’s activity.

You may override this method in a subclass. The standard run() method invokes the callable object passed to the object’s constructor as the target argument, if any, with sequential and keyword arguments taken from the args and kwargs arguments, respectively.

stop()

Request to stop this instance.

The instance will be stopped after receiving the next message. This method may not be very useful, but there is no clean way in Python to forcefully stop a running thread.

class pywebsocket3.msgutil.MessageSender(request)

This class sends messages to the client.

This class provides both synchronous and asynchronous ways to send messages.

Note: This class should not be used with the standalone server for wss because pyOpenSSL used by the server raises a fatal error if the socket is accessed from multiple threads.

run()

Method representing the thread’s activity.

You may override this method in a subclass. The standard run() method invokes the callable object passed to the object’s constructor as the target argument, if any, with sequential and keyword arguments taken from the args and kwargs arguments, respectively.

send(message)

Send a message, blocking.

send_nowait(message)

Send a message, non-blocking.

pywebsocket3.msgutil.close_connection(request)

Close connection.

Parameters:

request – mod_python request.

pywebsocket3.msgutil.receive_message(request)

Receive a WebSocket frame and return its payload as a text in unicode or a binary in str.

Parameters:

request – mod_python request.

Raises:
pywebsocket3.msgutil.send_message(request, payload_data, end=True, binary=False)

Send a message (or part of a message).

Parameters:

  • request – mod_python request.

  • payload_data – unicode text or str binary to send.

  • end – True to terminate a message. False to send payload_data as part of a message that is to be terminated by next or later send_message call with end=True.

  • binary – send payload_data as binary frame(s).

Raises:

BadOperationException – when server already terminated.

pywebsocket3.util

WebSocket utilities.

class pywebsocket3.util.DeflateSocket(socket)

A wrapper class for socket object to intercept send and recv to perform deflate compression and decompression transparently.

recv(size)

Receives data from the socket specified on the construction up to the specified size. Once any data is available, returns it even if it’s smaller than the specified size.

class pywebsocket3.util.NoopMasker

A NoOp masking object.

This has the same interface as RepeatedXorMasker but just returns the string passed in without making any change.

mask(s)

NoOp.

class pywebsocket3.util.RepeatedXorMasker(masking_key)

A masking object that applies XOR on the string.

Applies XOR on the byte string given to mask method with the masking bytes given to the constructor repeatedly. This object remembers the position in the masking bytes the last mask method call ended and resumes from that point on the next mask method call.

mask(s)

Perform the mask via python.

pywebsocket3.util.get_class_logger(o)

Return the logging class information.

pywebsocket3.util.get_script_interp(script_path, cygwin_path=None)

Get #!-interpreter command line from the script.

It also fixes command path. When Cygwin Python is used, e.g. in WebKit, it could run “/usr/bin/perl -wT hello.pl”. When Win32 Python is used, e.g. in Chromium, it couldn’t. So, fix “/usr/bin/perl” to “<cygwin_path>perl.exe”.

Parameters:

  • script_path – pathname of the script

  • cygwin_path – directory name of cygwin binary, or None

Returns:

#!-interpreter command line, or None if it is not #!-script.

pywebsocket3.util.pack_byte(b)

Pack an integer to network-ordered byte

pywebsocket3.util.prepend_message_to_exception(message, exc)

Prepend message to the exception.

pywebsocket3.memorizingfile

Memorizing file.

A memorizing file wraps a file and memorizes lines read by readline.

class pywebsocket3.memorizingfile.MemorizingFile(file_, max_memorized_lines=9223372036854775807)

MemorizingFile wraps a file and memorizes lines read by readline.

Note that data read by other methods are not memorized. This behavior is good enough for memorizing lines SimpleHTTPServer reads before the control reaches WebSocketRequestHandler.

get_memorized_lines()

Get lines memorized so far.

readline(size=-1)

Override file.readline and memorize the line read.

Note that even if size is specified and smaller than actual size, the whole line will be read out from underlying file object by subsequent readline calls.

pywebsocket3.websocket_server

Standalone WebsocketServer

This file deals with the main module of standalone server. Although it is fine to import this file directly to use WebSocketServer, it is strongly recommended to use standalone.py, since it is intended to act as a skeleton of this module.

class pywebsocket3.websocket_server.WebSocketServer(options)

HTTPServer specialized for WebSocket.

fileno()

Override SocketServer.TCPServer.fileno.

get_request()

Override TCPServer.get_request.

handle_error(request, client_address)

Override SocketServer.handle_error.

serve_forever(poll_interval=0.5)

Override SocketServer.BaseServer.serve_forever.

server_activate()

Override SocketServer.TCPServer.server_activate to enable multiple sockets listen.

server_bind()

Override SocketServer.TCPServer.server_bind to enable multiple sockets bind.

server_close()

Override SocketServer.TCPServer.server_close to enable multiple sockets close.

shutdown()

Override SocketServer.BaseServer.shutdown.

pywebsocket3.server_util

Server related utilities.

class pywebsocket3.server_util.ThreadMonitor(interval_in_sec)
run()

Method representing the thread’s activity.

You may override this method in a subclass. The standard run() method invokes the callable object passed to the object’s constructor as the target argument, if any, with sequential and keyword arguments taken from the args and kwargs arguments, respectively.

pywebsocket3.standalone

Standalone WebSocket server.

Use this file to launch pywebsocket as a standalone server.

BASIC USAGE

Go to the src directory and run

$ python pywebsocket3/standalone.py [-p <ws_port>]

[-w <websock_handlers>] [-d <document_root>]

<ws_port> is the port number to use for ws:// connection.

<document_root> is the path to the root directory of HTML files.

<websock_handlers> is the path to the root directory of WebSocket handlers. If not specified, <document_root> will be used. See __init__.py (or run $ pydoc pywebsocket3) for how to write WebSocket handlers.

For more detail and other options, run

$ python pywebsocket3/standalone.py –help

or see _build_option_parser method below.

For trouble shooting, adding “–log_level debug” might help you.

TRY DEMO

Go to the src directory and run standalone.py with -d option to set the document root to the directory containing example HTMLs and handlers like this:

$ cd src $ PYTHONPATH=. python pywebsocket3/standalone.py -d example

to launch pywebsocket with the sample handler and html on port 80. Open http://localhost/console.html, click the connect button, type something into the text box next to the send button and click the send button. If everything is working, you’ll see the message you typed echoed by the server.

USING TLS

To run the standalone server with TLS support, run it with -t, -k, and -c options. When TLS is enabled, the standalone server accepts only TLS connection.

Note that when ssl module is used and the key/cert location is incorrect, TLS connection silently fails while pyOpenSSL fails on startup.

Example

$ PYTHONPATH=. python pywebsocket3/standalone.py -d example -p 10443 -t -c ../test/cert/cert.pem -k ../test/cert/key.pem

Note that when passing a relative path to -c and -k option, it will be resolved using the document root directory as the base.

USING CLIENT AUTHENTICATION

To run the standalone server with TLS client authentication support, run it with –tls-client-auth and –tls-client-ca options in addition to ones required for TLS support.

Example

$ PYTHONPATH=. python pywebsocket3/standalone.py -d example -p 10443 -t -c ../test/cert/cert.pem -k ../test/cert/key.pem –tls-client-auth –tls-client-ca=../test/cert/cacert.pem

Note that when passing a relative path to –tls-client-ca option, it will be resolved using the document root directory as the base.

CONFIGURATION FILE

You can also write a configuration file and use it by specifying the path to the configuration file by –config option. Please write a configuration file following the documentation of the Python ConfigParser library. Name of each entry must be the long version argument name. E.g. to set log level to debug, add the following line:

log_level=debug

For options which doesn’t take value, please add some fake value. E.g. for –tls option, add the following line:

tls=True

Note that tls will be enabled even if you write tls=False as the value part is fake.

When both a command line argument and a configuration file entry are set for the same configuration item, the command line value will override one in the configuration file.

THREADING

This server is derived from SocketServer.ThreadingMixIn. Hence a thread is used for each request.

SECURITY WARNING

This uses CGIHTTPServer and CGIHTTPServer is not secure. It may execute arbitrary Python code or external programs. It should not be used outside a firewall.