[manpage_begin comm_wire n 3]
[copyright {2005 Docs. Andreas Kupries <[email protected]>}]
[moddesc {Remote communication}]
[titledesc {The comm wire protocol}]
[require comm_wire]
[description]
[para]
The [package comm] command provides an inter-interpreter remote
execution facility much like Tk's [cmd send(n)], except that it uses
sockets rather than the X server for the communication path. As a
result, [package comm] works with multiple interpreters, works on
Windows and Macintosh systems, and provides control over the remote
execution path.
[para]
This document contains a specification of the various versions of the
wire protocol used by comm internally for the communication between
its endpoints. It has no relevance to users of [package comm], only to
developers who wish to modify the package, write a compatible facility
in a different language, or some other facility based on the same
protocol.
[comment {
An example of some other facility could be a router layer which is
able to get messages for many different endpoints and then routes
them to the correct one. Why is this interesting ? Because it
allows mesh-routing, i.e. an application fires a command to some
other endpoint without having to worry if there is direct
connection to this endpoint or not. A secure tunnel then neatly
fits into this. Its endpoints are routing agents which take
arbitrariry commands, route them through the tunnel and then
dispatch them to the correct endpoint on the other side.
Note: A special case would be to have such a router facility built
into the core comm package, making each endpoint a router as
well. Like with the ability to listen to for non-local connection
this is something the user should be able to disable.
}]
[comment {
Motivation for documenting the protocol
---------------------------------------
While the comm package allows the transport and execution of arbitrary
Tcl scripts a particular application can use the hooks to restrict the
scripts to single commands, and the legal commands to a specific set
as well.
If this is done (*) comm becomes more of a transport layer for a
regular RPC, and the data transported over the wire is less of a Tcl
script and more of a declaration of which remote procedure is wanted,
plus arguments.
At this point it begins to make sense to have implementations in other
scripting languages. Because then it beomes irrelevant in what
language the server is implemented. The comm protocol becomes a
portable RPC protocol, which can also be used for transport Tcl
scripts when both sides are Tcl and allowing this.
(*) And IMHO it should be done 90% of the time, just to get proper
security. Note that just using a safe interp is not quite enough, as
it still allows aribtrary scripts. The interp has to contains aliases
for the wanted commands, and only them for us to get a large security
wall.
}]
[section {Wire Protocol Version 3}]
[subsection {Basic Layer}]
The basic encoding for [emph all] data is UTF-8. Because of this
binary data, including the NULL character, can be sent over the wire
as is, without the need for armoring it.
[subsection {Basic Message Layer}]
On top of the [sectref {Basic Layer}] we have a
[term {message oriented}] exchange of data. Each [term message] is a
syntactically valid Tcl list terminated with an additional EOL. Note
that EOL characters can occur within the list as well. They can be
distinguished from the terminating EOL by the fact that the data from
the beginning up to their location is not a valid Tcl list.
[para]
EOL is signaled through the character [const "\n"]. This is following
the unix convention for line-endings.
[para]
As a list each message is composed of [term words]. Their meaning
depends on when the message was sent in the overall exchange. This is
described in the section sections.
[subsection {Negotiation Messages - Initial Handshake}]
The command protocol is defined like this:
[list_begin bullet]
[bullet]
The first message send by a client to a server, when opening the
connection, contains two words. The first word is a list as well, and
contains the versions of the wire protocol the client is willing to
accept, with the most prefered version first. The second word is the
TCP port the client is listening on for connections to itself.
[bullet]
The first message sent by the server to the client, in response to the
message above contains only one word. This word is a list, containing
the string [const vers] as its first element, and the version of the
wire protocol the server has selected from the offered versions as the
second.
[comment {
NOTE / DANGER
The terminating EOL for this first response will be the socket
specific default EOL (Windows/Internet convention: "\r\n").
However all future messages use Unix convention, i.e. "\n",
for their EOLs, embedded or terminating.
Reason: The internal command commNewComm does the common
processing for new connections, doing the
fconfigure -translation lf
However the handshake response containing the accepted
version is sent before commNewComm is called (in
commIncoming).
NOTE 2:
This inconsistency has been fixed locally already, but
not been committed yet.
}]
[list_end]
[subsection {Script/Command Messages}]
All messages coming after the
[sectref {Negotiation Messages - Initial Handshake} {initial handshake}]
consist of three words. These are an instruction, a transaction id,
and the payload. The valid instructions are shown below. The
transaction ids are used by the client to match any incoming replies
to the command messages it sent. This means that a server has to copy
the transaction id from a command message to the reply it sends for
that message.
[list_begin definitions]
[lst_item [const send]]
[lst_item [const async]]
[lst_item [const command]]
The payload is the Tcl script to execute on the server. It is actually
a list containing the script fragments. These fragement are
[cmd concat]enated together by the server to form the full script to
execute on the server side.
This emulates the Tcl "eval" semantics.
In most cases it is best to have only one word in the list, a list
containing the exact command.
[nl]
Examples:
[nl]
[example {
(a) {send 1 {{array get tcl_platform}}}
(b) {send 1 {array get tcl_platform}}
(c) {send 1 {array {get tcl_platform}}}
are all valid representations of the same command. They are
generated via
(a') send {array get tcl_platform}
(b') send array get tcl_platform
(c') send array {get tcl_platform}
respectively
}]
[nl]
Note that (a), generated by (a'), is the usual form, if only single
commands are sent by the client.
For example constructed using [cmd list], if the command contains
variable arguments. Like
[nl]
[example {
send [list array get $the_variable]
}]
[nl]
These three instructions all invoke the script on the server
side. Their difference is in the treatment of result values, and thus
determines if a reply is expected.
[list_begin definitions]
[lst_item [const send]]
A reply is expected. The sender is waiting for the result.
[lst_item [const async]]
No reply is expected, the sender has no interest in the result and is
not waiting for any.
[lst_item [const command]]
A reply is expected, but the sender is not waiting for it. It has
arranged to get a process-internal notification when the result
arrives.
[list_end]
[lst_item [const reply]]
Like the previous three command, however the tcl script in the payload
is highly restricted.
It has to be a syntactically valid Tcl [cmd return] command. This
contains result code, value, error code, and error info.
[nl]
Examples:
[nl]
[example {
{reply 1 {return -code 0 {}}}
{reply 1 {return -code 0 {osVersion 2.4.21-99-default byteOrder littleEndian machine i686 platform unix os Linux user andreask wordSize 4}}}
}]
[list_end]
[comment {
Socket Miscellanea
------------------
It is possible to have two sockets between a client and a
server. This happens if both sides connected to each other at
the same time.
Current protocol versions
-------------------------
V2
V3 This is prefered version and uses UTF 8 encoding.
This is actually the only version which will work IIU
the code right. Because the server part of comm will
send the version reply if and only if version 3 was
negotiated.
IOW if v2 is used the client will not see a version
reply during the negotiation handshake.
}]
[keywords socket {remote communication} communication ipc message rpc]
[see_also comm]
[manpage_end]
