Attachment "registry" to
ticket [481019ffff]
added by
andreas_kupries
2001-11-26 03:57:29.
Registry for [comm] based applications
======================================
Intro
-----
Our goal is to implement interpreter discovery functionality for
[comm] based applications similar to the functionality in existence
for [tk send] based applications.
Properties of the [tk] based registry
-------------------------------------
* Uses the X server as the database of known applications,
through the upload of atoms and associating special
properties with the master Tk toplevel window of an application.
From the above automatically follows that the [tk] registry is user
centric. I.e. each user has its own display, therefore its own X
server and thus a unique location for the placement of registry
information without interfering with other users.
This property of the [tk] registry should be preserved by the [comm]
registry as it automatically provides some amount of protection
against attacks.
Beyond the basics
-----------------
Each registry essentially constitutes a namespace for applications
with a want for communication, sort of a chatroom for
meeting. Depending on the context a per-user space can be to broad, or
not broad enough. How can we do this ?
* An easy to implement restrictive scheme is to have one
application launch a private registry (independent of the
per-user registry) which is then available only to the child
processes to this application. This allows a close knit group
of processes a private room for communication without
disturbing the greater scheme of things. Where are two
alternate behaviours for this private registry: Independent
and semi-private. In the first mode all child processes see
only each other and nothing more. In the second mode the
application holding the private registry also acts as a proxy
to the per-user registry, making all per-user applications
visible to the private group, but not the other way around.
* A broader scheme is to have a per-host registry.
* Another broader scheme is remote communication and remote
registries.
Note that the above schemes are not entirely hierarchical. In the
implementation of the basic registry (see below) I mention that the
users home directory can be shared between multiple machines. This
already allows us to not only implement a per-user-and-host registry,
but also a per-user registry crossing the boundaries of the local
host.
I have currently no good idea how to implement a per-host registry
with good security. The point is that not only applications should be
able to choose whether to offer services to the full host or not, but
the registry should be able to have a say in this too, IOW allowed to
reject registration offers seen as dangerous.
As for accessing remote registries (beyond the distributed per-user
registry) I believe that a proxy-based solution provides better
security. IOW the applications and/or registries on a host talking to
a single local registry application which in turn talks to the
requested remote registries instead of all applications directly
talking to the remote registries they want.
The main problem I currently have is how to provide the information
about these more global registries to the applications and local
registries. Given that this is a system administration issue we can
accept here solutions requiring more manual intervention and setup
than for the simple case of a user just wanting his applications to
talk to each other. Best bet currently seems to be to have
world-readable global configuration files in the directory
"/etc/tcl/comm/" which provide this information to the user
registries.
Now that the namespacing issues of registries are discussed we can
move to another issue, advanced functionality. In contrast to the [tk]
registry which has to make do with the services provided by the X
server we can implement functionality which goes beyond the basic
(de)register of and searching for applications.
One exmaple of such functionality is the ability to ask for and get
notifications when applications (de)register.
Contents of the registry
------------------------
For the normal purposes of an application using the registry to find
other applications it is enough to store the name of a registered
application ( and a serial number if there are several instances of
the same application) together with the connection information (host
and port).
The host information (ip address) is also enough to allow a registry
to purge everything from a single host when the user-registry for that
host is going down.
Implementation of the basic [comm] registry
-------------------------------------------
* The [comm] registry considers the home directory of a user as
a resource unique to that user, possibly shared between
several machines.
* Inside of the home directory [comm] claims the directory
".tcl/comm" for its own purposes, i.e. for the storage of
configuration and state information.
* When [comm] is loaded it looks for a file called
"$HOME/.tcl/comm/registry/<hostname>". This file is called the
"host-specific port file".
(a) [comm] will reads its contents and interprets them as
the port the registry for the current user and host is
listening on. [comm] when tries to connect to that
port. If this is succesful [comm] will establish that
it truly talks to the registry. If this is sucessful
the application will register itself.
If [comm] was unable to connect to a server on that
port, or the server is not a registry [comm] will
assume that there was a registry which lived on that
port, but that this registry is now gone. It will
erase the file and then proceed to become the registry
itself. See (b) and (c).
(b) If the file exists, but is empty [comm] knows that
another application is currently becoming the registry
and will wait until the file is not empty. To prevent
an early read of incomplete information the data in
the file is considered valid if and only if it ends in
a EOL marker.
(c)
If the file is missing [comm] will create it as empty,
thus locking out other contenders (see last paragraph)
and then proceed in becoming the registry itself. To
this end it will start a server on a free listening
port (as chosen by the OS) and then write the number
of the port to the port file, enabling any waiting
contender to proceed and talk to the newly created
registry.
* If the process currently holding the registry dies all
registered applications are notified and will then try to turn
themselves into the new registry. One of them will succeed and
the others will then re-register themselves at the new
registry application.
Here the process of becoming the registry is easier as the
identity of the port to use by the registry is already known
to all applications. The applications will take that
information and just try to start server on that port. The
application which succeeds becomes the registry, thereas the
others will find a server on that port and just reconnect.
* The items above deal with a per-user registry local to a
single host. They are not sufficient when trying to deal with
per-user registry crossing the boundaries of a host to all
machines sharing the users home directory.
We could try to create a truly unique registry which is
accessed by all hosts sharing the users home directory. There
is however the diffcult problem of file locking. The rules
above work good enough for one host accessing its port file,
even if the file resides on a different host, as caching is
the same for all the applications on the host. But when it
comes to accessing the file from different hosts different
cache policies, network latencies etc. make this quite
fragile.
Instead we go for a distributed solution. On each host the
user executes his applications on one of the processes becomes
the per-user registry for that host. As all these registries
share the home directory their host-specific port files are
visisible to each other. This means that the host-specific
registries can watch the port files of the other registries
for changes and are able to communicate with each other. This
we use to distribute the information about (de)registrations
across all registries for the user, creating one (virtual)
host-independent registry for that user.
This solution also works in the case if one of the registries
dies. The other applications on that host will elect a new
registry per the rules above and this registry will then
proceed to open talks to the other registries. If the last
application on a host dies this host simply drops out of the
network until a new application starts and becomes the
registry.
Whenever a part of the distributed registry goes down all
applications registered through that part are removed from the
caches of the other parts. After the reelection of new
registry the cache is filled again through the
reregistrations.
Future: We should make it possible for the parts of the
distributed registry to talk to each other using encrypted
communication lines to enhance the security.
Implementation of the restricted [comm] registry
------------------------------------------------
* The application launching the private registry considers the
environment as a unique resource it is sharing with its child
processes. It relies on the fact that if an application makes
modifications to the environment these changes are visible
only to child processes, but not the parent.
* Inside of the environment the restricted registry claims the
variable "TCL_COMM_REGPORT" for its own purposes, i.e. the
transmission of contact information to its childs.
* A [comm] registry which finds the aforementioned variable in
its environment will try to contact the registry at the port
given as the value of the variable. If this succeeds the
application will register itself. However if this fails the
normal startup process is entered instead.
* If the process holding the registry dies the child processes
will be notified of this. Instead of selecting one of their
kind as the new registry they will do nothing. They can keep
up their current communication, and if they cached the
name/port mapping for their peers even establish new
communication in their group, but they are unable to talk
outside of this group.
Packaging
---------
The configuration functionality of the registry and the management of
all data structures is placed into a separate package,
"comm::registry". This separation allows an application to configure
the behaviour of the registry according to its wishes (like: start a
private registry, no talking to a per-user registry, per-user-and-host
only instead of full per-user, etc.) before the loading of the "comm"
package actually launches the service and then provides access to it.
