Tcl Source Code

View Ticket
Login
Ticket UUID: 579633
Title: why are there static paths in libtcl?
Type: Bug Created on: 2002-07-10 14:05:26
Submitter: jcw Assigned to: dgp
Subsystem: 38. Init - Library - Autoload Severity:
Priority: 5 Medium Last modified: 2003-11-13 07:58:51
Status: Open Closed by:
Resolution: None Closed on:
Version: obsolete: 8.5a0
Description:
There are three places where static path strings are 
compiled into the executable:

- tkFont.c has an assert, which is perhaps somewhat out 
of place (being the only one)?

- there are two static string buffers, with TCL_LIBRARY 
and TCL_PACKAGE_PATH values passed from the Makefile to 
tclUnixInit.c

Why are these needed?
User Comments:
dgp added on 2002-07-10 21:21:18:
Logged In: YES 
user_id=80530


First let's get the question right.

The paths are compiled into the library itself,
and so they will appear in any executable that
links that library.

I won't address the tkFont.c.  I don't know, and
that's part of Tk.  Ask in the Tk tracker.

After an explanation appears here, it would be
good to transfer it to be part of TIP 66?

dgp added on 2002-07-10 21:55:53:
Logged In: YES 
user_id=80530


The main product produce by 'make' on
the Tcl sources is a binary library linkable
into other (C) programs.  It may be static
or shared.  It may have a name matching
libtcl*.* or tcl*.dll, but I'll call it "libtcl"

Unlike most binary libraries, libtcl doesn't
contain everything it needs to perform
its functions.  It relies on a script library
stored somewhere on some filesystem.
It also expects to be able to [load] or
[source] installed files to extend its
functionality.  The locations to search
for installed extensions/packages are
also somewhere on some filesystem.

So, libtcl has to know path information
in a filesystem for complete operation.
In a completely default build/install of
Tcl, these are known relative paths,
relative to the installed location of libtcl.
However, Unix offers no way within
an executable to discover the full path
of a library linked into that executable.
For example, inside a running wish,
there's no C function that can be called
to find the path to the libtcl*.* that's
linked into that wish.  So there's no base
path off of which to extend the relative
paths to "init.tcl" or to the install directories
for packages.  Instead, the paths
themselves must be compiled into the
library, *and* one must assume that
the library and its support files are in fact
installed in the location where ./configure
said they would be installed.

Contrast this with the Windows initialization
where AppendDllPath() is able to call
GetModuleFileName() to discover the path
to tcl*.dll and base further paths on it.

Note the importance of matching a
configure-time declaration of where
things will be installed to the actual
install location determined at install time.
This really complicates binary distributions.
In fact, the ActiveTcl installer has to
solve this by actually *binary patching*
the correct install paths into libtcl as
it installs it.

That's a start at the explanation, but that's
limited to the default install.  When people
get adventurous with their ./configure
options, things only get worse.  More to come...

jcw added on 2002-07-11 08:02:24:
Logged In: YES 
user_id=1983

Ok, so the shared lib loader loads libtcl8.4.so from *some* 
spot, but the lib can't figure out where, so it can not find 
further runtime support scripts without help.  I think I 
understand that.

But what if everything is loaded statically, i.e. a single 
exe?  Isn't Tcl_FindExecutable all one needs?  Isn't that in 
fact the whole point of T_FE?

dgp added on 2002-07-11 09:22:13:
Logged In: YES 
user_id=80530


Shared vs. static library has, I believe, nothing to do with it.

The Tcl library -- in any form -- needs path information that
is only determined during configuration, and cannot be
determined at run time, except be having those paths
compiled into the library itself.

The library can, and does, make lots of guesses about
paths, based on various conventions -- relative to
the executable location, etc. -- but the only failsafe
for all apps that link libtcl, no matter where they are
installed, is the compiled-in path.

jcw added on 2002-07-11 16:17:17:
Logged In: YES 
user_id=1983

Don, you said "Shared vs. static library has, I believe, 
nothing to do with it" - that's why I talked about exe's.

Static libs end up in exe's.  There *is* no relativeness for 
a static lib, because incorporating it means it loses its 
identity.

In terms of deployment of binaries, i.e. applications people 
use, having a requirement on some dir in a fixed spot, 
regardless where, is bound to lead to trouble.

With shared libs, I can see a case for them being in a 
standard spot, and their support files being in a standard 
spot as well, though IMO the usual way to do it is either an 
env var, a registry entry, or a .somethingrc file.

But when bound inside an executable, built-in paths make no 
sense (unless developers choose so, i.e. people linking *to* 
Tcl, not the Tcl developers themselves).  One could keep the 
current structure for backward-compatibility, but please as 
an option - not as default build behavior.

I can't see binary patching after the fact as anything other 
than covering up for a design flaw.

dkf added on 2002-07-11 16:42:21:
Logged In: YES 
user_id=79902

Fundamentally, Tcl needs to find three things at startup:
the place to source its library-scripts from, the place to
grab the encodings from, and the place to get package
indices from (the first two are really combined into the
same problem by putting the encodings in a subdir.)

You can't expect users to set environment variables properly
(experience suggests that they tend to get it wrong), the
registry is not available on Unix, and there is no guarantee
that an RC file will help either.  The problems with an RC
file are:
 * In a home directory, it might either not exist or (worse)
   might be out of date.
 * /etc is not necessarily writable by all.  Plus you've
   still got the problem of keeping things up to date.
 * Relative to the prefix (or exec_prefix) is much better,
   because that can be known to be writable at install time
   and it should be up-to-date for the library too.  But
   then you have the problem of knowing where to find that
   RC file!  All you've done is rearrange the deckchairs and
   provided a way for the various parts of an installation
   to get out of sync.  :^(

I don't see any nice way out of this.  Building bootstrap
paths into the library is just the least-sucky way (of a lot
of ways that all suck) out of this hole.  Perhaps a way
forward would be for there to be some kind of pluggable hook
that the likes of tclkit could use to help out with the
bootstrapping?

dgp added on 2002-07-11 20:34:25:
Logged In: YES 
user_id=80530

My point on focusing on the library, is that these
paths get embedded in *every" executable
that links linktcl, not just tclsh/wish.  Many people
miss this and propose "solutions" that involve
changes to tclsh/wish and do nothing for those
embedding libtcl in other programs.

I think you may have misunderstood me on one
point.  I'm only trying to explain the historical
reasons why the embedded paths are there.
I'm not trying to defend that design.  I do not
dispute that it's a design flaw -- one that's
caused a substantial amount of grief.  But
it's not something that can be easily changed.

Keep in mind too that I haven't even raised
the complexities that arise when TCL_EXEC_PREFIX
and TCL_PREFIX are not the same directory.  Then
there is no known path between different parts of
the Tcl installation, and the embedded path is the
only thing that works.

dgp added on 2002-07-11 20:47:00:
Logged In: YES 
user_id=80530

Note that one potential way out of this
was the notion of an "internal virtual
file system" to hold the "files" needed
for initializing the Tcl library completely
within the library itself.  See the
bottom of

http://mini.net/tcl/2138

dkf added on 2002-11-16 19:53:12:
Logged In: YES 
user_id=79902

That's not a bad idea.  Like that, we could make sure that
all the common encodings are present (we already do
something vaugely like this for a few of them) while still
allowing for extension/overriding of encodings (e.g. where
the core is wrong - it's happened in the past - or where
we've not defined a particular encoding because it is rare
or something like that.)

dgp added on 2003-07-11 03:53:16:
Logged In: YES 
user_id=80530


Should note that TIP 59 now encourages
all extensions (not just Tcl itself) to
compile into themselves various static
paths.