| Ticket UUID: | 579633 | ||
| Title: | why are there static paths in libtcl? | ||
| Type: | Bug | Created on: | 2002-07-10 14:05:26 |
| Submitter: | jcw | Assigned to: | dgp |
| Subsystem: | 38. Init - Library - Autoload | Severity: | |
| Priority: | 5 Medium | Last modified: | 2003-11-13 07:58:51 |
| Status: | Open | Closed by: | |
| Resolution: | None | Closed on: | |
| Version: | obsolete: 8.5a0 | ||
| Description: | ||||
There are three places where static path strings are compiled into the executable: - tkFont.c has an assert, which is perhaps somewhat out of place (being the only one)? - there are two static string buffers, with TCL_LIBRARY and TCL_PACKAGE_PATH values passed from the Makefile to tclUnixInit.c Why are these needed? | ||||
| User Comments: | ||||
dgp added on 2002-07-10 21:21:18:
Logged In: YES user_id=80530 First let's get the question right. The paths are compiled into the library itself, and so they will appear in any executable that links that library. I won't address the tkFont.c. I don't know, and that's part of Tk. Ask in the Tk tracker. After an explanation appears here, it would be good to transfer it to be part of TIP 66? dgp added on 2002-07-10 21:55:53:
Logged In: YES user_id=80530 The main product produce by 'make' on the Tcl sources is a binary library linkable into other (C) programs. It may be static or shared. It may have a name matching libtcl*.* or tcl*.dll, but I'll call it "libtcl" Unlike most binary libraries, libtcl doesn't contain everything it needs to perform its functions. It relies on a script library stored somewhere on some filesystem. It also expects to be able to [load] or [source] installed files to extend its functionality. The locations to search for installed extensions/packages are also somewhere on some filesystem. So, libtcl has to know path information in a filesystem for complete operation. In a completely default build/install of Tcl, these are known relative paths, relative to the installed location of libtcl. However, Unix offers no way within an executable to discover the full path of a library linked into that executable. For example, inside a running wish, there's no C function that can be called to find the path to the libtcl*.* that's linked into that wish. So there's no base path off of which to extend the relative paths to "init.tcl" or to the install directories for packages. Instead, the paths themselves must be compiled into the library, *and* one must assume that the library and its support files are in fact installed in the location where ./configure said they would be installed. Contrast this with the Windows initialization where AppendDllPath() is able to call GetModuleFileName() to discover the path to tcl*.dll and base further paths on it. Note the importance of matching a configure-time declaration of where things will be installed to the actual install location determined at install time. This really complicates binary distributions. In fact, the ActiveTcl installer has to solve this by actually *binary patching* the correct install paths into libtcl as it installs it. That's a start at the explanation, but that's limited to the default install. When people get adventurous with their ./configure options, things only get worse. More to come... jcw added on 2002-07-11 08:02:24:
Logged In: YES user_id=1983 Ok, so the shared lib loader loads libtcl8.4.so from *some* spot, but the lib can't figure out where, so it can not find further runtime support scripts without help. I think I understand that. But what if everything is loaded statically, i.e. a single exe? Isn't Tcl_FindExecutable all one needs? Isn't that in fact the whole point of T_FE? dgp added on 2002-07-11 09:22:13:
Logged In: YES user_id=80530 Shared vs. static library has, I believe, nothing to do with it. The Tcl library -- in any form -- needs path information that is only determined during configuration, and cannot be determined at run time, except be having those paths compiled into the library itself. The library can, and does, make lots of guesses about paths, based on various conventions -- relative to the executable location, etc. -- but the only failsafe for all apps that link libtcl, no matter where they are installed, is the compiled-in path. jcw added on 2002-07-11 16:17:17:
Logged In: YES user_id=1983 Don, you said "Shared vs. static library has, I believe, nothing to do with it" - that's why I talked about exe's. Static libs end up in exe's. There *is* no relativeness for a static lib, because incorporating it means it loses its identity. In terms of deployment of binaries, i.e. applications people use, having a requirement on some dir in a fixed spot, regardless where, is bound to lead to trouble. With shared libs, I can see a case for them being in a standard spot, and their support files being in a standard spot as well, though IMO the usual way to do it is either an env var, a registry entry, or a .somethingrc file. But when bound inside an executable, built-in paths make no sense (unless developers choose so, i.e. people linking *to* Tcl, not the Tcl developers themselves). One could keep the current structure for backward-compatibility, but please as an option - not as default build behavior. I can't see binary patching after the fact as anything other than covering up for a design flaw. dkf added on 2002-07-11 16:42:21:
Logged In: YES user_id=79902 Fundamentally, Tcl needs to find three things at startup: the place to source its library-scripts from, the place to grab the encodings from, and the place to get package indices from (the first two are really combined into the same problem by putting the encodings in a subdir.) You can't expect users to set environment variables properly (experience suggests that they tend to get it wrong), the registry is not available on Unix, and there is no guarantee that an RC file will help either. The problems with an RC file are: * In a home directory, it might either not exist or (worse) might be out of date. * /etc is not necessarily writable by all. Plus you've still got the problem of keeping things up to date. * Relative to the prefix (or exec_prefix) is much better, because that can be known to be writable at install time and it should be up-to-date for the library too. But then you have the problem of knowing where to find that RC file! All you've done is rearrange the deckchairs and provided a way for the various parts of an installation to get out of sync. :^( I don't see any nice way out of this. Building bootstrap paths into the library is just the least-sucky way (of a lot of ways that all suck) out of this hole. Perhaps a way forward would be for there to be some kind of pluggable hook that the likes of tclkit could use to help out with the bootstrapping? dgp added on 2002-07-11 20:34:25:
Logged In: YES user_id=80530 My point on focusing on the library, is that these paths get embedded in *every" executable that links linktcl, not just tclsh/wish. Many people miss this and propose "solutions" that involve changes to tclsh/wish and do nothing for those embedding libtcl in other programs. I think you may have misunderstood me on one point. I'm only trying to explain the historical reasons why the embedded paths are there. I'm not trying to defend that design. I do not dispute that it's a design flaw -- one that's caused a substantial amount of grief. But it's not something that can be easily changed. Keep in mind too that I haven't even raised the complexities that arise when TCL_EXEC_PREFIX and TCL_PREFIX are not the same directory. Then there is no known path between different parts of the Tcl installation, and the embedded path is the only thing that works. dgp added on 2002-07-11 20:47:00:
Logged In: YES user_id=80530 Note that one potential way out of this was the notion of an "internal virtual file system" to hold the "files" needed for initializing the Tcl library completely within the library itself. See the bottom of http://mini.net/tcl/2138 dkf added on 2002-11-16 19:53:12:
Logged In: YES user_id=79902 That's not a bad idea. Like that, we could make sure that all the common encodings are present (we already do something vaugely like this for a few of them) while still allowing for extension/overriding of encodings (e.g. where the core is wrong - it's happened in the past - or where we've not defined a particular encoding because it is rare or something like that.) dgp added on 2003-07-11 03:53:16:
Logged In: YES user_id=80530 Should note that TIP 59 now encourages all extensions (not just Tcl itself) to compile into themselves various static paths. | ||||
