Tcl Source Code

OriginalBugID: 1187 Bug
Version: 8.0.4
SubmitDate: '1999-02-11'
LastModified: '2000-02-02'
Severity: MED
Status: Closed
Submitter: pat
ChangedBy: ericm
RelatedBugIDs: 1115
OS: All
FixedDate: '2000-10-25'
ClosedDate: '2000-02-02'


Name: Jim Ingham

ReproducibleScript:
Make two files:

foo.tcl:
namespace eval foo {
    namespace export bar dada
    proc bar {} {puts "I am here"}
}

dada.tcl:
namespace eval foo {
    proc dada {} {puts "Me too!"}
}

Now, make a tclIndex file for these two files, and then do:

$ tclsh8.0
% lappend auto_path [pwd]
/home/jingham/install/lib/tcl8.0 /home/jingham/install/lib /tmp
% namespace import foo::dada
% dada
invalid command name "dada"
% 

ObservedBehavior:
The command dada should be imported.  After all, it IS on the export list...

DesiredBehavior:
invoking dada should print out

Me too!

This is like the "namespace import is broken in Itcl3.0.1" bug, though 
admittedly more contrived.  The problem is that namespace import uses the 
auto_index to determine what files it should source when doing the 
auto_loading of commands.  To work correctly, it also needs to know what 
the export list for the module was.

I quoted a semi-bogus stab at getting this to work in the Itcl bug report.
The idea is that the export list for namespaces needs to get recorded in
the tclIndex file as well as the list of commands.

This is not a big deal, since people probably tend to put all the code for
one namespace in a single file, or they just blanket import all the exported
functions using

namespace import foo::*

In either case, everything will work fine.  But the bug is there nonetheless...


05/09/1999 23:01 - foster - per Brent, suspects fixed in 8.1.
--
This wasn't fixed per se in 8.1, but I think that this is not
a correct assumption to make of the auto_load mechanism.
Calling dada, or doing the namespace import will not cause
the file that has been auto-indexed to be sourced.  The user
should instead consider making a package with -direct, and
doing package require. 
-- 12/03/1999 hobbs
Of course, that doesn't solve the entire problem.  The real
question is how to handle the case:
    package require Iwidgets
    namespace import ::iwidgets::*
Can it be done with lazy loading, or are we stuck with slower
load times? 
-- 12/06/1999 hobbs

Yes, namespace import *does* cause the auto index'ed file to be loaded.  However, I think it is reasonable to say that this is not how auto indexing is meant to be used; if you have a multi-file package, you should use package.  In that case, you should follow the model used by Iwidgets, which has a main package file that is directly loaded on package require, and sub-files that are auto loaded as needed.  If you do namespace import ::iwidgets::*, that will cause everything to be loaded, but I don't think that's unreasonable.  I don't think there is really an issue here anymore, so long as we have the understanding that auto indexing is for very simple packages only, and anything more complex should use the package mechanism.

Jim, if you think there is still an issue, please respond.

- eric

-- 02/01/2000 ericm

From Brent:

Here is what I did at one point.
There is a namespace command emulation in auto.tcl, right?
I added a bit of code to pay attention to the "namespace export" command,
basically remember the export pattern for a particular file.
I believe I invented an "auto_export" array indexed by ?filename?
Then, inside auto_import, I was somehow able to create a list of
the procedures that would be defined if I were to source a file,
all without actually sourcing it.  This achieves a two-level scheme
where auto_import, which is called in response to 
namespace import iwidgets::*
sets up the auto_index array for all the iwidgets commands.

Ponder this, then I'll drop by and perahps we can recreate the scenario
more fully.

Also, as to the original issue of having a namespace spread across
multiple files, why can't we legislate that every one of those files
has a namespace export command in it?  In fact, I think it is necessary
for the above scheme.

-- 02/02/2000 ericm

From Jim:

 > 
 > On Tue, 1 Feb 2000, James Ingham wrote:
 > 
 > > Well...  I am not sure it is desirable to source in a whole package in
 > > order to do namespace import iwidgets::*.  Over the long term, this
 > > will mean that big packages are a real pain.  It might be okay to
 > > force the package to be broken up, but if I have to source in ALL the
 > > Iwidgets code just to use a couple of megawidgets, then it will give
 > > me more pause about whether I want to use it or not.  It does cause a
 > > noticable hit to the startup time.
 > 
 > Agreed, but if you are not using all of the widgets, why namespace import
 > iwidgets::*?  You could import just those commands that you need, and thus
 > avoid sourcing the whole package.  Iwidgets is already broken up into its
 > component pieces, and I think it's reasonable to think that any
 > sufficiently large package will be similarly split.
I would like to structure my program so that I do all my package
requires and imports in the startup code.  Importing the iwidget
commands bit by bit means that I have to keep adding statements to
this file every time I decide, in some wholly other bit of code, that
I want to use a new bit of the Iwidgets.  This is a pain in the neck.

 > 
 > > So the overall issue is sort of moot, but a real solution to namespace 
 > > importing a package which consists of multiple files, has not, to my
 > > mind, been adequately solved.
 > 
 > Probably true.  One idea that comes to mind is to change the behavior of
 > namespace import to (essentially) lazily source the files, rather than
 > forcing an immediate load.  This requires some serious changes to the auto
 > indexer and the entire command infrastructure, so I don't realistically
 > see something like it happening any time soon.
Actually, there is code in IncrTcl to do just this (it just deals with 
namespaces, and is not IncrTcl specific).  Look at the auto_import
function defined in itcl.tcl, and the Itcl_Stub{Create,Exists}Cmd
functions in itcl_cmds.c.

The problem is that these commands don't work because of
bugs/shortcomings in how the import mechanism works.  Basically, there
is no record of the export list for a namespace other than the export
commands in the source files.  This means that in order to know what
commands match a certain import pattern (remember the match is only
against commands in the export list), you have to source in all the
source files.

Brent & I worked up a fix where the auto_mkindex process would also
record the namespace exports, so you could add the export list to the
tclIndex file.  This way you would have all you needed from the
tclIndex file, and would not need to peek in the source files.  Brent
never finished this work off, however, and I also got too busy.  There
is not that much work needed to solve the problem correctly, but
unfortunately it is more than I have time for.

If you are still curious, you can ask Brent for some more details.  He 
even sent me some patches that almost work in an email about a year
ago.  He might be able to dig them up for you.

Jim

-- 02/02/2000 ericm