Tcl Source Code

View Ticket
Login
Bounty program for improvements to Tcl and certain Tcl packages.
Tcl 2019 Conference, Houston/TX, US, Nov 4-8
Send your abstracts to [email protected]
or submit via the online form by Sep 9.
Ticket UUID: 8b9854c3d8ba8d64a7d3a9b20dfec552c8ee5f54
Title: [info level 0] returns incompatible result in exported command since Tcl 8.6.9
Type: Bug Version: 8.6.9
Submitter: RP. Created on: 2019-01-15 12:16:37
Subsystem: 21. [namespace] Assigned To: dgp
Priority: 7 High Severity: Critical
Status: Closed Last Modified: 2019-06-22 11:01:06
Resolution: Fixed Closed By: pooryorick
    Closed on: 2019-06-22 11:01:06
Description:

Following code returns different results for Tcl 8.6.8 and 8.6.9:

namespace eval n1 {
   proc p1 {} {
      return [info level 0]
   }
   
   namespace export -clear *
   namespace ensemble create -subcommands {}
}

n1 p1
Returns for Tcl 8.6.8:
::n1::p1

Returns for Tcl 8.6.9:
p1

User Comments: pooryorick added on 2019-06-22 11:01:06:

Here is the example from the previous comment, but indented:

puts "Using Tcl [package present Tcl]"
namespace eval origin {
    namespace path ::tcl::mathop
    proc foo n {
        if {$n == 0} {return 1}
        return [* $n [namespace inscope [namespace current] [lreplace [info level 0] 1 1 [incr n -1]]]]
    }
    proc fact {n} {error BROKEN!}
    namespace export foo
}

namespace eval demo {
    namespace import ::origin::foo
    rename foo fact
	fact 3
}

Using Tcl 8.6.9

BROKEN!

Using Tcl 8.6.8

BROKEN!

Using Tcl 8.5.19

BROKEN!


pooryorick added on 2019-06-21 20:57:08:

So in the face of renamed imported commands, info level 0 as implemented before 8.6.9 is the only way to get at the name of renamed imported command. Below is an example where Tcl 8.6.8 and 8.5.19 are broken, as well as 8.6.9. Granted, uplevel would work in this example:

puts "Using Tcl [package present Tcl]" namespace eval origin { namespace path ::tcl::mathop proc foo n { if {$n == 0} {return 1} return [* $n [namespace inscope [namespace current] [lreplace [info level 0] 1 1 [incr n -1]]]] } proc fact {n} {error BROKEN!} namespace export foo }

namespace eval demo { namespace import ::origin::foo rename foo fact fact 3 }

Using Tcl 8.6.9

BROKEN!

Using Tcl 8.6.8

BROKEN!

Using Tcl 8.5.19

BROKEN!

I'll close this ticket now and look towards finding a better resolution in core-8-branch.


dgp added on 2019-06-21 18:54:04:
puts "Using Tcl [package present Tcl]"
namespace eval origin {
    namespace path ::tcl::mathop
    proc foo n {
        if {$n == 0} {return 1}
        return [* $n [namespace inscope [namespace current] [lreplace [info level 0] 1 1 [incr n -1]]]]
    }
    namespace export foo
}
namespace eval demo {
    namespace import ::origin::foo
    rename foo fact
    namespace export fact
    namespace ensemble create
}
namespace eval caller {
    proc fact {n} {error BROKEN!}
    puts [demo fact 3]
}


Using Tcl 8.5.19
6

Using Tcl 8.6.8
6

Using Tcl 8.6.9
invalid command name "fact"
    while executing
"fact 2"
    (in namespace inscope "::origin" script line 1)
    invoked from within
"namespace inscope [namespace current] [lreplace [info level 0] 1 1 [incr n -1]]"
    (procedure "fact" line 3)
    invoked from within
"demo fact 3"
    (in namespace eval "::caller" script line 3)
    invoked from within
"namespace eval caller {
    proc fact {n} {error BROKEN!}
    puts [demo fact 3]
}"
    (file "/home/dgp/recur.tcl" line 16)

Using Tcl 8.6.10
6

pooryorick added on 2019-06-21 17:28:00:

The recur.tcl example shows that in the case of command ensembles, uplevel isn't the right mechanism. Still, the current namespace of a command is always available, and can be used in every case to fully-qualify the name of the routine. Given that info level 0 has never before worked in every case to resolve a routine name, the newly-introduced behaviour should be considered a necessary bug fix, and a script that breaks can be replaced with namespace inscope [namespace current] $script.


dgp added on 2019-06-21 14:27:44:
$ cat recur.tcl
puts "Using Tcl [package present Tcl]"
namespace eval demo {
    namespace path ::tcl::mathop
    proc fact n {
        if {$n == 0} {return 1}
        return [* $n [uplevel 1 [lreplace [info level 0] 1 1 [incr n -1]]]]
    }
    namespace export fact
    namespace ensemble create
}
namespace eval caller {
    proc fact {n} {error BROKEN!}
    puts [demo fact 3]
}

$ ./tclsh ~/recur.tcl
Using Tcl 8.5.19
6

$ ./tclsh ~/recur.tcl
Using Tcl 8.6.8
6

$ ./tclsh ~/recur.tcl
Using Tcl 8.6.9
BROKEN!
    while executing
"error BROKEN!"
    (procedure "fact" line 1)
    invoked from within
"fact 2"
    ("uplevel" body line 1)
    invoked from within
"uplevel 1 [lreplace [info level 0] 1 1 [incr n -1]]"
    (procedure "fact" line 3)
    invoked from within
"demo fact 3"
    (in namespace eval "::caller" script line 3)
    invoked from within
"namespace eval caller {
    proc fact {n} {error BROKEN!}
    puts [demo fact 3]
}"
    (file "/home/dgp/recur.tcl" line 11)

$ ./tclsh ~/recur.tcl
Using Tcl 8.6.10
6

The pyk-core-8-6-branch goes back to being broken.

pooryorick added on 2019-06-21 10:24:59:

Even when not fully qualified, the first word of the command is sufficient to resolve the command from the namespace of the caller. A command wishing to recursively call itself can use [uplevel] to ensure proper resolution. Given this, what is the real need to store the qualified name of the command in <cod>framePtr->objv[0]?

Storing the fully qualified name of the command in framePtr->objv[0] does not scale well, as the memory footprint of each command becomes unacceptably high.

Currently, there's no example of functionality that can only be accessed by having [info level 0] converting the first word to a fully-qualified name.

I'm reopening this ticket to complete the discussion.


dgp added on 2019-06-17 18:02:59:
Fix committed for release in 8.6.10

dgp added on 2019-06-13 16:58:35:
Because the caller of an ensemble, the definition of the ensemble, and
the target command can all three have different namespaces, and because
the [info level 0] list may use only its first element alone to
identify the command (the arguments to the proc have to be
[lrange [info level 0] 1 end]) the only feasible solution
(maybe the only possible one?) is to fully qualify the command
name stored in framePtr->objv[0].

This adds yet another example to the list of ways Tcl deeply
insists that every command must have a fully qualified name. The
existing commands that lack one are bugs in need of fixing. TIP to come.

dgp added on 2019-06-13 16:22:25:
It is true that the [::watchdog::every] command in the earlier
comments said to come from Wub server is not as robust as it would
need to be to handle all the strange circumstances that Tcl might
throw at it.

(In fact not one of the [every] implementations on the Wiki is robust
enough at the moment.)

That said, at a minimum the body of a proc has to be assured that
when no renames or other changes that would require resolution epochs
to be incremented have taken place that

   uplevel 1 [info level 0]

will re-invoke the same procedure with the same arguments as got
the current body going. If we cannot rely on at least that much, it
becomes impossible to do things like code robust recursive procs.

The regression change to ensembles has broken that minimum need.

dgp added on 2019-03-08 19:53:42:
FWIW, it appears that the fully qualified command returned
until recently was in place from the very arrival of ensembles
in TIP 112.

https://core.tcl.tk/tips/doc/trunk/tip/112.md

https://core.tcl.tk/tcl/tktview/786509

dgp added on 2019-03-08 18:28:27:
Returning to this, it seems to me that neither the old nor new
behavior is the most desirable. I'd be happier if the demo script
in the ticket returned "n1 p1", making namespace ensembles agree
with how the oo machinery deals with this:

% oo::class create Demo {
method dingo {} {return [info level 0]}
}
::Demo
% Demo create d
::d
% d dingo
d dingo

pooryorick added on 2019-02-08 19:43:35:

These examples are good illustrations of code that isn't robust and should be fixed. A case like proc demo, where there is no namespace ensemble, is likely to fail often depending on context. In the case of watchdog::every if someone decided to namespace import ::watchdog::every, things would break (in any version of Tcl). Let's fix that in wub.


RP. added on 2019-02-08 13:02:53:

For example sample from Wub server (which was in fact how I found out about this bug (or feature)):

namespace eval ::watchdog {
   variable timeout 60000

   proc every {interval script} {
      after $interval [info level 0]
      uplevel #0 $script
   }

   # some more definitions here

   namespace export -clear *
   namespace ensemble create -subcommands {}

   ::watchdog every $timeout {::watchdog reaper}
}
Fails with error:
invalid command name "every"
    while executing
"every 60000 {::watchdog reaper}"
    ("after" script)


dgp added on 2019-02-08 11:43:22:
I haven't gone examining my own code, or examples in books and tutorials, etc. but it seems to me I'm likely to find code like:

proc demo {} {
   # Change something ...
   # and re-run myself
   uplevel 1 [info level 0]
}

and given this ticket I have less confidence about that being reliable.

I know we have [tailcall] now, but that's not a good reason to break code that was written before [tailcall] arrived.

dgp added on 2019-02-08 11:37:52:
Can you please offer an example of code that has been broken by the change? Not a minimal demo like in the original ticket (which is good), but something functional you were doing that you cannot do anymore?

Thanks!

RP. added on 2019-02-08 07:46:19:

Maybe it is not a bug, but it worked that way so far and all codes that depends on it will stop working after Tcl upgrade. Maybe info level 0 is more consistent now, but at cost of incompatibility with all previous versions of Tcl.


pooryorick added on 2019-02-07 15:55:16:

info level 0 simply returns all the words in the command associated with a level. It was not intended as a mechanism for resolving the command to a fully-qualified name, and it generally does not:

namespace eval n1 {} {
	proc p args {
		puts [info level 0]
	}

	namespace export *
	namespace ensemble create
	p n1
}

namespace eval n2 {} {
	namespace path ::n1
	p n2
}

The fact that the namespace ensemble command resolution routines ever rewrote the command name was incidental, and not intended to be part of the public interface. In order to fix [16fe1b5807] the ensemble subcommand lookup routines no longer make this artifact visible to info level. This is not a bug. Furthermore, info level 0 is now more consistent. It no longer rewrites the command provided by the caller.


pooryorick added on 2019-02-07 14:59:08:

See [16fe1b5807] for a description of the changes that lead to this.


dgp added on 2019-02-07 14:39:58:
....which was reportedly an effort to fix the bug reported in ticket

https://core.tcl-lang.org/tcl/info/16fe1b5807

dgp added on 2019-02-07 14:38:38:
The history manipulation in fossil is still mysterious, but the
originating checking seems to be

https://core.tcl-lang.org/tcl/info/b433ae397cddec65

dgp added on 2019-02-04 18:38:34:
The earliest checkin I see suffering from this changed behavior is

https://core.tcl-lang.org/tcl/info/e5003b19a31cf961

The parent checkin of that checking appears to not be present
in the core.tcl-lang.org/tcl repository.

I do have a copy in my clone, and I'm not enough of a fossil wizard to understand the unusual things that have been done to it.

pooryorick appears to be the operator at work on all of it. I think we have to hear from him.

sebres added on 2019-01-15 12:58:29:

It looks like rewrite ensemble (something like TclInitRewriteEnsemble or round about) does something wrong or ensembleRewrite info doesn't affect level info for some reasons anymore.

Strange is, if I extend the test (namespace) with:

proc p2 {} {error test}
...
the error-info still contains correct (rewritten) line "n1 p2":
...
"error test"
    (procedure "p2" line 1)
    invoked from within
"n1 p2"