Tcl Source Code

View Ticket
Login
Ticket UUID: 3017879
Title: proc emptyness is cached too aggressively
Type: Bug Created on: 2010-06-18 04:24:07
Submitter: a_kovalenko Assigned to: msofer
Subsystem: 47. Bytecode Compiler Severity:
Priority: 5 Medium Last modified: 2010-06-18 16:49:42
Status: Open Closed by:
Resolution: None Closed on:
Version: obsolete: 8.6b1.1
Description:
There is one known, long-standing feature of the TCL bytecode
compiler: if a procedure is defined with empty body and all-matching
parameter list {args}, TCL doesn't compile any real procedure call for
its callers. In the output of tcl::unsupported::disassemble, it's visible as an absence of invokeStk1 instruction where an empty proc is called.

This optimization may be regarded as a special case of inlining, like
inlining an empty proc body and an all-accepting argument
check. Inlining doesn't play well with redefinitions and traces, but
for all years I look at that, TCL developers counted only
_transparent_ optimizations as acceptable. Various `bytecode epoch'
counters triggering bytecode recompilation create an illusion that
there is no byte-compiler at all, it's just a code runs faster.

# ==== Example 1, all ok ======
proc do-nothing {args} {}
proc talk {} {
  puts "Going to do nothing.."
  do-nothing 1 2 3 4
  puts "Done nothing successfuly."
}
# Here we may take a look at 
  puts [tcl::unsupported::disassemble proc talk]
# no invokeStk1, just a {push1 ""; pop} pair.

#.......later......
proc do-nothing {args} {
  puts "You have to do _something_.."
}

puts [tcl::unsupported::disassemble proc talk]
# And now we see invokeStk1, do-nothing is really called 
talk
# and does something.

Unfortunately, redefining an _imported_ target procedure 
is not noticed in importing namespaces:

namespace eval foo {namespace export *}
proc foo::do-nothing {args} {}
proc talk ....................
proc foo::do-nothing {args} {puts "doing something.."}
talk ;# says nothing. 
# Imported command retained the information somewhere,
# that its target is an empty proc.
namespace import -force foo::*
# Reimporting with -force actually destroys target command
# before reimport, and triggers byte-recompilation of `talk'
# Reimporting without -force is regarded as a no-op,
# as long as target is the same (and as an error otherwise).

I hope for a solution propagating command redefinition down to each
importing namespace, triggering bytecode recompilation there (it's
currently possible on the script level, traversing all namespaces and
their imports and reimporting a redefined command with -force).
Or, if it turns out to be too complicated, I'd prefer to see this kind
of bytecode optimization abandoned.

Redefinition of an imported target is a convenient way to provide
e.g. debugging facilities (zero-cost while disabled, triggering
recompilation when turned on, low-cost afterwards). It turned out to
be a fine technique to manage built-in command interception (consider
an imported ::puts that points either to renamed original `puts' or to
a redefined, redirecting version). But for the latter, bytecode not
invalidated when appropriate may cause more severe semantical problems
than simply not running a procedure that used to be empty.

Several things on the same subject:

1. TCL 8.5.8 release and CVS head expose similar behavior with
   imported proc redefinitons; the problem affects both.

2. If a command in a foreign namespace is resolved via 
   namespace path and _not_ imported, the problem vanishes:
   all bytecode calling it is invalidated as it's redefined
   from "proc name args {}" to something other, and back.

3. Execution traces (enter, leave) on an empty proc aren't invoked
   (even if no namespace import is involved, i.e. in the same (root) 
   namespace). However, I'm not sure that recompiling callers
   when a trace is added is a good thing.

4. If we forget about redefining renamed compilable built-ins, the
   only transition that is semantically problematic is the one from an
   empty proc to a non-empty one. A code generated for non-empty proc
   (pushes + invokeStk1) remains valid for any other proc or command
   definition, including the empty one (if it's ever redefined back).

5. There is a missing feature in namespace imports introspection:
   [namespace origin] spec creates an illusion that all imports of a
   command are like independent links to the same target, and only the
   target is `special' among them. In fact, imports of the same command
   may form a complicated dependency graph, and for now its structure 
   can't be introspected on script level in any non-intrusive way.
   (of course it's a reason for a separate report, maybe a TIP).
User Comments:
msofer added on 2010-06-18 12:45:06:
Not yet studied the bug description in full. Just to note that tcllib's control::assert may be doing things the way you need? http://tcllib.sourceforge.net/doc/control.html

As for "a solution propagating command redefinition down to each importing namespace", this is not a feature of imported commands. The import refers to the original command at the time it is imported. If you want to resolve the command by name at the time of command invocation, either [namespace path] or [interp alias] are the way to go.

  % namespace eval foo {namespace export *}
  % proc foo::a {} {puts a0}
  % namespace import foo::a
  % a
  a0
  % rename foo::a foo::b
  % a
  a0
  % namespace origin a
  ::foo::b
  % rename a x
  % x
  a0
  % namespace origin x
  ::foo::b
  % proc foo::a {} {puts a1}
  % namespace import foo::a
  % a
  a1
  % rename a y
  % y
  a1
  % x
  a0

a_kovalenko added on 2010-06-18 16:49:42:
Miguel,

You are right: when we speak of renames, e.g. of renaming to {} and
putting some other command instead of an original one, imported
"command pointers" behave like we expect: they follow original commands
and vanish when originals are deleted. 

But the problem I described is linked to a fact that proc redefinition
really ISN'T interpreted as command deletion + new command creation.
It's something like a "half-deletion": command delete traces are called,
but existing imports are retargeted to new proc instead of vanishing. 

More: overriding a non-proc command with a proc having the same name
(old command may be e.g. a renamed built-in, an extension-provided
command, an alias, an ensemble, an "import link" to another
command, etc..)  _ALSO_ retargets imports to a proc. 

So overriding something with [proc] is a special case, a kind of `two
renames done in a transaction while blocking importers from noticing
it', and I fully understand why it should be allowed: a lot of
programmers want to modify behavior of existing commands without
breaking past imports. For all such redefinitions, _except_ overriding
an empty proc (proc name args {}), old import entries _are_ relinked
to a new proc. Failing to imitate the same behavior with empty procs
is therefore a bug.