Tcl Source Code

View Ticket
Login
Ticket UUID: e76f0a6ca52a2ff3c2b1e9cb2585f3de0c44ab5d
Title: [namespace delete] unbearably slow
Type: Bug Created on: 2015-07-20 09:59:01
Submitter: anonymous Assigned to: dkf
Subsystem: 21. [namespace] Severity: Minor
Priority: 5 Medium Last modified: 2016-06-07 07:24:41
Status: Closed Closed by: adrianmedranocalvo
Resolution: Duplicate Closed on: 2016-06-07 07:24:41
Version: 8.6.3
Description:
Invoking [namespace delete] in a namespace with a huge number of procs/commands takes an excessively long time.  That is not the case when removing the procedures individually.  This causes issues to packages that create great many commands in a namespace, such as TclXML (one command per used node).

The following snippet exhibits the issue:

~~~
proc timed {script} {
    puts [::uplevel 1 [list time $script 1]];
}
foreach N {1000 10000 50000 100000 200000} {
    puts "== $N ==";

    # [rename] before [namespace delete]
    timed {
	namespace eval test {};
	for {set i 0} {$i < $N} {incr i} {
	    ::proc ::test::abc$i {args} { puts $args; };
	}
	foreach cmd [info proc test::*] {
	    ::rename $cmd {}
	}
	namespace delete ::test;
    };

    # plain [namespace delete];
    timed {
	namespace eval test {};
	for {set i 0} {$i < $N} {incr i} {
	    proc ::test::abc$i {args} { puts $args; };
	}
	namespace delete ::test;
    };
}
~~~

This is the output:

~~~
== 1000 ==
5266 microseconds per iteration
6140 microseconds per iteration
== 10000 ==
44048 microseconds per iteration
32618 microseconds per iteration
== 50000 ==
2153807 microseconds per iteration
167686 microseconds per iteration
== 100000 ==
3373670 microseconds per iteration
338030 microseconds per iteration
== 200000 ==
25682504 microseconds per iteration
691066 microseconds per iteration
~~~

The elapsed time is greater than linear on the number of commands for [namespace delete].
User Comments:
dgp added on 2015-07-20 11:58:17:

I believe this is a duplicate of

http://core.tcl.tk/tcl/info/f97d4ee020

dgp added on 2015-07-20 12:10:33:
See branch bug-f97d4ee020 for a patch you can
test to confirm this is the same issue.

anonymous (claiming to be Adrián Medraño Calvo <[email protected]>) added on 2015-07-23 09:45:58:
Sadly there seems to be no change with the patch.

The test script is remarkably similar to that of 4ee020.  I have run the latter with the patched tcl and am able to see the improvement.  Sadly there is no improvement for this case.

Please, note that we are comparing the following two forms of deleting a namespace:

~~~
namespace delete ::test;
~~~

and:

~~~
foreach cmd [info proc test::*] {
    ::rename $cmd {}
}
namespace delete ::test;
~~~

The difference in the time it takes is so large, that one wonders whether one of them (TclTeardowNamespace) or ::rename are doing something wrong.  From my understanding, the behaviour is no different.

------------------------

Please, note also that I have modified the test script to make it a little more clear.  The results with patched Tcl are below:

~~~
== 10 ==
150 microseconds per iteration
87 microseconds per iteration
== 100 ==
599 microseconds per iteration
370 microseconds per iteration
== 1000 ==
6458 microseconds per iteration
4784 microseconds per iteration
== 10000 ==
37656 microseconds per iteration
41629 microseconds per iteration
== 100000 ==
351145 microseconds per iteration
3421361 microseconds per iteration
== 200000 ==
747001 microseconds per iteration
26004022 microseconds per iteration
~~~

dgp added on 2015-07-23 14:45:07:
I checked, and the patch to namespace command table teardown
does indeed correct this demonstration script:

== 1000 ==
3777 microseconds per iteration
2077 microseconds per iteration
== 10000 ==
35204 microseconds per iteration
17298 microseconds per iteration
== 50000 ==
130501 microseconds per iteration
72423 microseconds per iteration
== 100000 ==
250332 microseconds per iteration
145699 microseconds per iteration
== 200000 ==
520258 microseconds per iteration
301174 microseconds per iteration


All I can guess is that you were not testing what you 
believed you were testing.

anonymous (claiming to be Adrián Medraño Calvo <[email protected]>) added on 2015-07-24 07:43:27:
Upon further inspection, I attest that the code in branch bug-f97d4ee020 cannot possibly fix the issue in this ticket.  The underlying cause is analogous: deleting a hash table requires more work if the table is subject to change during the iteration.

The difference between this issue and that of http://core.tcl.tk/tcl/info/f97d4ee020 is that the latter fills the namespace with sub-namespaces, where this one fills the namespace with procedures.

The changes in branch bug-f97d4ee020 affect the iteration through the nsPtr->childTable.  Changing the iteration through the nsPtr->cmdTable in a similar way:

~~~
--- generic/tclNamesp.c 2015-07-24 09:26:47.621134045 +0200
+++ generic/tclNamesp.c.orig    2015-07-24 09:27:21.598132784 +0200
@@ -1128,11 +1128,7 @@
 
     for (entryPtr = Tcl_FirstHashEntry(&nsPtr->cmdTable, &search);
            entryPtr != NULL;
-#if 0
            entryPtr = Tcl_FirstHashEntry(&nsPtr->cmdTable, &search)
-#else
-           entryPtr = Tcl_NextHashEntry(&search)
-#endif
        ) {
        cmd = Tcl_GetHashValue(entryPtr);
        Tcl_DeleteCommandFromToken((Tcl_Interp *) iPtr, cmd);
~~~

does indeed "fix" the timings.

It could be said that this issue duplicates http://core.tcl.tk/tcl/info/f97d4ee020 in that a fix for the latter would probably apply as well for this one.  Nevertheless, the explanation as to why renaming the commands then deleting the namespace is faster than simply deleting the namespace would remain (for me) mysteriously unknown.  That is, why can the former perform linearly and the latter not?

dgp added on 2016-06-01 19:17:22:
Please confirm the Dup of this was fixed
and close.  Thanks!

adrianmedranocalvo added on 2016-06-07 07:24:41:
Indeed, it is fixed by bug-f97d4ee020 branch.

Thank you very much!