Tcl Source Code

Could this just be history building up, in an interactive shell ?
If not, try increasing the [time] itercount and checking the effect on the slope.

To remove history problems do one of:
  - run the thing in an infinite loop (with an [after 10] in there), watch what top has to say about mem
  - do [proc history args {}] before the test

Ran a new test, non-interactive, wrapped in a proc, 100000 iterations. The results are:

            VIRT    RES  SHR
START  32476 4192 1960
END      36284 7088 1960

OK thanks. Now what about 1*100000 vs. 10*10000 ? This to decide whether it leaks rather "per [times]" or "per [thread::*]" invocation.

Here are the 10*10000 results:

32872 4652 1960
34120 4964 1960
34384 5244 1960
34664 5504 1960
34888 5732 1960
35116 5956 1960
35380 6204 1960
35584 6428 1960
35868 6672 1960
36092 6916 1960

Try this:

time {set ::t [thread::create -joinable]; thread::release $t; thread::join $t} 10000

Results for:
 time {set ::t [thread::create -joinable]; thread::release $t; thread::join $t} 10000

ran ten times --- 

24392 4284 1968
24616 4528 1968
24844 4752 1968
25068 4996 1968
25292 5220 1968
25516 5464 1968
25744 5692 1968
25968 5936 1968
26192 6160 1968
26416 6404 1968

any progress?

Not yet. I do not think we can do anything here fast, as most probably
this is some deep finalization issue in Tcl library. The extension itself
does not leak at this point but the core could. I do not have resources at
the moment for deeper analysis but will throw a valgrind againt it just to
be 100% sure. If you aim at speedy releasing 2.6.7 I think you should not
wait.

This is not the issue of the threading extension. I have tried the Tcl 8.4 core
with 2.6.6 thr-extension on Linux and Mac and double-checked with valgrind.
There are no leaks there at all. The Linux top shows pretty much same virtual
process size no matter now often I repeat the following command;

time {set ::t [thread::create -joinable]; thread::release $t; thread::join $t} 

I run if 1000000 times and then check vith valgrind and top and there is no
leak. I guess later (8.5/8.6) cores might not be leak-free; i haven't tested those releases as we are still using 8.4 core.

Unfortunately I missed the new move-fro-sf-cvs so I do not yet) khow where
are the new sources and I need to catch-up with that.

Using 8.6b2rc1 I can reproduce a leak on WIN32 using the time{} loop recommended by Zoran.  Only the Windows build leaks about 135,000 bytes per iteration.  Task Manager's thread count confirms that all additional threads have stopped.

For reference, a simple interp create/delete loop doesn't seem to leak memory, implying that this isn't an interp leak.

I think the problem is in the Tcl finalization area. 
The interp itself is clean. The finalization isn't :-( 
Some years ago I contemplated to rewrite the 
finalization code using refcounts but this was a 
major undertaking and I never found time for it. 
The 8.4 seems to be more stable in this respect
which is why we still stick to it.  

In the meantime, creating a set of worker threads
upfront (or using tpool with fixed number of workers)
should keep the memory in the "reasonable" area.
Also, turning off the greedy threading allocator that
never releases memory to the system and using the
regular system malloc instead could also be a good 
move. I recall we turned that off some years ago and
I cannot recall our app becomming slower (yet it 
became a fifth of its original memory footprint!).

I think this could be a hard nut to crack. In some days
I will get more spare time to checkout the latest head
and see what all happened there since the last time 
I looked at it.

Using the usual tcltest harness for memory leak testing,
I tried this:

testConstraint memory [llength [info commands memory]]
test thread-8.1 {Bug 3381371} -constraints memory -setup {
    proc getbytes {} {
        set lines [split [memory info] "\n"]
        lindex $lines 3 3
    }
    package require Thread
} -body {
    set end [getbytes]
    for {set i 0} {$i < 5} {incr i} {
        set t [thread::create -joinable]
        thread::release $t
        thread::join $t
        set start $end
        set end [getbytes]
    }
    expr {$end - $start}
} -cleanup {
    rename getbytes {}
    unset -nocomplain i start end t
} -result 0

It shows no leak, using the trunk of Tcl & Thread.

What is it that's claimed to be leaking?  Apparently
it's not Tcl_Alloc'ed  memory?  Or is this a leak I 
should not be able to see testing on Linux?

On Linux I cannot reproduce this with 8.6.17, 9.0.3 and 9.1 and Thread 3.0.4: the memory stays flat over 100000 threads. On Windows each joinable thread which exits before it is joined leaks about 9 KB: TclSignalExitThread() creates the mutex of the join record in the exiting thread, after its allocator cache has been finalized, so a new cache is created and never freed.

Proposed fix in branch win-join-mutex-leak: create the mutex in TclRememberJoinableThread().

Fixed in [b2f3ca31ad|trunk] and core-9-0-branch