Tcl Source Code

View Ticket
Login
2023-03-24
21:39 Closed ticket [ea69b0258a]: Crash when using a channel transformation on TCP client socket plus 4 other changes artifact: 2e85ab4d60 user: pooryorick
2023-03-22
17:24
Merge trunk a76dee9eb4: Further fix for issue [ea69b0258a9833cb], crash when using a channel transf... check-in: ca28ab2f5d user: pooryorick tags: unchained
17:15
Merge trunk 62058cafe0: Fix for issue [ea69b0258a9833cb], crash when using a channel transformation ... check-in: 13eca467c0 user: pooryorick tags: unchained
2023-03-15
11:48 Ticket [ea69b0258a] Crash when using a channel transformation on TCP client socket status still Pending with 3 other changes artifact: e4e8d32166 user: pooryorick
11:43 Ticket [ea69b0258a]: 3 changes artifact: 5e9fa93a3d user: pooryorick
11:37
Further fix for issue [ea69b0258a9833cb], crash when using a channel transformation on TCP client so... check-in: a76dee9eb4 user: pooryorick tags: trunk, main
09:25
Further fix for issue [ea69b0258a9833cb], crash when using a channel transformation on TCP client so... check-in: 294e0130dc user: pooryorick tags: core-8-branch
08:42
Further fix for issue [ea69b0258a9833cb], crash when using a channel transformation on TCP client so... Closed-Leaf check-in: 26a20919fb user: pooryorick tags: core-8-branch-bug-ea69b0258a9833cb6
2023-03-14
20:30
Further fix for issue [ea69b0258a9833cb], crash when using a channel transformation on TCP client so... check-in: a719a1392c user: pooryorick tags: core-8-6-branch
20:27
Further fix for issue [ea69b0258a9833cb], crash when using a channel transformation on TCP client so... Closed-Leaf check-in: 8aee50448c user: pooryorick tags: bug-ea69b0258a9833cb6
2023-03-13
21:28 Ticket [ea69b0258a] Crash when using a channel transformation on TCP client socket status still Pending with 3 other changes artifact: b8ce31539e user: pooryorick
21:09 Ticket [ea69b0258a]: 3 changes artifact: 7e9ffdac51 user: chw
21:02 Pending ticket [ea69b0258a]. artifact: 851b534171 user: pooryorick
20:57
Fix for issue [ea69b0258a9833cb], crash when using a channel transformation on TCP client socket. check-in: 62058cafe0 user: pooryorick tags: trunk, main
19:19
Fix for issue [ea69b0258a9833cb], crash when using a channel transformation on TCP client socket. check-in: 6d017aacac user: pooryorick tags: core-8-branch
19:07
Fix for issue [ea69b0258a9833cb], crash when using a channel transformation on TCP client socket. check-in: a8b8ecbc03 user: pooryorick tags: core-8-6-branch
16:55 Ticket [ea69b0258a] Crash when using a channel transformation on TCP client socket status still Open with 3 other changes artifact: 629cb07cfe user: chw
16:09 Ticket [ea69b0258a]: 3 changes artifact: 54043e6e59 user: apnadkarni
13:36
Fix for issue [ea69b0258a9833cb], crash when using a channel transformation on TCP client socket. check-in: ac8ea99a1a user: pooryorick tags: core-8-branch-bug-ea69b0258a9833cb6
12:45 Ticket [ea69b0258a] Crash when using a channel transformation on TCP client socket status still Open with 4 other changes artifact: 80a3895bd9 user: oehhar
12:22
Fix for issue [ea69b0258a9833cb], crash when using a channel transformation on TCP client socket. check-in: 052f54ddfb user: pooryorick tags: bug-ea69b0258a9833cb6
2023-03-07
21:35 Ticket [ea69b0258a] Crash when using a channel transformation on TCP client socket status still Open with 3 other changes artifact: 006f692866 user: pooryorick
15:23 Ticket [ea69b0258a]: 3 changes artifact: 308c74d199 user: apnadkarni
15:00 Ticket [ea69b0258a]: 3 changes artifact: ea95732851 user: apnadkarni
13:10 Ticket [ea69b0258a]: 4 changes artifact: 87df5baed2 user: oehhar
12:48 Ticket [ea69b0258a]: 3 changes artifact: 95aadbf939 user: petrokaz
12:42 Ticket [ea69b0258a]: 4 changes artifact: c7d5b89f63 user: oehhar
10:32 Ticket [ea69b0258a]: 3 changes artifact: c8920c9d34 user: petrokaz
10:31 Add attachment crash3.tcl to ticket [ea69b0258a] artifact: d3b6cc9933 user: petrokaz
06:53 Ticket [ea69b0258a] Crash when using a channel transformation on TCP client socket status still Open with 3 other changes artifact: 1377f7d150 user: apnadkarni
2023-01-12
16:33 Ticket [ea69b0258a]: 5 changes artifact: af54bc6f6e user: petrokaz
15:04 New ticket [ea69b0258a]. artifact: 5b629d1669 user: petrokaz

Ticket UUID: ea69b0258a9833cb61ada42d1fc742d90aec04d0
Title: Crash when using a channel transformation on TCP client socket
Type: Bug Version: 8.6.13
Submitter: petrokaz Created on: 2023-01-12 15:04:29
Subsystem: 26. Channel Transforms Assigned To: pooryorick
Priority: 5 Medium Severity: Critical
Status: Closed Last Modified: 2023-03-24 21:39:56
Resolution: Fixed Closed By: pooryorick
    Closed on: 2023-03-24 21:39:56
Description:

Tcl crashes when I setup a channel transformation on a non-blocking TCP client socket, and then remove it.

Here is the minimal reproducible example. A dummy echo server:

ncat -l -k -e /bin/cat --crlf 0.0.0.0 4400

And my Tcl script:

package require tcl::transform::observe

proc writable {sock} {
    chan event $sock writable {}
    chan configure $sock -translation {crlf binary} -blocking 0 -buffering full -encoding binary
    chan event $sock readable [list readable $sock]
    set ::status "connected"
}

proc readable {sock} {
    set readCount [chan gets $sock line]
    if {$readCount <= 0} {
        if {[eof $sock]} { 
            puts "Connection aborted"
            close $sock
            set :status "closed"
            break
        }
        # we don't have a full line yet - wait for next chan event
        return
    }
    set ::msg $line
}

set sock [socket -async localhost 4400]
chan event $sock writable [list writable $sock]
vwait ::status
puts "Status $status"
tcl::transform::observe $sock stdout ""

puts $sock "Hello\r\n"
flush $sock

vwait ::msg
puts "Got $msg"
# update ;#- avoids crash
chan pop $sock

after 100 {set ::forever 1}
vwait forever ;# crashes here

close $sock
puts DONE

This produces the following output:

Status connected
Hello

Got Hello
Segmentation fault (core dumped)

The lines "hello\r\n" are printed by the channel observer. Then Tcl crashes after I remove the channel transformation and enter the event loop.

The crash does NOT occur if I uncomment the line update ;#- avoids crash

The crash does NOT occur if I don't use the channel transformation.

I can reproduce the crash with 100% chance on OpenSUSE 15.4/Tcl 8.6.12 (from repos) and Windows/Tcl 8.6.13 (from Magicsplat).

Thanks in advance.

User Comments: pooryorick added on 2023-03-15 11:48:22:

The fix includes a test that, prior to the fix, reproduced the problem on a Linux system. I would expect the following script to reproduce the issue on Windows as well. Could someone try it out and report the results?

#! /usr/bin/env tclsh

namespace eval rchan {

	proc initialize {chan mode} {
		namespace eval $chan {
			variable source "line one\nline two"
		}
		after 0 [list ::chan postevent $chan read]
		return {initialize finalize watch read}
	}


	proc finalize chan {
		namespace delete $chan
	}


	proc read {chan count} {
		namespace upvar $chan counter counter source source
		set res [string range $source 0 $count-1]
		set source [string range $source $count end]
		return $res
	}


	proc watch {chan events} {
		after 0 [list chan postevent $chan read]
		return read
	}


	namespace ensemble create
	namespace export *
}


namespace eval filter {
	proc initialize {chan mode} {
		return {initialize finalize read}
	}
	
	proc read {chan buffer} {
		return $buffer
	}

	namespace ensemble create
	namespace export *
}


proc read1 {chan args} {
	variable read
	catch {
		set res [gets $chan]
	}
	incr  read
}

set chan [chan create read rchan]
chan configure $chan -blocking 0
chan push $chan filter
chan event $chan read [list read1 $chan]
vwait [namespace current]::read
chan pop $chan
vwait [namespace current]::read
return


pooryorick added on 2023-03-15 11:43:05:

Proper cleanup added for 8.6 in [a719a1392ca50359], for 8.7 in [294e0130dcff79a3], and for 9.0 in [a76dee9eb447f3f9].


pooryorick added on 2023-03-13 21:28:23:

chw, I missed your previous remark today. I see what you mean. Looking into it now.


chw added on 2023-03-13 21:09:16:
Nathan, at least in core-8-6-branch there are two locations
where Tcl_DeleteTimerHandler() is called for the timer.
Which has incremented the refCount when created. Which does
not get decremented when the timer is deleted. Which might
result in the refCount never reaching zero. Which might be
kind of a leak. Do you agree?

pooryorick added on 2023-03-13 21:02:19:

Fixed for 8.6 in [a8b8ecbc039fb4e0], for 8.7 in [6d017aacac9dd19d], and for 9.0 in [62058cafe065cb35].


chw added on 2023-03-13 16:55:52:
Nathan's fix in [052f54ddfb] looks plausible but IMO isn't complete yet.
When the creation of the ChannelTimerProc timer increments the refCount
per TclChannelPreserve() and it's expiry does the opposite with
TclChannelRelease(), all locations where the (unexpired) timer is deleted
must perform the TclChannelRelease(), too.

apnadkarni added on 2023-03-13 16:09:05:
Harald,

Petro could reproduce on Windows as well though I could not despite many different attempts with configurations.

Given it looks like an early release of memory, it could all be timing dependent for the bug to manifest itself depending on when the memory gets reused for example.

Just guessing...

/Ashok

oehhar added on 2023-03-13 12:45:17:

Pooryorik, thanks for the fix in [052f54ddfb]. Can you imagine, why this only fails on Linux ? It is in the gereric code...

Pietro, can you check, if the fix is ok for you ?

Take care, Harald


apnadkarni added on 2023-03-07 15:23:37:
Well, the good news is that I tried it on my Ubuntu WSL and it crashed right away. The bad news is I have no clue about Linux crash dumps or debuggers. So this will take some time. But at least it is reproducible on Linux with 8.6.13.

/Ashok

apnadkarni added on 2023-03-07 15:00:43:

I've tried it on Windows using magicsplat, as well as the latest repository under debug and release modes, in a batch loop. Have not been able to get it to crash. (Using the Windows ncat from the nmap site).

Output is always

Status=connected
Hello1
Hello2
Connection aborted
Got closed
pending input before pop: 0
pending input after pop: 0
DONE

Note pending input is always 0. I just cannot get it to be non-0 which may be why my test does not crash. When I get time, I'll experiment different configs to try and get non-0 input.

BTW what version of tcllib are you using? I presume same as what shipped with the 8.6.13 magicsplat distribution?


oehhar added on 2023-03-07 13:10:45:

Yes, I fear the reproduction recipe is "Linux only". stdout is used, what is not available with Windows wish.

Maybe, some Linux foks will jump in...

Take care, Harald


petrokaz added on 2023-03-07 12:48:35:

Hello Harald, you need some 'echo' TCP server running on the port 4400. On Linux the easiest way is to run netcat as I show in the bug report. I'm not sure if netcat with the same features is available on Windows. You could write it in Tcl as well.


oehhar added on 2023-03-07 12:42:53:

Thanks ! What I have done:

  • start wish 8.6.13 32 bit on windows
  • take tcllib 1.21
lappend auto_path {C:\test\tcllib1.21}
  • Paste your script
  • it waits some time in the line
vwait ::msg
  • It shows the background error
error reading "sock03F5A828": socket is not connected
    while executing
"chan gets $sock line"
    (procedure "readable" line 2)
    invoked from within
"readable sock03F5A828"

Would you have additional instructions to reach the line where it should crash ?

Thank you and take care, Harald


petrokaz added on 2023-03-07 10:32:52:
Hello Ashok,
sorry for the poor bug report... I made some mistakes while trying to extract a minimal reproducible example from my original code that is much bigger.

I've taken your script and changed it so that it crashes again - attached crash3.tcl

So, AFAICS [chan pop] crashes if there is pending input.

apnadkarni added on 2023-03-07 06:53:28:

Although Tcl should never crash from anything you do at the scripting level, there are several problems with your code which I presume you have fixed in your script but updated the ticket.

In the readcount proc,

  • there is a break outside of a loop which causes the event handler to abort. I presume that should be a return.
  • the $readcount <= 0 check should be $readcount < 0 since getting back an empty line is perfectly kosher. In particular note your puts Hello\r\n is sending two newlines, one explicit and one because of the \r\n. So you will get back an empty line from the server.
  • The set :status "closed" line is missing a : - should be ::status
  • Moreover, strictly speaking you should also set ::msg in there in case the server closes without sending anything back (the vwait is waiting on ::msg)
  • In case of EOF without receiving any lines, the handler closes the socket and the chan pop is attempted on a closed socket. Again this is only in the case the server closes without sending anything.

In any case, I could not reproduce the crash. With your script, I simply get Tcl exceptions, as expected. With my changes (see script below), I get a print out

Status connected
Hello
Connection aborted
Got closed
DONE

This is irrespective of whether the update is commented or not.

My script is below, please try and see if the crash is reproduced on your system with my script:

lappend auto_path d:/tcl/lib
package require tcl::transform::observe

proc writable {sock} {
    chan event $sock writable {}
    chan configure $sock -translation {crlf binary} -blocking 0 -buffering full -encoding binary
    chan event $sock readable [list readable $sock]
    set ::status "connected"
}

proc readable {sock} {
    set readCount [chan gets $sock line]
    if {$readCount < 0} {
        if {[eof $sock]} { 
            puts "Connection aborted"
            # close $sock
            chan event $sock readable {}
            set ::msg "closed"
            return
        }
        # we don't have a full line yet - wait for next chan event
        return
    }
    set ::msg $line
}

set sock [socket -async localhost 4400]
chan event $sock writable [list writable $sock]
vwait ::status
puts "Status $status"
tcl::transform::observe $sock stdout ""

puts $sock "Hello"
flush $sock

vwait ::msg
puts "Got $msg"
# update ;#- avoids crash
chan pop $sock

after 100 {set ::forever 1}
vwait forever ;# crashes here

close $sock
puts DONE


petrokaz (claiming to be Petro Kazmirchuk) added on 2023-01-12 16:33:47:
replace "sleep" with a plain "update" for even smaller example

Attachments: