Tcl Library Source Code

Check-in [8de453c348]
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:The issue with the skip code was not the limit to 64K, but how the code (a) assumed the wrong type for the result of 'read' (number of bytes, but is the read data), and (b) mis-handled short reads. Using 'string length' to convert the result type into what was needed, and using the proper condition was what fixed the issue, and not getting rid of the limit. I have now re-instated the limit, with in-code explanation of why it makes sense, and verified that this still passes the testsuite. I further changed the variable names to be much more self-documenting as well.
Downloads: Tarball | ZIP archive
Timelines: family | ancestors | descendants | both | pyk-tar
Files: files | file ages | folders
SHA1: 8de453c348b85718a0884e8cdd9b78daa6f218f8
User & Date: andreask 2013-11-23 00:00:39.292
Context
2013-11-23
00:14
Bumped package version, updated logs, ready to merge. Fixed ticket [6b7aa0aecc]. check-in: 4a2f75b5da user: andreask tags: pyk-tar
00:00
The issue with the skip code was not the limit to 64K, but how the code (a) assumed the wrong type for the result of 'read' (number of bytes, but is the read data), and (b) mis-handled short reads. Using 'string length' to convert the result type into what was needed, and using the proper condition was what fixed the issue, and not getting rid of the limit. I have now re-instated the limit, with in-code explanation of why it makes sense, and verified that this still passes the testsuite. I further changed the variable names to be much more self-documenting as well. check-in: 8de453c348 user: andreask tags: pyk-tar
2013-11-22
23:39
Fixed the testsuite up somewhat. (1) Moved helper code into separate file. (2) Replaced helper code blocks in variables with proper procedures. (3) Simplified setup1, cleanup1 No need to search for and create lots of temp dirs in setup, just remove the directory in cleanup. Now use single temp dir, fixed name. (4) Fixed setup of tests in general (4a) Added emacs mode marker to get proper Tcl mode. (4b) Added missing Tcl 8.5 requirement (4c) Use canonical 'use' command for access to local package (memchan). (5) Constrained the one test requiring Tcl 8.6 (zlib) It is not yet clear to me why the 'stream' helper (to create a large file) is so complex. check-in: 624a1ee7b8 user: andreask tags: pyk-tar
Changes
Unified Diff Ignore Whitespace Patch
Changes to modules/tar/tar.tcl.
52
53
54
55
56
57
58
59
60

61







62



63

64


65


66
67
68
69
70
71
72
	skip $ch $off
	return
    }
    seek $ch $off $wh
    return
}

proc ::tar::skip {ch len} {
    while {$len>0} {

	set readsize [string length [read $ch $len]]







	if {$readsize == 0 && [eof $ch]} {



	    return

	}


	incr len -$readsize


    }
    return
}

proc ::tar::readHeader {data} {
    binary scan $data a100a8a8a8a12a12a8a1a100a6a2a32a32a8a8a155 \
                      name mode uid gid size mtime cksum type \







|
|
>
|
>
>
>
>
>
>
>
|
>
>
>
|
>
|
>
>
|
>
>







52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
	skip $ch $off
	return
    }
    seek $ch $off $wh
    return
}

proc ::tar::skip {ch skipover} {
    while {$skipover > 0} {
	set requested $skipover

	# Limit individual skips to 64K, as a compromise between speed
	# of skipping (Number of read requests), and memory usage
	# (Note how skipped block is read into memory!). While the
	# read data is immediately discarded it still generates memory
	# allocation traffic, gets copied, etc. Trying to skip the
	# block in one go without the limit may cause us to run out of
	# (virtual) memory, or just induce swapping, for nothing.

	if {$requested > 65536} {
	    set requested 65536
	}

	set skipped [string length [read $ch $requested]]

	# Stop in short read into the end of the file.
	if {!$skipped && [eof $ch]} break

	# Keep track of how much is (not) skipped yet.
	incr skipover -$skipped
    }
    return
}

proc ::tar::readHeader {data} {
    binary scan $data a100a8a8a8a12a12a8a1a100a6a2a32a32a8a8a155 \
                      name mode uid gid size mtime cksum type \