| Ticket UUID: | 2788582 | ||
| Title: | [binary] ugly formatSpec parsing | ||
| Type: | Bug | Created on: | 2009-05-07 18:51:35 |
| Submitter: | dgp | Assigned to: | dgp |
| Subsystem: | 12. ByteArray Object | Severity: | |
| Priority: | 5 Medium | Last modified: | 2026-10-17 08:44:34 |
| Status: | Open | Closed by: | |
| Resolution: | None | Closed on: | |
| Version: | 8.6.18, 9.0.5, 9.1.0 | ||
| Description: | ||||
The formatSpec parser for [binary format|scan] uses strtoul() to parser the count part, but then stores in an int. This yields ugly, and platform dependent results: On 32-bit system: % binary scan \x41 a0 a 1 % set a % binary scan \x41 a1 b 1 % set b A % binary scan \x41 a2 c 0 % binary scan \x41 a[expr 0x7fffffff] c 0 % binary scan \x41 a[expr 0x80000000] c 1 % set c % binary scan \x41 a[expr 0x80000001] d 1 % set d % binary scan \x41 a[expr 0xfffffffd] e 1 % set e % binary scan \x41 a[expr 0xfffffffe] f; # BINARY_NOCOUNT 1 % set f A % binary scan \x41 a[expr 0xffffffff] g; # BINARY_ALL 1 % set g A % binary scan \x41 a[expr 0x100000000] h 1 % set h A % binary scan \x41 a[expr 0x100000001] i 1 % set i A % binary scan \x41 a[expr 0x100000002] j 1 % set j A On 64-bit system: % binary scan \x41 a0 a 1 % set a % binary scan \x41 a1 b 1 % set b A % binary scan \x41 a2 c 0 % binary scan \x41 a[expr 0x7fffffff] c 0 % binary scan \x41 a[expr 0x80000000] c 1 % set c % binary scan \x41 a[expr 0x80000001] d 1 % set d % binary scan \x41 a[expr 0xfffffffd] e 1 % set e % binary scan \x41 a[expr 0xfffffffe] f; # BINARY_NOCOUNT 1 % set f A % binary scan \x41 a[expr 0xffffffff] g; # BINARY_ALL 1 % set g A % binary scan \x41 a[expr 0x100000000] h 1 % set h % binary scan \x41 a[expr 0x100000001] i 1 % set i A % binary scan \x41 a[expr 0x100000002] j 0 This has always been ugly and nonportable. Seems even more so now that bignums are the norm in many places. Good news is it's unlikely programs get here on purpose, and those that were intentionally trying to parse such long bytearrays are about to either run out of memory or overrun Tcl's limits on value storage anyway. Reporting just to get it on the record. Also someone might want to look closer if there are cases which could lead to a crash. | ||||
| User Comments: | ||||
dgp added on 2009-05-08 02:21:25:
besides the problems noted, the current GetFormatSpec() routine permits the return of negative values for "count" and those are the values that have the power to make Tcl_SetObjLength panic. It may be that when adding code to prevent T_SOL panics, fixing this may be the simplest route to success. dkf added on 2009-05-11 16:21:59:
Perhaps it should use strtol() instead (we know there's no minus sign anyway). dgp added on 2009-05-11 20:38:27:
I have a patch near ready; thanks. serhiy.storchaka added on 2026-10-17 08:44:34:
The [binary scan] part is fixed since 8.6.7 ([56c454c814]). [binary format] still adds the field sizes without checking for overflow: [binary format x99999999999999999999] and [binary format @9223372036854775807] panic in 9.0 and 9.1, and in 8.6 [binary format x2147483647x2147483647] returns an empty string. Proposed fix in branch binary-format-size-overflow, with [ce4483c2cb]: report "max size for a Tcl value exceeded" instead. Tests binary-81.*. | ||||
