Tcl Source Code

View Ticket
Login
2026-10-17
08:44 • Ticket [ce4483c2cb] Audit GetFormatSpec for size_t status still Open with 4 ... artifact: 7185c84cc3 user: serhiy.storchaka
08:44 • Ticket [2788582fff] binary ugly formatSpec parsing status still Open with ... artifact: 370117495b user: serhiy.storchaka
08:43
Fix [2788582fff] and [ce4483c2cb]: [binary format] with huge counts or offsets panicked or returned ... leaf check-in: dd5cd86e81 user: serhiy.storchaka tags: binary-format-size-overflow
2009-06-15
16:16 • Ticket [2788582fff] binary ugly formatSpec parsing status still Open with ... artifact: 31202891cc user: dkf
2009-05-11
20:38 • Ticket [2788582fff]: 4 changes artifact: 87365f95bf user: dgp
16:21 • Ticket [2788582fff]: 4 changes artifact: 1bd86b6141 user: dkf
2009-05-08
02:21 • Ticket [2788582fff]: 4 changes artifact: 7336f80cde user: dgp
2009-05-07
18:51 • New ticket [2788582fff]. artifact: 9299e69401 user: dgp

Ticket UUID: 2788582
Title: [binary] ugly formatSpec parsing
Type: Bug Created on: 2009-05-07 18:51:35
Submitter: dgp Assigned to: dgp
Subsystem: 12. ByteArray Object Severity:
Priority: 5 Medium Last modified: 2026-10-17 08:44:34
Status: Open Closed by:
Resolution: None Closed on:
Version: 8.6.18, 9.0.5, 9.1.0
Description:
The formatSpec parser for [binary format|scan] uses
strtoul() to parser the count part, but then stores in an
int.  This yields ugly, and platform dependent results:

On 32-bit system:

% binary scan \x41 a0 a
1
% set a
% binary scan \x41 a1 b
1
% set b
A
% binary scan \x41 a2 c
0
% binary scan \x41 a[expr 0x7fffffff] c
0
% binary scan \x41 a[expr 0x80000000] c
1
% set c
% binary scan \x41 a[expr 0x80000001] d
1
% set d
% binary scan \x41 a[expr 0xfffffffd] e
1
% set e
% binary scan \x41 a[expr 0xfffffffe] f; # BINARY_NOCOUNT
1
% set f
A
% binary scan \x41 a[expr 0xffffffff] g; # BINARY_ALL
1
% set g
A
% binary scan \x41 a[expr 0x100000000] h
1
% set h
A
% binary scan \x41 a[expr 0x100000001] i
1
% set i
A
% binary scan \x41 a[expr 0x100000002] j
1
% set j
A


On 64-bit system:

% binary scan \x41 a0 a
1
% set a
% binary scan \x41 a1 b
1
% set b
A
% binary scan \x41 a2 c
0
% binary scan \x41 a[expr 0x7fffffff] c
0
% binary scan \x41 a[expr 0x80000000] c
1
% set c
% binary scan \x41 a[expr 0x80000001] d
1
% set d
% binary scan \x41 a[expr 0xfffffffd] e
1
% set e
% binary scan \x41 a[expr 0xfffffffe] f; # BINARY_NOCOUNT
1
% set f
A
% binary scan \x41 a[expr 0xffffffff] g; # BINARY_ALL
1
% set g
A
% binary scan \x41 a[expr 0x100000000] h
1
% set h
% binary scan \x41 a[expr 0x100000001] i
1
% set i
A
% binary scan \x41 a[expr 0x100000002] j
0

This has always been ugly and nonportable.
Seems even more so now that bignums are
the norm in many places.

Good news is it's unlikely programs get here
on purpose, and those that were intentionally trying to
parse such long bytearrays are about to
either run out of memory or overrun Tcl's
limits on value storage anyway.

Reporting just to get it on the record.  Also
someone might want to look closer if there
are cases which could lead to a crash.
User Comments:
dgp added on 2009-05-08 02:21:25:
besides the problems noted, the current
GetFormatSpec() routine permits the
return of negative values for "count" and
those are the values that have the power
to make Tcl_SetObjLength panic.

It may be that when adding code to prevent
T_SOL panics, fixing this may be the simplest
route to success.

dkf added on 2009-05-11 16:21:59:
Perhaps it should use strtol() instead (we know there's no minus sign anyway).

dgp added on 2009-05-11 20:38:27:
I have a patch near ready; thanks.

serhiy.storchaka added on 2026-10-17 08:44:34:

The [binary scan] part is fixed since 8.6.7 ([56c454c814]). [binary format] still adds the field sizes without checking for overflow: [binary format x99999999999999999999] and [binary format @9223372036854775807] panic in 9.0 and 9.1, and in 8.6 [binary format x2147483647x2147483647] returns an empty string. Proposed fix in branch binary-format-size-overflow, with [ce4483c2cb]: report "max size for a Tcl value exceeded" instead. Tests binary-81.*.