Tcl Source Code

View Ticket
Login
Ticket UUID: 9f4feaaaa11646ec51d345a2697de898850c16f9
Title: tcltest test contraints with a dash (-) in the name doesn't work
Type: Bug Created on: 2023-04-24 19:12:35
Submitter: pointsman Assigned to: dgp
Subsystem: 34. tcltest Package Severity: Severe
Priority: 5 Medium Last modified: 2026-10-22 11:24:36
Status: Open Closed by: nobody
Resolution: None Closed on:
Version: 8.6.18, 9.0.5, 9.1.0
Description:
Put the following stripped down example in a new file under tests, say rde.test

if {"::tcltest" ni [namespace children]} {
    package require tcltest

    namespace import -force ::tcltest::*
}

test sometest {some test description} -constraints {
    bug-0306a5563
}  -body {
    set var a
    set var
} -result a

cleanupTests

Now try to run this (say from the unix directory of the same soure tree) with

./tcltest ../tests/all.tcl -file rde.test

(You may even use a Tcl 8.6 tclsh or an installed Tcl 9 tclsh; they all show the same behaviour.)

Here on debian I get:

Test files run in separate interpreters
Running tests that match:  *
Skipping test files that match:  l.*.test
Only running test files that match:  rde.test
Tests began at Mon Apr 24 20:57:21 CEST 2023
rde.test
        1
    bug-0306a5563


Tests ended at Mon Apr 24 20:57:21 CEST 2023
all.tcl:        Total   1       Passed  0       Skipped 1       Failed  0
Sourced 1 Test Files.

Notice the strange out put and the missing skip reason.

To set the constraint on the cmd line to true with

./tcltest ../tests/all.tcl -file rde.test -constraints bug-0306a5563

doesn't change anything on the tests run output (other then the start/end time of
course.)

If I rename the constraint to bug0306a5563 in the example above everything works as expected. The run with 

./tcltest ../tests/all.tcl -file rde.test

returns

[...]
Test files run in separate interpreters
Running tests that match:  *
Skipping test files that match:  l.*.test
Only running test files that match:  rde.test
Tests began at Mon Apr 24 21:03:39 CEST 2023
rde.test

Tests ended at Mon Apr 24 21:03:39 CEST 2023
all.tcl:        Total   1       Passed  0       Skipped 1       Failed  0
Sourced 1 Test Files.
Number of tests skipped for each constraint:
        1       bug0306a5563


so no strange output inbetween and a table with the skip reasons.

Running with

./tcltest ../tests/all.tcl -file rde.test -constraints bug0306a5563

returns

[...]
Only running test files that match:  rde.test
Tests began at Mon Apr 24 21:05:32 CEST 2023
rde.test

Tests ended at Mon Apr 24 21:05:32 CEST 2023
all.tcl:        Total   1       Passed  1       Skipped 0       Failed  0
Sourced 1 Test Files.


The tcltest man page say: "Each constraint has a name, which may be any string".

The example above shows something else.
User Comments:
dgp added on 2023-04-24 20:38:51:
You call the character a dash, but it is also a minus sign.  The
presence of an operator causes tcltest to interpret the -constraints
argument as a constraint expression, a combination of the constraint "bug"
and the constraint "0306a5563".

I'm happy to bring this to an end, either completely, or converting
to expressions only on logic operators instead of all operators.

Just need to bump to tcltest 3.

pointsman added on 2023-04-24 21:26:44:
Well, I quoted what I found in the man page about that already in the ticket:

"Each constraint has a name, which may be any string"

I suspected that it may be something with the dichotomy of the -constraints attribute value "keywordList|expression". But the documentation is mostly silent what "expression" means, perhaps because it recommends to not use it.

Seems, even if it should be not used it comes easily into the way of using tcltest. That's unfortunate.

I have no clue how often and for which cases contraints expressions are used. So I can easily say remove that completely. It may be important to some.

But an expression which _substract_ a boolen value from another boolean doesn't seem a good solution for no matter what problem.

serhiy.storchaka added on 2026-10-20 14:02:40:

The stray output and the missing skip table were fixed by [98f03d1dc3] ([0aaeb9c380]), but on main, 9.0 and 8.6 such names are still parsed as bug - 0306a5563, so -constraints bug-0306a5563 on the command line has no effect.

Proposed fix in branch [98752639ed|tcltest-dash-constraints] (stacked on [e055230591|tcltest-nested-match], for [773924ffff]): a "-" between name characters is part of the name, so bug-1, a-b || c and !a-b work; a "-" next to an operator or a space is still an operator. Documented in tcltest.n, new test tcltest-28.2. No test in the Tcl test suite used "-" as a subtraction in constraints. As suggested, this needs a tcltest version bump, which I'd do once for the series of tcltest fixes.


dgp added on 2026-10-20 19:14:31:
This is not a new feature, but a compatibility breaking change,
so best introduced in a new major release tcltest 3.

Given the freedom to break compat, I would be happier breaking
it harder and eliminating constraint expressions completely.

We can sort out the enthusiasts for the different approaches
in a TIP.

jan.nijtmans added on 2026-10-22 11:24:36:

I already merged the proposed fix to trunk, but - thinking about it - better discuss about this first seperately.