Tcl Source Code

View Ticket
Login
Ticket UUID: 556837
Title: nested tests are not counted correctly
Type: Bug Created on: 2002-05-16 16:02:54
Submitter: vincentdarley Assigned to: dgp
Subsystem: 34. tcltest Package Severity:
Priority: 3 Low Last modified: 2002-06-26 07:20:40
Status: Open Closed by:
Resolution: None Closed on:
Version: obsolete: 8.4a5
Description:
If one test executes a number of others, the 'inside' 
tests are not reported in the statistics at the end of 
the test run.

A complex example of this is available in the tclvfs 
test suite (vfsArchive.test)

But a simple example is as follows:

% package require tcltest
% tcltest::test name {blah} { tcltest::test in in 
{expr 1} {0} } {1}


==== in in FAILED
==== Contents of test case:
expr 1
---- Result was:
1
---- Result should have been (exact matching):
0
==== in FAILED


==== name blah FAILED
==== Contents of test case:
 tcltest::test in in {expr 1} {0} 
---- Result was:
0
---- Result should have been (exact matching):
1
==== name FAILED

0

% tcltest::cleanupTests 1
:Total1Passed0Skipped0
Failed1
Sourced 0 Test Files.

But of course this should say 'total 2', 'failed 2'.
User Comments:
dgp added on 2002-05-21 00:16:51:
Logged In: YES 
user_id=80530

This is one of the KNOWN ISSUES mentioned
in the tcltest 2 documentation.

Can you provide an example -- other than
testing the tcltest package itself -- where
it is useful for a test to run a test?

While it would be nice to overcome this
limitation, I don't see it as a high priority
matter.

vincentdarley added on 2002-05-21 00:48:36:
Logged In: YES 
user_id=32170

An example: sure, as mentioned above, this capability is 
used in tclvfs.  Tclvfs contains a fair number of tests of 
the way a given filesystem works, etc.  Of course what you 
really want to do as well is check whether /any/ filesystem 
behaves properly.  So, the way vfsArchive.test does this is 
by creating a .zip archive of the test suite and running 
the tests from within that archive.  Then it creates a 
metakit archive of the suite and runs those, etc.  Each of 
these archive-tests can be considered a 'meta-test' (and is 
implemented as a tcltest::test) but each in turn actually 
causes dozens of other tests to be executed.  This is a 
very natural, clean way of performing this task.

sorry for not reading the 'known issues'...

dgp added on 2002-06-26 07:20:40:
Logged In: YES 
user_id=80530


I took a brief look at vfsArchive.test.  Is that something
one could do with slave interpreters?  That might
be a way to work around this deficiency in tcltest.