Tcl Library Source Code

Documentation
Login
Core Home Blog Docs Downloads Tickets Timeline Branches Tags Wiki

[ Main Table Of Contents | Table Of Contents | Keyword Index | Categories | Modules | Applications ]

NAME

tcllib_devguide - Tcllib - The Developer's Guide

Table Of Contents

SYNOPSIS

Module name code-action doc-action example-action
Application name
Exclude name

DESCRIPTION

Welcome to Tcllib, the Tcl Standard Library. Note that Tcllib is not a package itself. It is a collection of (semi-independent) Tcl packages that provide utility functions useful to a large collection of Tcl programmers.

This document is a guide for developers working on Tcllib, i.e. maintainers fixing bugs, extending the collection's functionality, etc.

Please read

  1. Tcllib - How To Get The Sources and

  2. Tcllib - The Installer's Guide

first, if that was not done already.

Here we assume that the sources are already available in a directory of your choice, and that you not only know how to build and install them, but also have all the necessary requisites to actually do so. The guide to the sources in particular also explains which source code management system is used, where to find it, how to set it up, etc.

Commitments

Contributor

As a contributor to Tcllib you are committing yourself to:

  1. keep the guidelines written down in Tcl Community - Kind Communication in your mind. The main point to take away from there is to be kind to each other.

  2. Your contributions getting distributed under a BSD/MIT license. For the details see Tcllib - License

Contributions are made by entering tickets into our tracker, providing patches, bundles or branches of code for inclusion, or posting to the Tcllib related mailing lists.

Maintainer

When contributing one or more packages for full inclusion into Tcllib you are committing yourself to

  1. Keep the guidelines written down in Tcl Community - Kind Communication (as any contributor) in your mind. The main point to take away from there is to be kind to each other.

  2. Your packages getting distributed under a BSD/MIT license. For the details see Tcllib - License

  3. Maintenance of the new packages for a period of two years under the following rules, and responsibilities:

    1. A maintainer may step down after the mandatory period as they see fit.
    2. A maintainer may step down before the end of the mandatory period, under the condition that a replacement maintainer is immediately available and has agreed to serve the remainder of the period, plus their own mandatory period (see below).
    3. When stepping down without a replacement maintainer taking over the relevant packages have to be flagged as unmaintained.

    4. When a replacement mantainer is brought in for a package it is (kept) marked as maintained (again).

      A replacement maintainer is bound by the same rules as the original maintainer, except that the mandatory period of maintenance is shortened to one year.

    5. For any unmaintained package a contributor interested in becoming its maintainer can become so by flagging them as maintained with their name and contact information, committing themselves to the rules of a replacement maintainer (see previous point).

    6. For any already maintained package a contributor interested in becoming a co-maintainer can become so with the agreement of the existing maintainer(s), committing themselves to the rules of a replacement maintainer (see two points previous).

    The responsibilities as a maintainer include:

    1. Watching Tcllib's ticket tracker for bugs, bug fixes, and feature requests related to the new packages.
    2. Reviewing the aforementioned tickets, rejecting or applying them
    3. Coordination and discussion with ticket submitter during the development and/or application of bug fixes.

  4. Follow the Branching and Workflow of this guide.

Branching and Workflow

Package Dependencies

Regarding packages and dependencies between them Tcllib occupies a middle position between two extremes:

  1. On one side a strongly interdependent set of packages, usually by a single author, for a single project. Looking at my (Andreas Kupries) own work examples of such are Marpa, CRIMP, Kinetcl, etc.

    For every change the author of the project handles all the modifications cascading from any incompatibilities it introduced to the system.

  2. On the other side, the world of semi-independent projects by many different authors where authors know what packages their own creations depend on, yet usually do not know who else depends on them.

    The best thing an author making an (incompatible) change to their project can do is to for one announce such changes in some way, and for two use versioning to distinguish the code before and after the change.

    The world is then responsible for adapting, be it by updating their own projects to the new version, or by sticking to the old.

As mentioned already, Tcllib lives in the middle of that.

While we as maintainers cannot be aware of all users of Tcllib's packages, and thus have to rely on the mechanisms touched on in point 2 above for that, the dependencies between the packages contained in Tcllib are a different matter.

As we are collectively responsible for the usability of Tcllib in toto to the outside world, it behooves us to be individually mindful even of Tcllib packages we are not directly maintaining, when they depend on packages under our maintainership. This may be as simple as coordinating with the maintainers of the affected packages. It may also require us to choose how to adapt affected packages which do not have maintainers, i.e. modify them to use our changed package properly, or modify them to properly depend on the unchanged version of our package.

Note that the above is not only a chore but an opportunity as well. Additional insight can be had by forcing ourselves to look at our package and the planned change(s) from an outside perspective, to consider the ramifications of our actions on others in general, and on dependent packages in particular.

Trunk

The management and use of branches is an important part of working with a Distributed Version Control System (DVCS) like fossil.

For Tcllib the main branch of the collection is trunk. In git this branch would be called master, and this is exactly the case in the github mirror of Tcllib.

To properly support debugging each commit on this branch has to pass the entire testsuite of the collection. Using bisection to determine when an issue appeared is an example of an action made easier by this constraint.

This is part of our collective responsibility for the usability of Tcllib in toto to the outside world. As fossil has no mechanism to enforce this condition this is handled on the honor system for developers and maintainers.

To make the task easier Tcllib comes with a tool ("sak.tcl") providing a number of commands in support. These commands are explained in the following sections of this guide.

While it is possible and allowed to commit directly to trunk remember the above constraint regarding the testsuite, and the coming notes about other possible issues with a commit.

Branches

Given the constraints placed on the trunk branch of the repository it is (strongly) recommended to perform any development going beyond trivial changes on a non-trunk branch.

Outside of the trunk developers are allowed to commit intermediate broken states of their work. Only at the end of a development cycle, when the relevant branch is considered ready for merging, will it be necessary to perform full the set of validations ensuring that the merge to come will create a good commit on trunk.

Note that while a review from a second developer is not a required condition for merging a branch it is recommended to seek out such an independent opinion as a means of cross-checking the work.

It also recommended to give any new branch a name which aids in determining additional details about it. Examples of good things to stick into a branch name would be

Further, while most development branches are likely quite short-lived, no prohibitions exist against making longer-lived branches. Creators should however be mindful that the longer such a branch exists without merges the more divergent they will tend to be, with an associated increase in the effort which will have to be spent on either merging from and merging to trunk.

Working with Branches

In the hope of engendering good work practices now a few example operations which will come up with branches, and their associated fossil command (sequences).

Version numbers

In Tcllib all changes to a package have to come with an increment of its version number. What part is incremented (patchlevel, minor, major version) depends on the kind of change made. With multiple changes in a commit the highest "wins".

When working in a development branch the version change can be deferred until it is time to merge, and then has to cover all the changes in the branch.

Below a list of the kinds of changes and their associated version increments:

Note that a commit containing a version increment has to mention the new version number in its commit message, as well as the kind of change which caused it.

Note further that the version number of a package currently exists in three places. An increment has to update all of them:

  1. The package implementation.

  2. The package index ("pkgIndex.tcl")

  3. The package documentation.

The "sak.tcl" command validate version helps finding discrepancies between the first two. All the other validate methods are also of interest to any developer. Invoke it with

sak.tcl help validate

to see their documentation.

Structural Overview

Main Directories

The main directories in the Tcllib toplevel directory and of interest to a developer are:

More Directories

Top Files

File Types

The most common file types, by file extension, are:

Testsuite Tooling

Testsuites in Tcllib are based on Tcl's standard test package tcltest, plus utilities found in the directory "modules/devtools"

Tcllib developers invoke the suites through the test run method of the "sak.tcl" tool, with other methods of test providing management operations, for example setting a list of standard Tcl shells to use.

Invoke the testsuites of a specific module

Invoke either

./sak.tcl test run foo

or

./sak.tcl test run modules/foo

to invoke the testsuites found in a specific module "foo".

Invoke the testsuites of all modules

Invoke the tool without a module name, i.e.

./sak.tcl test run

to invoke the testsuites of all modules.

Detailed Test Logs

In all the previous examples the test runner will write a combination of progress display and testsuite log to the standard output, showing for each module only the tests that passed or failed and how many of each in a summary at the end.

To get a detailed log, it is necessary to invoke the test runner with additional options.

For one:

./sak.tcl test run --log LOG foo

While this shows the same short log on the terminal as before, it also writes a detailed log to the file "LOG.log", and excerpts to other files ("LOG.summary", "LOG.failures", etc.).

For two:

./sak.tcl test run -v foo

This writes the detailed log to the standard output, instead of the short log.

Regardless of form, the detailed log contains a list of all test cases executed, which failed, and how they failed (expected versus actual results).

Shell Selection

By default the test runner will use all the Tcl shells specified via test add to invoke the specified testsuites, if any. If no such are specified it will fall back to the Tcl shell used to run the tool itself.

Use option --shell to explicitly specify the Tcl shell to use, like

./sak.tcl test run --shell /path/to/tclsh ...

Help

Invoke the tool as

./sak.tcl help test

to see the detailed help for all methods of test, and the associated options.

Documentation Tooling

The standard format used for documentation of packages and other things in Tcllib is doctools. Its supporting packages are a part of Tcllib, see the directories "modules/doctools" and "modules/dtplite". The latter is an application package, with the actual application "apps/dtplite" a light wrapper around it.

Tcllib developers gain access to these through the doc method of the "sak.tcl" tool, another (internal) wrapper around the "modules/dtplite" application package.

Generate documentation for a specific module

Invoke either

./sak.tcl doc html foo

or

./sak.tcl doc html modules/foo

to generate HTML for the documentation found in the module "foo". Instead of html any other supported format can be used here, of course.

The generated formatted documentation will be placed into a directory "doc" in the current working directory.

Generate documentation for all modules

Invoke the tool without a module name, i.e.

./sak.tcl doc html

to generate HTML for the documentation found in all modules. Instead of html any other supported format can be used here, of course.

The generated formatted documentation will be placed into a directory "doc" in the current working directory.

Available output formats, help

Invoke the tool as

./sak.tcl help doc

to see the entire set of supported output formats which can be generated.

Validation without output

Note the special format validate.

Using this value as the name of the format to generate forces the tool to simply check that the documentation is syntactically correct, without generating actual output.

Invoke it as either

./sak.tcl doc validate (modules/)foo

or

./sak.tcl doc validate

to either check the packages of a specific module or check all of them.

Notes On Writing A Testsuite

While previous sections talked about running the testsuites for a module and the packages therein, this has no meaning if the module in question has no testsuites at all.

This section gives a very basic overview on possible methodologies for writing tests and testsuites.

First there are "drudgery" tests. Written to check absolutely basic assumptions which should never fail.

For example for a command FOO taking two arguments, three tests calling it with zero, one, and three arguments. The basic checks that the command fails if it has not enough arguments, or too many.

After that come the tests checking things based on our knowledge of the command, about its properties and assumptions. Some examples based on the graph operations added during Google's Summer of Code 2009 are:

What was described above via examples is called black-box testing. Test cases are designed and written based on the developer's knowledge of the properties of the algorithm and its inputs, without referencing a particular implementation.

Going further, a complement to black-box testing is white-box. For this we know the implementation of the algorithm, we look at it and design our tests cases so that they force the code through all possible paths in the implementation. Wherever a decision is made we have a test case forcing a specific direction of the decision, for all possible combinations and directions. It is easy to get a combinatorial explosion in the number of needed test-cases.

In practice I often hope that the black-box tests I have made are enough to cover all the paths, obviating the need for white-box tests.

The above should be enough to make it clear that writing tests for an algorithm takes at least as much time as coding the algorithm, and often more time. Much more time. See for example also http://sqlite.org/testing.html, a writeup on how the Sqlite database engine is tested. Another article of interest might be https://www.researchgate.net/publication/298896236. While geared to a particular numerical algorithm it still shows that even a simple-looking algorithm can lead to an incredible number of test cases.

An interesting connection is to documentation. In one direction, the properties checked with black-box testing are exactly the properties which should be documented in the algorithm's man page. And conversely, the documentation of the properties of an algorithm makes a good reference to base the black-box tests on.

In practice test cases and documentation often get written together, cross-influencing each other. And the actual writing of test cases is a mix of black and white box, possibly influencing the implementation while writing the tests. Like writing a test for a condition like startnode not in input graph serving as reminder to put a check for this condition into the code.

Installation Tooling

A last thing to consider when adding a new package to the collection is installation.

How to use the "installer.tcl" script is documented in Tcllib - The Installer's Guide.

Here we document how to extend said installer so that it may install new package(s) and/or application(s).

In most cases only a single file has to be modified, the "support/installation/modules.tcl" holding one command per module and application to install.

The relevant commands are:

If, and only if the above actions are not suitable for the new module then a second file has to be modified, "support/installation/actions.tcl".

This file contains the implementations of the available actions, and is the place where any custom action needed to handle the special circumstances of module has to be added.

This page was generated in about 0.085s by Fossil 2.24 [f4ffefe708] 2024-04-17 14:02:19