 Author:         Donal K. Fellows <[email protected]>
 State:          Final
 Type:           Project
 Tcl-Version:    9.1
 Vote:           Done
 Created:        31-Oct-2025
 Keywords:       Tcl
 Tcl-Branch:     tip-735
 Vote-Summary:   Accepted 8/0/0
 Votes-For:      DKF, HO, JN, KW, AN, MC, SL, AM

Abstract

Sometimes, it's useful to pick out a selection of elements from a list according to some boolean predicate. This TIP makes that more convenient.

Rationale and Example

It's not exactly unheard of for code to want to pick items out of a list. Tcl does it inside the ICU interface with:

set unmappedNames {}
foreach tclName [encoding names] {
    # Note entry will always exist. Check if empty
    if {[llength [tclToIcu $tclName]] == 0} {
        lappend unmappedNames $tclName
    }
}

In other cases, lsearch -all -inline is used for that, where the condition can be expressed as a pattern match.

However, that's not very obvious. It would be significantly more obvious if we could instead do:

set unmappedNames [lfilter tclName [encoding names] {
    # Note entry will always exist. Check if empty
    string equal [tclToIcu $tclName] ""
}]

So let's make that work! The body is a script that produces a value interpreted as a boolean, and that we select the values that can be interpreted as true (and reject the ones that are false). Note in the specific example that Tcl 9.1 optimizes the comparison with an empty string case so that it's efficient for testing for empty lists among other things. This is a consequence of TIPs 711 and 720.

The name and functionality are inspired partially by the general filter() predicate in some functional programming languages, given that map() corresponds to lmap.

Specification

Let there be a command, lfilter, with the signature:

lfilter varList0 list0 ?varList list...? bodyScript

The pattern of varList/list items is as for foreach and lmap.

The command will iterate over the items of the list arguments in order, binding the variables from the varLists. For each of those steps, the bodyScript will be evaluated (as if with eval) and the result will be interpreted as a boolean; if it is interpretable as a true value (according to Tcl_GetBoolean) then the value that the the first variable of varList0 was set to (though not the value that it necessarily now contains) will be appended to the result list being built. If the script result is a false value, the item will not be appended.

If writing any variable produces an error, so too will the lfilter command.

If the evaluation of the script yields a TCL_ERROR then the overall lfilter command will produce an error (possibly with the error info extended). If the evaluation yields a TCL_RETURN, so too will the lfilter command. If the evaluation yields a TCL_BREAK then the iteration will stop. If the evaluation yields a TCL_CONTINUE, it will be treated as if the script evaluated to false. User-defined error codes will be passed through.

If no error, return or user-defined condition occurs, the result of the lfilter command will be the list of (lead) items of list0 for which the script evaluated to a true value, in order.

Options Considered and Rejected

The original version of this TIP had the bodyScript be an expression, but this conflicted in semantics with the existing dict filter ... script command. The author considers keeping the number of extra concepts involved in complex filtering to be better.

Unlike dict filter, there is no option to select for filtering other than by script; filtering by glob match or regexp match can be done by lsearch with appropriate options, or string match or regexp can be used in the bodyScript.

Implementation

See the tip-735 branch.

Examples

NB: This is a cut-n-paste from an interactive session running the implementation branch.

% lfilter x [lseq 100] {
    # Is this value a square number?
    expr {round($x**0.5)**2 == $x}
}
0 1 4 9 16 25 36 49 64 81

To select just the strings that are true palindromes:

% lfilter s {abc foo dad} {
    # Is the string exactly equal to itself when reversed
    string equal $s [string reverse $s]
}

Illustrative Approximate Implementations

An approximate scripted version of this command is:

proc lfilter args {
    set test [list eval [lpop args end]]
    set v0 [lindex $args 0 0]
    set cmd [list if $test [list set $v0] else continue]
    tailcall lmap {*}$args $cmd
}

A simpler version that only supports a single variable and list:

proc lfilter_simple {varName list body} {
    upvar 1 $varName var
    set test [list eval $body]
    lmap var $list {
        if {[uplevel 1 $test]} {set var} else continue
    }
}

These are not expected to be full versions.

Copyright

This document has been placed in the public domain.

TIP 735: Simpler List Filtering