Check-in [16ec396728]

# TIP 479: Add Named Procedures as a New Command in Tcl (procx)
	Author:         Sean Woods <[email protected]>
	State:          Draft
	Type:           Project
	Vote:           Pending
	Created:        23-Oct-2017
	Post-History:
	Keywords:       Tcl,procedure,argument handling
................................................................................
option database. Also, those option/value pairs go directly into C data structures,
they at no time feed a script with local variables.

# Specification

Currently if the last parameter named *args* is given a default value, that information
is essentially ignored by the workings inside of tclProc.c. This spec calls for storing
a dict stored where a default would go for the args parameter. Each key in the dict











will be the name of a local variable that the parameter will populate. The values in the
dict wil be a key/value set of configuration option for the named parameter system.


Developers can specify if a named parameter has a default if not given. They can also
specify if the parameter is required. A non-mandatory parameter will not be mapped to
a local variable, but it is understood that developers may want to advertise their
presence for documentation purposes.

		proc foo {{{args {
		   bar {default: baz}
		}}}} {
		   puts $bar
		}
		% foo
		> baz
		% foo bar bing
		> bing

The primary use case for this tip are new functions that take only named parameters. This
TIP also calls for the creation of a new command, provisionally named *procx*. _procx_
will accept the same number of arguments as _proc_, but the format for the argument
spec will be a dict instead of a list. For TclOO methods, we will also create an
methodx as a shortcut to:

		oo::define myclass method namemethod {{{args {
		   bar {default: baz}
		}}}} { ... }

The workings of procx and methodx are essentially:

		proc procx {name argspec body} { proc $name [list [list args $argspec]] $body }

Named parameters are designed to be deliverable in any order. Named parameters
can also be omitted. As such it is necessary to provide more introspection than the
current incarnation of **proc** provides. If the procedure requires
more than the default behavior or populating local variables with the argumemnt given,
it can introspect with **$args** with **\[dict exists\]** to detect if an argument was
given at all. The argument spec that was given by the developer will also be available
as the local variable **argspec**.

		% procx p {a {default: {}}} { return $args }
		% p
		a {}
		% p foo bar
		a {} foo bar

Given these rules, we can get into non-intuitive failures. For instance, what would
happen if a default value was not provided for an argument, and the command call
does not provide one.

		% procx q {a {default: {}} b {}} {
		  # Proc assumes b will be given
		  set result [list $a $b]
		  foreach {f v} $args {
		    if {$item ni [dict keys ${argspec}]} {
		       lappend result $item [dict get $args $item]
		    }
		  }
................................................................................

It would be helpful to have that field (b) treated as a mandatory argument,
and have the error thrown in the procedure's argument parser, not in the procedure
itself. After some play testing with the rules I have worked out what properties will
be tracked for each named parameter, and how those properties are calculated if not
specified directly in the options.

## Configuration Options for Named Parameters

The following general rules apply to all values given in the configuration dict for
a named parameter.

1. If a value for any option is given as an empty string, it is inferred to be null.
2. A variable will be created with the same name as the named parameter.
3. A field that is given, but not otherwise reserved by the spec, is ignored by the parser but made available for introspection via the **argspec** dictionary.
................................................................................
Type: value
Default: null

When non-null, if a named parameter is missing from the call assume the value specified.

Example:

		% procx u {
		  a { default: {A useful value} }
		} {
		  puts $a
		}
		% u
		> A useful value
		% u a {Less usefull}
................................................................................
the following rules apply to aliases:

1. If a parameter with the canonical name is given, alternatives will be ignored.
2. If multiple non-canonical parameters are given, the last value given will be the value for the canonical field.

Example:

		% procx u {
		  a {
		    default: {A useful value}
		    aliases: {alpha A}
		  }
		} {
		  puts [list $a $args]
		}
................................................................................

This TIP will be rolled out in 3 stages.

## Stage 1 - Pure Tcl (finished)

Stage 1 is a pure-tcl implementation to allow the community to try out the rules and
see if this new concept is a good fit for Tcl. The implementation will use a macro-like
template to place the implementation rules as a pre-amble to the body given to procx or
methodx. This implementation will affect line numbers in traces, but should provide
the new features without introducing a major impact on performance.

This implementation has been posted to the http://wiki.tcl.tk/49071 and will be kept up
to date.

## State 2 - C Extension (finished)

................................................................................
      cd tip479
      fossil clone http://fossil.etoyoc.com/fossil/tip479 tip479.fossil
      fossil open tip479.fossil
      tclsh make.tcl all

## State 3 - Core patch

Stage 3 will introduce named parameters as a feature of the core. Modifications
have been checked into the Tcl core as the tip479 branch.


At present if the last positional argument is named *args* a default value
for that field, while recorded, is ignored. We will utilize that space to
overload what arguments the procedure will accept beyond the positional arguments.

Within tclProc.c are modifications to read that argument spec (if given).

### TclCreateProc

In TclCreateProc, if an named argument spec was registered for the function, additional
*localPtr* entries are registered beyond *args* to hold the expected values added by
the spec.

................................................................................
named argument spec registered. If a spec was present, after it performs its
normal population of local variable matching the parameters for the procedure,
the function calls *ProcEatArgs* which pairs key/value pairs beyond the last positional
parameter with registered name parameters (and their associated local variable.

### init.tcl

Two macros for procx and methodx are placed in the init.tcl file to
ensure those facilities are present at runtime. The community is presently
in a debate as to whether seperate commands are the best approach, or modifying
proc and oo::define::method to accept flags is better.

### info argspec

In the info namespace, add an *argspec* command to return the named argument spec for
................................................................................
    }

## Extra Metadata for argument handling

The standard only defines the behavior for several properties for parameters, but
does not prohibit storing additional properties. Consider:

    procx prettyproc {
      color {
        comment: {Color of the button}
        type: color
        default: green
      }
      flavor {
        comment: {What flavor candy emerges when pressed.}
................................................................................
         if {$flavor ni $flavors} {
            error "Invalid flavor $flavor. Valid: random OR $flavors"
         }
      }
    }

# Spec Revisions












2017-11-03 - Renamed @proc to procx, @args to argsx and @method to methodx. Removed
the internal variable @spec from bodies. Added

2017-10-28 - Removed the *variable* option. The core implementation really needs
a canonical name for the variable in the index. If a user really wants a different
name in the arguments, we have the aliases facility. Also clarified that this
tip will be modifying how a default for the args parameter is interpreter and
that procxs and methodx are just convenience wrappers.

# TIP 479: Add Named Procedures as a New Command in Tcl (dictargs::proc)
	Author:         Sean Woods <[email protected]>
	State:          Draft
	Type:           Project
	Vote:           Pending
	Created:        23-Oct-2017
	Post-History:
	Keywords:       Tcl,procedure,argument handling
................................................................................
option database. Also, those option/value pairs go directly into C data structures,
they at no time feed a script with local variables.

# Specification

Currently if the last parameter named *args* is given a default value, that information
is essentially ignored by the workings inside of tclProc.c. This spec calls for storing
information where a default would go for the args parameter. This information is a list
with two fields: *namespace* and *spec*.

*namespace* is a Tcl namespace which contains the command *parse*. Parse accepts two
arguments, a specification and list containing the contents of *args*. How that
specification is interpreted is left to the imagination of the developer.

This spec defines a reference implementation called dictargs.

# dictargs

Under dictargs the specification for non positional arguments is a dict. Each key in the
dict will be the name of a local variable that the parameter will populate. The values
in the dict will be a key/value set of configuration option for the named parameter
system.

Developers can specify if a named parameter has a default if not given. They can also
specify if the parameter is required. A non-mandatory parameter will not be mapped to
a local variable, but it is understood that developers may want to advertise their
presence for documentation purposes.

		proc foo {{{args {dictargs {
		   bar {default: baz}
		}}}}} {
		   puts $bar
		}
		% foo
		> baz
		% foo bar bing
		> bing

The primary use case for this tip are new functions that take only named parameters. This
TIP also calls for the creation of a new command, provisionally named *dictargs::proc*. _dictargs::proc_
will accept the same number of arguments as _proc_, but the format for the argument
spec will be a dict instead of a list. For TclOO methods, we will also create an
dictargs::method as a shortcut to:

		oo::define myclass method namemethod {{{args {
		   bar {default: baz}
		}}}} { ... }

The workings of dictargs::proc and dictargs::method are essentially:

		proc dictargs::proc {name argspec body} { proc $name [list [list args $argspec]] $body }

Named parameters are designed to be deliverable in any order. Named parameters
can also be omitted. As such it is necessary to provide more introspection than the
current incarnation of **proc** provides. If the procedure requires
more than the default behavior or populating local variables with the argumemnt given,
it can introspect with **$args** with **\[dict exists\]** to detect if an argument was
given at all. The argument spec that was given by the developer will also be available
as the local variable **argspec**.

		% dictargs::proc p {a {default: {}}} { return $args }
		% p
		a {}
		% p foo bar
		a {} foo bar

Given these rules, we can get into non-intuitive failures. For instance, what would
happen if a default value was not provided for an argument, and the command call
does not provide one.

		% dictargs::proc q {a {default: {}} b {}} {
		  # Proc assumes b will be given
		  set result [list $a $b]
		  foreach {f v} $args {
		    if {$item ni [dict keys ${argspec}]} {
		       lappend result $item [dict get $args $item]
		    }
		  }
................................................................................

It would be helpful to have that field (b) treated as a mandatory argument,
and have the error thrown in the procedure's argument parser, not in the procedure
itself. After some play testing with the rules I have worked out what properties will
be tracked for each named parameter, and how those properties are calculated if not
specified directly in the options.

## Configuration Options for Dictargs Named Parameters

The following general rules apply to all values given in the configuration dict for
a named parameter.

1. If a value for any option is given as an empty string, it is inferred to be null.
2. A variable will be created with the same name as the named parameter.
3. A field that is given, but not otherwise reserved by the spec, is ignored by the parser but made available for introspection via the **argspec** dictionary.
................................................................................
Type: value
Default: null

When non-null, if a named parameter is missing from the call assume the value specified.

Example:

		% dictargs::proc u {
		  a { default: {A useful value} }
		} {
		  puts $a
		}
		% u
		> A useful value
		% u a {Less usefull}
................................................................................
the following rules apply to aliases:

1. If a parameter with the canonical name is given, alternatives will be ignored.
2. If multiple non-canonical parameters are given, the last value given will be the value for the canonical field.

Example:

		% dictargs::proc u {
		  a {
		    default: {A useful value}
		    aliases: {alpha A}
		  }
		} {
		  puts [list $a $args]
		}
................................................................................

This TIP will be rolled out in 3 stages.

## Stage 1 - Pure Tcl (finished)

Stage 1 is a pure-tcl implementation to allow the community to try out the rules and
see if this new concept is a good fit for Tcl. The implementation will use a macro-like
template to place the implementation rules as a pre-amble to the body given to dictargs::proc or
dictargs::method. This implementation will affect line numbers in traces, but should provide
the new features without introducing a major impact on performance.

This implementation has been posted to the http://wiki.tcl.tk/49071 and will be kept up
to date.

## State 2 - C Extension (finished)

................................................................................
      cd tip479
      fossil clone http://fossil.etoyoc.com/fossil/tip479 tip479.fossil
      fossil open tip479.fossil
      tclsh make.tcl all

## State 3 - Core patch

The original specification required modifications to the core. These modifications,
while they did not seem to impact normal operations, also did not seem to actually
improve performance over the pure-tcl or C compiled extension.


The modifications




### TclCreateProc

In TclCreateProc, if an named argument spec was registered for the function, additional
*localPtr* entries are registered beyond *args* to hold the expected values added by
the spec.

................................................................................
named argument spec registered. If a spec was present, after it performs its
normal population of local variable matching the parameters for the procedure,
the function calls *ProcEatArgs* which pairs key/value pairs beyond the last positional
parameter with registered name parameters (and their associated local variable.

### init.tcl

Two macros for dictargs::proc and dictargs::method are placed in the init.tcl file to
ensure those facilities are present at runtime. The community is presently
in a debate as to whether seperate commands are the best approach, or modifying
proc and oo::define::method to accept flags is better.

### info argspec

In the info namespace, add an *argspec* command to return the named argument spec for
................................................................................
    }

## Extra Metadata for argument handling

The standard only defines the behavior for several properties for parameters, but
does not prohibit storing additional properties. Consider:

    dictargs::proc prettyproc {
      color {
        comment: {Color of the button}
        type: color
        default: green
      }
      flavor {
        comment: {What flavor candy emerges when pressed.}
................................................................................
         if {$flavor ni $flavors} {
            error "Invalid flavor $flavor. Valid: random OR $flavors"
         }
      }
    }

# Spec Revisions
2018-10-19 - Replaced the idea that this tip would be providing the only means of
named argument parsing with the concept that this tip would provide the shims for
developers to provide such a mechanism, as well as provide a reference implementation
called "dictargs".

Renamed procx to dictargs::proc and methodx to dictargs::method, and clarified that
argument parsing will be performed by dictargs::parse

Removed the core patches, and instead casting this TIP as an information one for
developers who wish to either exploit dictargs or utilize their own non-positional
argument processing scheme.

2017-11-03 - Renamed @proc to procx, @args to argsx and @method to methodx. Removed
the internal variable @spec from bodies. Added

2017-10-28 - Removed the *variable* option. The core implementation really needs
a canonical name for the variable in the index. If a user really wants a different
name in the arguments, we have the aliases facility. Also clarified that this
tip will be modifying how a default for the args parameter is interpreter and
that procx and methodx are just convenience wrappers.