|Title:||Tcl_ParseArgv as a useful non-Tk version of Tk_ParseArgv|
|Submitter:||sbromle||Created on:||2006-03-09 19:29:43|
|Status:||Closed||Last Modified:||2009-07-29 19:54:59|
|Closed on:||2008-12-02 22:40:56|
In porting an existing package based on C to one that uses Tcl as the glue, I've had the desire to have the Tk_ParseArgv functionality without Tk. This lead me to find the following old conversation regarding this: http://www.google.com/url?sa=D&q=http://groups.google.ca/group/comp.lang.tcl/browse_thread/thread/c4fea8f0346cf8ae/036961bf476a3b99%3Flnk%3Dst%26q%3DTcl_ParseArgv%26rnum%3D1%23036961bf476a3b99 The arguments against Tcl_ParseArgv is that if you are using tcl, then just use it as the main loop, and call out to special function C codes as necessary. By doing so, you can use one of the many Tcl Command line parsing tools listed at http://wiki.tcl.tk/1730. Now while the arguments against this functionality are valid within the context of the process' main loop, it breaks down when you want to parse options in stand-alone commands. At least this is how I see it. To be more precise, I have a C library to which I want to provide a Tcl interface. So for each of the functions in the library, I need to write a wrapper in C, to provide that command to Tcl. If I want these commands to have libpopt or Tk-like command options I either have to 1) Parse the command arguments myself (on the C side); or 2) Write additional wrappers to the commands in Tcl, to pre-parse the options. So in the case of (2), I'm making work for myself, when all I really want to be able to do is call Tcl_ParseArgv and be done with it in the original C command wrapper. NEWays, to get things working quickly, I wrote a Tcl_ParseArgv routine, based (of course) heavily on the tkArgv.c source (which implements Tk_ParseArgv), but with all the Tk specific bits taken out. So now my wrappers can be ported very quickly from their current libpopt handled options, to the Tcl_ParseArgv. Summary: 1) Provide a Tcl_ParseArgv routine that can parse a standard argc,argv pair. 2) Useful when functions other than main() wish to parse arguments in argc, argv format. 3) Useful when one wishes to have libpopt style command line arguments to functions. 3a) Build argc,argv from objc,objv. 3b) Parse using Tcl_ParseArgv. 3c) Parse remaining arguments in familiar C fashion. 4) Makes re-writing of C-based wrappers written using libpopt trivial. Code (which can be used as a shared library), is attached.
dkf added on 2009-07-29 19:54:59:
IP - Comment Removed: 126.96.36.199
dkf added on 2008-12-03 05:40:56:
data_type - 210894 This was TIP#265, implemented in 8.6
[email protected] added on 2008-12-03 05:06:38:
sbromle added on 2006-07-14 08:15:03:
File Deleted - 170441:
sbromle added on 2006-07-14 08:15:01:
File Deleted - 184870:
sbromle added on 2006-07-14 08:13:03:
File Added - 184871: tclArgv-tcl8.4.13_20060713.patch
sbromle added on 2006-07-14 08:13:02:
Logged In: YES user_id=1472195 Hi, I've created a patch against the Tcl 8.4.13 sources as you requested. I've made most of the changes that were suggested. One thing outstanding is the addition of support for all of the TCL_LINK* types. As per Don's suggestion, the default help table has been removed, remaining arguments are now gathered if requested. and the flags argument has been dropped. Comments are most welcome. I'll do my best to get this into a form that is acceptable. Sam
sbromle added on 2006-07-14 08:12:49:
File Added - 184870: tclArgv-tcl8.4.13_20060713.patch Logged In: YES user_id=1472195 Hi, I've created a patch against the Tcl 8.4.13 sources as you requested. I've made most of the changes that were suggested. One thing outstanding is the addition of support for all of the TCL_LINK* types. As per Don's suggestion, the default help table has been removed, remaining arguments are now gathered if requested. and the flags argument has been dropped. Comments are most welcome. I'll do my best to get this into a form that is acceptable. Sam
sbromle added on 2006-06-19 02:59:34:
Logged In: YES user_id=1472195 Hi Stephen, Thanks for your reply. It looks like a lot of thought has been put into tclobjv.c, and perhaps it would make a better codebase to adopt into Tcl proper, tho I don't yet really have the Tcl experience to judge. :) If you would indulge me a little further, I'd like to raise a few more questions with you. In my use of tclArgv.c, we've implemented many commands, I guess effectively creating somewhat of a domain specific language. A typical command actually performs many functions and so we've found that our commands often have tens of options. Remembering them all is hard without an automatically generated help. Basically I feel that the existing GetIndexFromObj suffices for typical C-implemented Tcl commands, and in this context, an automaticly generated help is extraneous. However, I figure that if you need the extra complexity of tclArgv for your command parsing, then you'd probably want the automatically generated help. Being able to add descriptions to each option is then a real benefit. Would you consider adding the ability to attach descriptions to each of your options and arguments? (I really like the way you handle the arguments as well, as currenly, tclArgv does not generate automatic help for the arguments, but strapping this onto tclobjv could be a nice way to go.) Requiring a specific order for arguments (that is, coming before all options) is probably a good idea for consistency. Sure tclArgv doesn't care, but any user could get used to it. In one sense I like that your unknown arguments trigger an error. On the other hand, this prevents you from calling a nested command with the same objv. Doing this kind of thing is probably not very common outside of X application programming, so I'm sure we need not worry about it. I like that you already have a complete set of tests available. It is also good that you use Tcl_GetIndexFromObj. Aestetically (which means a trivial change only is required) I would like to see a version that removes all the "Ns" prepending so that it looks more core-Tcl-ish. (Yeah, this point is obvious. :) ) W.r.t. to Don's suggestions for tclArgv, I've made the following changes: (i) Removed TCL_DONT_SKIP_FIRST_ARG. (ii) TCL_ARGV_NO_ABBREV -> TCL_EXACT (iii) Added a &remObjv argument to collect unprocessed arguments, and thus also eliminating the TCL_ARGV_NO_LEFTOVERS flag. (iv) I disagree with ditching the -help option, for the reason I cited above regarding the developer/user already choosing the greater complexity of tclArgv over a Tcl_GetIndexFromObjv-only solution. I'm still continuing work on it. I haven't yet re-cast Tcl_ParseArgv as a thin wrapper around Tcl_ParseObjArgv, nor added the missing data types. I'll do my best to submit a patch with all the changes soon. Meanwhile, if Stephen could provide me with a little package that allowed me to use his version as a little shared library, with the modification to provide dynamic help, I'll merge it in with my application and give it a test drive to see what the users think. I look forward to hearing any suggestions from anyone regarding the suitability (or lack-thereof) of the two options (TclArgv and tclobjv). Cheers, Sam.
sdeasey added on 2006-06-18 03:03:41:
Logged In: YES user_id=87254 Hi Sam. I wrote the anonymous comment (sure I was logged in -- silly source forge) and tclobjv.c in naviserver. Yes, tclobjv.c parses both options and arguments, handling type conversion and defaults for both, and an optional 'args' to gather remaining arguments in. It does this with the Ns_ObjvArgs callback. There's lots of examples of it being used here: http://naviserver.cvs.sourceforge.net/naviserver/naviserver/nsd/tclsched.c?revision=1.5&view=markup You can parse the remaining arguments however you like. > (i) Does the order of options matter? No, which is the expected behaviour, I think. > (ii) Can options be interspersed with arguments? No. All options must come before arguments, which again I think is the expected behaviour. > (iii) What happens in the case of unknown arguments? Not sure what you mean here. If the args supplied don't match the signature, an error message is generated. If you expect an argument of an unusual type, you can use Ns_ObjvObj and the Tcl object will be returned, which you can interpret however you like. > (iv) Is there an automatically generated help? Error messages are automatically generated which descibe the command's signature. Perhaps the signature should be made available, so that commands could implement a -help switch (although this feels wrong to me for commands, fine for command lines). > (v) Do all arguments have to be known in advance? The number of remaining, unparsed arguments can be captured and dealt with however you like (see above). tclobjv.c is the fastest implementation I could come up with (standalone, not including the Tcl byte code compiler), which is important for Tcl commands, though probably not for application command lines. Some things which tclobjv.c does to make things fast are: no memory is allocated for parsing; uses GetIndexFromObj to shimmer switches to speed parsing in the future; splits option and arg parsing into two phases so that un-passed options can be skipped efficiently; direct jump to type conversion callbacks rather than giant switch statement. The last is obviously more felxible, too. The core Ns_ParseObjv() is only ~45 lines. The type conversion callbacks are often trivial wrappers. There's also a complete set of tests available: http://naviserver.cvs.sourceforge.net/naviserver/naviserver/tests/ns_parseargs.test?revision=1.3&view=markup
dgp added on 2006-06-17 03:00:36:
Logged In: YES user_id=80530 For me, it would be easier to review the code if it were a patch against the Tcl sources. Stick the new routines in tclIndexObj.c. Follow the engineering manual, please. The set of types indicated by TCL_ARGV_* is incomplete. Look at the full list of TCL_LINK_* types now supported by Tcl_LinkVar. In fact, can we just have these routines just re-use the TCL_LINK_* values? The Tcl_ParseArgv() and Tcl_ParseArgvObj() routines are separate implementations. Seems to me it would be better to implement one, and make the other a thin wrapper around it. I'd have the Tcl_Obj one be real, and the string-based interface as a wrapper, if we even bother to have it at all. The TCL_DONT_SKIP_FIRST_ARG flag seems pointless. Just expect the caller to pass in the correct value for objv. It's not clear to me that having a "-help" option always recognized is more useful than it is annoying. Who are the expected callers of these routines, and are they expected to want a "-help" more often than not? If we drop that, we drop the whole defaultTable, as well as the need for the TCL_ARGV_NO_DEFAULTS flag. TCL_ARGV_NO_ABBREV has the same meaning as the existing TCL_EXACT, right? Why not re-use that? Rather than require objv to be over-writable, I think I'd be happier with a routine that took a const objv and an optional &remainingObjv argument, for sake of those callers that want to know the unparsed options. With that revision, it would be possible for the caller to manage the reaction to whether or not all objv values are consumed, so TCL_ARGV_NO_LEFTOVERS would no longer be needed either. I think that does away with the need for the flags argument entirely. I guess that's enough for now.
sbromle added on 2006-06-15 05:45:08:
Logged In: YES user_id=1472195 Yep. Naviserver's tclobjv.c seems to provide similar functionality, without the need to make a local copy of the objects. It gets around this by parsing both options and arguments in the call, whereas the tclArgv version (based on TkArgv.c) parses only the options, and repacks the array leaving only arguments and unknown options. This ability to leave unknown arguments and options on the objv array is useful when one wants to strip different options with different functions (though I admit, I haven't needed to do so myself as of yet.) From a usability perspective I'm sure both approaches are valid. I have a few questions that could help bring me up to speed on the naviserver version: (i) Does the order of options matter? (ii) Can options be interspersed with arguments? (iii) What happens in the case of unknown arguments? (iv) Is there an automatically generated help? (v) Do all arguments have to be known in advance? Equivalently, can the number of arguments be dynamic, depending on, say, the value of options? (tclArgv allows this as the arguments are just repackaged for subsequent parsing by the caller). My bias (at least in terms of for inclusion into Tcl proper) is that the function be as stand-alone and as simple as possible. In this sense, the tclArgv.c file below might be seen as a more closed-form solution than the tclobjv.c version with its additional typedefs and subfunctions. Moreover, tclArgv.[hc] are the only two files needed to add this functionality to any project (which we use quite cleanly now compiled up as a shared library). Perhaps the naviserver tclobjv could be easily simplified as well. I welcome comments on these points. Should tclArgv.c be deemed sufficiently useful for consideration for inclusion into Tcl proper, I'd be happy to recode a GetIndexFromObject version of tclArgv.c if this is a sticking point.
nobody added on 2006-06-15 03:59:42:
Logged In: NO Please take a look at this version, which does use GetIndexFromObject, parses args as well options, and uses callbacks for type checking: http://naviserver.cvs.sourceforge.net/naviserver/naviserver/nsd/tclobjv.c?view=markup There's lots of examples of it being used, e.g. here: http://naviserver.cvs.sourceforge.net/naviserver/naviserver/nsd/tclrequest.c?revision=1.14&view=markup
sbromle added on 2006-06-14 09:21:07:
File Added - 181577: tclArgv-20060613.tar.gz
sbromle added on 2006-06-14 09:21:04:
Logged In: YES user_id=1472195 The version I wrote using Tcl_GetIndexFromObj was causing a crash. I've reverted to my original version in the latest version (20060613), and it is this version that I am currently using in production. I've included an example code that compiles a Tcl extension that you can load using "load", and it provides the "parseme" example command. In addition to the coded options, you also have automatic help via "-help". I will be happy to assist in any way you see fit.
dgp added on 2006-06-13 03:34:39:
Logged In: YES user_id=80530 I want to make a real patch file for this. Which of the attached files are the right one(s) to use?
sbromle added on 2006-04-04 02:06:49:
File Added - 173278: tclArgv.c
sbromle added on 2006-04-04 02:04:03:
Logged In: YES user_id=1472195 W.r.t. the use of Tcl_GetIndexFromObj(), I did consider it, and originally decided that it was not the way to go. If I wished to implement a single command, I would surely have used it, however, the benifit of the tclArgv approach, is that the writer of the C implemented command can quite cleanly package everything up into a single array of Tcl_ArgInfo structures, and specify type checking and conversion all in one neat place. So,I originally opted for the simplicity of the current code. I have however now rewritten it to use Tcl_GetIndexFromObjStruct, to remove the parsing bit, as per your suggestion. I hope this is less bizarre. :)
dgp added on 2006-04-04 00:02:06:
Logged In: YES user_id=80530 Took a quick glance at the submitted code and I see no use of Tcl_GetIndexFromObj(). Seems very bizarre to me, as I'd expect any command line parsing to be a thin wrapper around it.
dkf added on 2006-04-03 16:29:53:
Logged In: YES user_id=79902 Requires a TIP. Email me for details on the process.
sbromle added on 2006-03-11 05:47:52:
File Added - 170511: tclArgv.tar.gz
sbromle added on 2006-03-11 05:47:48:
Logged In: YES user_id=1472195 I've added a routine "Tcl_ParseArgsObj" that provides libpopt or Tk_ParseArgv type parsing, but for Tcl_Obj *objv array passed to C implemented Tcl commands.
sbromle added on 2006-03-10 21:23:32:
File Deleted - 170362:
sbromle added on 2006-03-10 21:23:31:
File Deleted - 170364:
sbromle added on 2006-03-10 21:19:12:
File Added - 170441: Tcl_ParseArgv.tar.gz
sbromle added on 2006-03-10 03:45:05:
File Added - 170364: Tcl_ParseArgv.tar.gz
sbromle added on 2006-03-10 03:25:09:
File Added - 170362: Tcl_ParseArgv.tar.gz
sbromle added on 2006-03-10 02:47:58:
File Added - 170359: Tcl_ParseArgv.tar.gz
sbromle added on 2006-03-10 02:42:32:
File Added - 170358: Tcl_ParseArgv.tar.gz
sbromle added on 2006-03-10 02:31:44:
File Added - 170355: Tcl_ParseArgv.tar.gz
sbromle added on 2006-03-10 02:30:33:
File Deleted - 170354:
sbromle added on 2006-03-10 02:29:46:
File Added - 170354: tclArgv.h