Tcl Library Source Code

Bounty program for improvements to Tcl and certain Tcl packages.
Tcl 2019 Conference, Houston/TX, US, Nov 4-8
Send your abstracts to [email protected]
or submit via the online form by Sep 9.

[ Main Table Of Contents | Table Of Contents | Keyword Index | Categories | Modules | Applications ]


grammar::me::cpu::core - ME virtual machine state manipulation

Table Of Contents


package require Tcl 8.4
package require grammar::me::cpu::core ?0.2?

::grammar::me::cpu::core disasm asm
::grammar::me::cpu::core asm asm
::grammar::me::cpu::core new asm
::grammar::me::cpu::core lc state location
::grammar::me::cpu::core tok state ?from ?to??
::grammar::me::cpu::core pc state
::grammar::me::cpu::core iseof state
::grammar::me::cpu::core at state
::grammar::me::cpu::core cc state
::grammar::me::cpu::core sv state
::grammar::me::cpu::core ok state
::grammar::me::cpu::core error state
::grammar::me::cpu::core lstk state
::grammar::me::cpu::core astk state
::grammar::me::cpu::core mstk state
::grammar::me::cpu::core estk state
::grammar::me::cpu::core rstk state
::grammar::me::cpu::core nc state
::grammar::me::cpu::core ast state
::grammar::me::cpu::core halted state
::grammar::me::cpu::core code state
::grammar::me::cpu::core eof statevar
::grammar::me::cpu::core put statevar tok lex line col
::grammar::me::cpu::core run statevar ?n?


This package provides an implementation of the ME virtual machine. Please go and read the document grammar::me_intro first if you do not know what a ME virtual machine is.

This implementation represents each ME virtual machine as a Tcl value and provides commands to manipulate and query such values to show the effects of executing instructions, adding tokens, retrieving state, etc.

The values fully follow the paradigm of Tcl that every value is a string and while also allowing C implementations for a proper Tcl_ObjType to keep all the important data in native data structures. Because of the latter it is recommended to access the state values only through the commands of this package to ensure that internal representation is not shimmered away.

The actual structure used by all state values is described in section CPU STATE.


The package directly provides only a single command, and all the functionality is made available through its methods.


A match program is represented by nested Tcl list. The first element, asm, is a list of integer numbers, the instructions to execute, and their arguments. The second element, pool, is a list of strings, referenced by the instructions, for error messages, token names, etc. The third element, tokmap, provides ordering information for the tokens, mapping their names to their numerical rank. This element can be empty, forcing lexicographic comparison when matching ranges.

All ME instructions are encoded as integer numbers, with the mapping given below. A number of the instructions, those which handle error messages, have been given an additional argument to supply that message explicitly instead of having it constructed from token names, etc. This allows the machine state to store only the message ids instead of the full strings.

Jump destination arguments are absolute indices into the asm element, refering to the instruction to jump to. Any string arguments are absolute indices into the pool element. Tokens, characters, messages, and token (actually character) classes to match are coded as references into the pool as well.

  1. "ict_advance message"

  2. "ict_match_token tok message"

  3. "ict_match_tokrange tokbegin tokend message"

  4. "ict_match_tokclass code message"

  5. "inc_restore branchlabel nt"

  6. "inc_save nt"

  7. "icf_ntcall branchlabel"

  8. "icf_ntreturn"

  9. "iok_ok"

  10. "iok_fail"

  11. "iok_negate"

  12. "icf_jalways branchlabel"

  13. "icf_jok branchlabel"

  14. "icf_jfail branchlabel"

  15. "icf_halt"

  16. "icl_push"

  17. "icl_rewind"

  18. "icl_pop"

  19. "ier_push"

  20. "ier_clear"

  21. "ier_nonterminal message"

  22. "ier_merge"

  23. "isv_clear"

  24. "isv_terminal"

  25. "isv_nonterminal_leaf nt"

  26. "isv_nonterminal_range nt"

  27. "isv_nonterminal_reduce nt"

  28. "ias_push"

  29. "ias_mark"

  30. "ias_mrewind"

  31. "ias_mpop"


A state value is a list containing the following elements, in the order listed below:

  1. code: Match instructions, see MATCH PROGRAM REPRESENTATION.

  2. pc: Program counter, int.

  3. halt: Halt flag, boolean.

  4. eof: Eof flag, boolean

  5. tc: Terminal cache, and input queue. Structure see below.

  6. cl: Current location, int.

  7. ct: Current token, string.

  8. ok: Match status, boolean.

  9. sv: Semantic value, list.

  10. er: Error status, list.

  11. ls: Location stack, list.

  12. as: AST stack, list.

  13. ms: AST marker stack, list.

  14. es: Error stack, list.

  15. rs: Return stack, list.

  16. nc: Nonterminal cache, dictionary.

tc, the input queue of tokens waiting for processing and the terminal cache containing the tokens already processing are one unified data structure simply holding all tokens and their information, with the current location separating that which has been processed from that which is waiting. Each element of the queue/cache is a list containing the token, its lexeme information, line number, and column index, in this order.

All stacks have their top element aat the end, i.e. pushing an item is equivalent to appending to the list representing the stack, and popping it removes the last element.

er, the error status is either empty or a list of two elements, a location in the input, and a list of messages, encoded as references into the pool element of the code.

nc, the nonterminal cache is keyed by nonterminal name and location, each value a four-element list containing current location, match status, semantic value, and error status, in this order.

Bugs, Ideas, Feedback

This document, and the package it describes, will undoubtedly contain bugs and other problems. Please report such in the category grammar_me of the Tcllib Trackers. Please also report any ideas for enhancements you may have for either package and/or documentation.

When proposing code changes, please provide unified diffs, i.e the output of diff -u.

Note further that attachments are strongly preferred over inlined patches. Attachments can be made by going to the Edit form of the ticket immediately after its creation, and then using the left-most button in the secondary navigation bar.


grammar, parsing, virtual machine


Grammars and finite automata


Copyright © 2005-2006 Andreas Kupries