<div class='fossil-doc' data-title='pt::pgen - Parser Tools'>
<style>
HTML {
background: #FFFFFF;
color: black;
}
BODY {
background: #FFFFFF;
color: black;
}
DIV.doctools {
margin-left: 10%;
margin-right: 10%;
}
DIV.doctools H1,DIV.doctools H2 {
margin-left: -5%;
}
H1, H2, H3, H4 {
margin-top: 1em;
font-family: sans-serif;
font-size: large;
color: #005A9C;
background: transparent;
text-align: left;
}
H1.doctools_title {
text-align: center;
}
UL,OL {
margin-right: 0em;
margin-top: 3pt;
margin-bottom: 3pt;
}
UL LI {
list-style: disc;
}
OL LI {
list-style: decimal;
}
DT {
padding-top: 1ex;
}
UL.doctools_toc,UL.doctools_toc UL, UL.doctools_toc UL UL {
font: normal 12pt/14pt sans-serif;
list-style: none;
}
LI.doctools_section, LI.doctools_subsection {
list-style: none;
margin-left: 0em;
text-indent: 0em;
padding: 0em;
}
PRE {
display: block;
font-family: monospace;
white-space: pre;
margin: 0%;
padding-top: 0.5ex;
padding-bottom: 0.5ex;
padding-left: 1ex;
padding-right: 1ex;
width: 100%;
}
PRE.doctools_example {
color: black;
background: #f5dcb3;
border: 1px solid black;
}
UL.doctools_requirements LI, UL.doctools_syntax LI {
list-style: none;
margin-left: 0em;
text-indent: 0em;
padding: 0em;
}
DIV.doctools_synopsis {
color: black;
background: #80ffff;
border: 1px solid black;
font-family: serif;
margin-top: 1em;
margin-bottom: 1em;
}
UL.doctools_syntax {
margin-top: 1em;
border-top: 1px solid black;
}
UL.doctools_requirements {
margin-bottom: 1em;
border-bottom: 1px solid black;
}
</style>
<hr> [
<a href="../../../../toc.html">Main Table Of Contents</a>
| <a href="../../../toc.html">Table Of Contents</a>
| <a href="../../../../index.html">Keyword Index</a>
| <a href="../../../../toc0.html">Categories</a>
| <a href="../../../../toc1.html">Modules</a>
| <a href="../../../../toc2.html">Applications</a>
] <hr>
<div class="doctools">
<h1 class="doctools_title">pt::pgen(n) 1.0.2 tcllib "Parser Tools"</h1>
<div id="name" class="doctools_section"><h2><a name="name">Name</a></h2>
<p>pt::pgen - Parser Generator</p>
</div>
<div id="toc" class="doctools_section"><h2><a name="toc">Table Of Contents</a></h2>
<ul class="doctools_toc">
<li class="doctools_section"><a href="#toc">Table Of Contents</a></li>
<li class="doctools_section"><a href="#synopsis">Synopsis</a></li>
<li class="doctools_section"><a href="#section1">Description</a></li>
<li class="doctools_section"><a href="#section2">API</a></li>
<li class="doctools_section"><a href="#section3">Example</a></li>
<li class="doctools_section"><a href="#section4">Bugs, Ideas, Feedback</a></li>
<li class="doctools_section"><a href="#keywords">Keywords</a></li>
<li class="doctools_section"><a href="#category">Category</a></li>
<li class="doctools_section"><a href="#copyright">Copyright</a></li>
</ul>
</div>
<div id="synopsis" class="doctools_section"><h2><a name="synopsis">Synopsis</a></h2>
<div class="doctools_synopsis">
<ul class="doctools_requirements">
<li>package require <b class="pkgname">Tcl 8.5</b></li>
<li>package require <b class="pkgname">pt::pgen <span class="opt">?1.0.2?</span></b></li>
</ul>
<ul class="doctools_syntax">
<li><a href="#1"><b class="cmd">::pt::pgen</b> <i class="arg">inputformat</i> <i class="arg">text</i> <i class="arg">resultformat</i> <span class="opt">?<i class="arg">options...</i>?</span></a></li>
</ul>
</div>
</div>
<div id="section1" class="doctools_section"><h2><a name="section1">Description</a></h2>
<p>Are you lost ?
Do you have trouble understanding this document ?
In that case please read the overview provided by the
<i class="term"><a href="pt_introduction.html">Introduction to Parser Tools</a></i>. This document is the
entrypoint to the whole system the current package is a part of.</p>
<p>This package provides a command implementing a
<i class="term"><a href="../../../../index.html#key189">parser generator</a></i>
taking parsing expression grammars as input.</p>
<p>It is the implementation of method <b class="method">generate</b> of <b class="cmd"><a href="../../apps/pt.html">pt</a></b>, the
<i class="term"><a href="../../apps/pt.html">Parser Tools Application</a></i>.</p>
<p>As such the intended audience of this document are people wishing to
modify and/or extend this part of <b class="cmd"><a href="../../apps/pt.html">pt</a></b>'s functionality. Users of
<b class="cmd"><a href="../../apps/pt.html">pt</a></b> on the other hand are hereby refered to the applications'
manpage, i.e. <i class="term"><a href="../../apps/pt.html">Parser Tools Application</a></i>.</p>
<p>It resides in the User Package Layer of Parser Tools.</p>
<p><img alt="arch_user_pkg" src="../../../../image/arch_user_pkg.png"></p>
</div>
<div id="section2" class="doctools_section"><h2><a name="section2">API</a></h2>
<dl class="doctools_definitions">
<dt><a name="1"><b class="cmd">::pt::pgen</b> <i class="arg">inputformat</i> <i class="arg">text</i> <i class="arg">resultformat</i> <span class="opt">?<i class="arg">options...</i>?</span></a></dt>
<dd><p>This command takes the parsing expression grammar in <i class="arg">text</i> (in
the format specified by <i class="arg">inputformat</i>), and returns the same
grammar in the format <i class="arg">resultformat</i> as the result of the command.</p>
<p>The two known input formats are <b class="const">peg</b> and <b class="const">json</b>.
Introductions to them, including their formal specifications, can be
found in the <i class="term"><a href="pt_peg_language.html">PEG Language Tutorial</a></i> and
<i class="term"><a href="pt_json_language.html">The JSON Grammar Exchange Format</a></i>. The packages used to
parse these formats are</p>
<dl class="doctools_definitions">
<dt><b class="const">peg</b></dt>
<dd><p><b class="package"><a href="pt_peg_from_peg.html">pt::peg::from::peg</a></b></p></dd>
<dt><b class="const">json</b></dt>
<dd><p><b class="package"><a href="pt_peg_from_json.html">pt::peg::from::json</a></b></p></dd>
</dl>
<p>On the output side the known formats, and the packages used to
generate them are</p>
<dl class="doctools_definitions">
<dt><b class="const">c</b></dt>
<dd><p><b class="package"><a href="pt_peg_to_cparam.html">pt::peg::to::cparam</a></b></p></dd>
<dt><b class="const">container</b></dt>
<dd><p><b class="package"><a href="pt_peg_to_container.html">pt::peg::to::container</a></b></p></dd>
<dt><b class="const">critcl</b></dt>
<dd><p><b class="package"><a href="pt_peg_to_cparam.html">pt::peg::to::cparam</a></b> +
<b class="package"><a href="pt_cparam_config_critcl.html">pt::cparam::configuration::critcl</a></b></p></dd>
<dt><b class="const">json</b></dt>
<dd><p><b class="package"><a href="pt_peg_to_json.html">pt::peg::to::json</a></b></p></dd>
<dt><b class="const">oo</b></dt>
<dd><p><b class="package"><a href="pt_peg_to_tclparam.html">pt::peg::to::tclparam</a></b> +
<b class="package"><a href="pt_tclparam_config_tcloo.html">pt::tclparam::configuration::tcloo</a></b></p></dd>
<dt><b class="const">peg</b></dt>
<dd><p><b class="package"><a href="pt_peg_to_peg.html">pt::peg::to::peg</a></b></p></dd>
<dt><b class="const">snit</b></dt>
<dd><p><b class="package"><a href="pt_peg_to_tclparam.html">pt::peg::to::tclparam</a></b> +
<b class="package"><a href="pt_tclparam_config_snit.html">pt::tclparam::configuration::snit</a></b></p></dd>
</dl>
<p>The options supported by each of these formats are documented
with their respective packages.</p></dd>
</dl>
</div>
<div id="section3" class="doctools_section"><h2><a name="section3">Example</a></h2>
<p>In this section we are working a complete example, starting with a PEG
grammar and ending with running the parser generated from it over some
input, following the outline shown in the figure below:</p>
<p><img alt="flow" src="../../../../image/flow.png"></p>
<p>Our grammar, assumed to the stored in the file "<b class="file">calculator.peg</b>"
is</p>
<pre class="doctools_example">
PEG calculator (Expression)
Digit <- '0'/'1'/'2'/'3'/'4'/'5'/'6'/'7'/'8'/'9' ;
Sign <- '-' / '+' ;
Number <- Sign? Digit+ ;
Expression <- Term (AddOp Term)* ;
MulOp <- '*' / '/' ;
Term <- Factor (MulOp Factor)* ;
AddOp <- '+'/'-' ;
Factor <- '(' Expression ')' / Number ;
END;
</pre>
<p>From this we create a snit-based parser
using the script "<b class="file">gen</b>"</p>
<pre class="doctools_example">
package require Tcl 8.5
package require fileutil
package require pt::pgen
lassign $argv name
set grammar [fileutil::cat $name.peg]
set pclass [pt::pgen peg $gr snit -class $name -file $name.peg -name $name]
fileutil::writeFile $name.tcl $pclass
exit 0
</pre>
<p>calling it like</p>
<pre class="doctools_example"> tclsh8.5 gen calculator </pre>
<p>which leaves us with the parser package and class written to the file
"<b class="file">calculator.tcl</b>".
Assuming that this package is then properly installed in a place where
Tcl can find it we can now use this class via a script like</p>
<pre class="doctools_example">
package require calculator
lassign $argv input
set channel [open $input r]
set parser [calculator]
set ast [$parser parse $channel]
$parser destroy
close $channel
... now process the returned abstract syntax tree ...
</pre>
<p>where the abstract syntax tree stored in the variable will look like</p>
<pre class="doctools_example">
set ast {Expression 0 4
{Factor 0 4
{Term 0 2
{Number 0 2
{Digit 0 0}
{Digit 1 1}
{Digit 2 2}
}
}
{AddOp 3 3}
{Term 4 4
{Number 4 4
{Digit 4 4}
}
}
}
}
</pre>
<p>assuming that the input file and channel contained the text</p>
<pre class="doctools_example"> 120+5 </pre>
<p>A more graphical representation of the tree would be</p>
<p><img alt="expr_ast" src="../../../../image/expr_ast.png"></p>
<p>Regardless, at this point it is the user's responsibility to work with
the tree to reach whatever goal she desires. I.e. analyze it,
transform it, etc. The package <b class="package"><a href="pt_astree.html">pt::ast</a></b> should be of help
here, providing commands to walk such ASTs structures in various ways.</p>
<p>One important thing to note is that the parsers used here return a
data structure representing the structure of the input per the grammar
underlying the parser. There are <em>no</em> callbacks during the
parsing process, i.e. no <i class="term">parsing actions</i>, as most other
parsers will have.</p>
<p>Going back to the last snippet of code, the execution of the parser
for some input, note how the parser instance follows the specified
<i class="term"><a href="pt_parser_api.html">Parser API</a></i>.</p>
</div>
<div id="section4" class="doctools_section"><h2><a name="section4">Bugs, Ideas, Feedback</a></h2>
<p>This document, and the package it describes, will undoubtedly contain
bugs and other problems.
Please report such in the category <em>pt</em> of the
<a href="http://core.tcl.tk/tcllib/reportlist">Tcllib Trackers</a>.
Please also report any ideas for enhancements you may have for either
package and/or documentation.</p>
<p>When proposing code changes, please provide <em>unified diffs</em>,
i.e the output of <b class="const">diff -u</b>.</p>
<p>Note further that <em>attachments</em> are strongly preferred over
inlined patches. Attachments can be made by going to the <b class="const">Edit</b>
form of the ticket immediately after its creation, and then using the
left-most button in the secondary navigation bar.</p>
</div>
<div id="keywords" class="doctools_section"><h2><a name="keywords">Keywords</a></h2>
<p><a href="../../../../index.html#key3">EBNF</a>, <a href="../../../../index.html#key15">LL(k)</a>, <a href="../../../../index.html#key6">PEG</a>, <a href="../../../../index.html#key7">TDPL</a>, <a href="../../../../index.html#key4">context-free languages</a>, <a href="../../../../index.html#key0">expression</a>, <a href="../../../../index.html#key11">grammar</a>, <a href="../../../../index.html#key5">matching</a>, <a href="../../../../index.html#key9">parser</a>, <a href="../../../../index.html#key8">parsing expression</a>, <a href="../../../../index.html#key14">parsing expression grammar</a>, <a href="../../../../index.html#key1">push down automaton</a>, <a href="../../../../index.html#key10">recursive descent</a>, <a href="../../../../index.html#key2">state</a>, <a href="../../../../index.html#key13">top-down parsing languages</a>, <a href="../../../../index.html#key12">transducer</a></p>
</div>
<div id="category" class="doctools_section"><h2><a name="category">Category</a></h2>
<p>Parsing and Grammars</p>
</div>
<div id="copyright" class="doctools_section"><h2><a name="copyright">Copyright</a></h2>
<p>Copyright © 2009 Andreas Kupries <[email protected]></p>
</div>
</div>
