History: Jison Parser

[http://sourceforge.net/p/tikiwiki/code/45838/|The Jison Parser is a developer feature only.]

! Issues reported
* unparsed ~np~{TAG(tag=br,style=clear: both) /}~/np~ on https://nextparserdev.tiki.org/Development  {sign user="marclaporte" datetime="2013-02-01T17:45:39+00:00"}
* Accents are not the same. Bug or feature because the source code is as such? {sign user="marclaporte" datetime="2013-02-01T17:43:48+00:00"}
** http://nextparserdev.tiki.org/WYSIWYG_torture+test  premières
** http://dev.tiki.org/WYSIWYG_torture+test  Deux premières parties
* |nocache links in the footer {sign user="marclaporte" datetime="2013-02-01T17:43:48+00:00"}
* --http://nextdoc.tiki.org/tiki-index.php?page=Custom%20Home -> REAL_GT-- fixed
* --http://nextdoc.tiki.org/Requirements -> §e7cacb97964f755a6295f0e94914be7e§ instead of maketoc-- fixed
* Spurious "Preference wikiplugin expandingoutline disabled" warning in the admin panel when Jison Parser is activated
** {img src="img/wiki_up/2012-10-19_jison_parser_activation.png" }
* http://nextdoc.tiki.org/tiki-admin.php?page=textarea
** --Failed to execute "doc-dev-crosslinks" module Syntax Error in template "2c5dea3fb5fe0d80db2828c7c6705368591b4d59" on line 1 "{if isset($objectCategoryIds) and in_array(12, $objectCategoryIds)}<div class="clearfix rbox note"><img src="img/icons/close.png" alt="Close" width="16" height="16" class="rbox-close" onclick="$(this).parent().fadeOut();" title="Close" /><div class="rbox-title"><img src="img/icons/information.png" alt="note" width="16" height="16" title="note" class="icon" /><span>Note</span></div><div class="rbox-data">This page is to document "what Tiki does". For "what Tiki <em>should</em> do", please see <a class="wiki external" target="_blank"" href="http://dev.tiki.org/Tiki10" rel="external nofollow">corresponding page on dev site</a></div></div>{/if}" unknown tag "page"-- seems fixed
* https://nextparserdev.tiki.org/tiki-index.php?page=Newsletter 
**--§e7cacb97964f755a6295f0e94914be7e§ instead of maketoc-- fixed
** PluginWishes is not being interpreted
** Interested community members Error No field indicated
* Various plugin parsing problems documented at [https://nextparserdev.tiki.org/Plugin+Parsing+Problems]. These probems also exist with existing parser, see ((Plugin Parsing Problems))
* Jison parser eats up all PHP memory
Processing WYSIWYG testing
Fatal error: Allowed memory size of 267386880 bytes exhausted (tried to allocate 198377 bytes) in …\lib\core\JisonParser\Html.php on line 57

Call Stack:
    0,0010     126760   1. {main}() …\doc\devtools\htmlconvert.php:0
    2,3104   23660560   2. tf() …\doc\devtools\htmlconvert.php:44
    3,6918   26972696   3. JisonParser_Html_Handler->parse() ...\doc\devtools\htmlconvert.php:14
    3,6919   26982520   4. JisonParser_Html->parse() ...\lib\core\JisonParser\Html\Handler.php:97
   80,4134   73707968   5. JisonParser_Html->parser_performAction() …\lib\core\JisonParser\Html.php:253
"
The page is not large, and parsing it should not break a 255MB memory limit.

To reproduce
#Get the HTML source from http://dev.tiki.org/WYSIWYG+testing
#Create a local "WYSIWYG testing" page.
#Paste the source in the CkEditors "Source" view.
#Save the page
#Run doc/devtools/htmlconvert.php 
#>> Crashes as shown above.


!! What is it
Jison is a new approach to building simplified parsers that was incorporated into Tiki around Tiki 8.  Jison was originally a JavaScript port of the Bison project, and has since been ported to PHP, creating a unified parser for both the client and server.  As of now the Jison parser is used in several places in Tiki:
* ((Spreadsheet|Tiki Spreadsheet))
* ((Jison Wiki Parser|Experimental Wiki Parser))
* ((doc:Forward Link Protocol|ForwardLink Protocol))

There are a few things to note when it comes to the Jison parser if you want to help development with it.  Robert Plummer has lead the development of it, and because it is mostly experimental at time of writing, there is still some active development between the [https://github.com/zaach/jison|master] repository and the [https://github.com/robertleeplummerjr/jison|fork].  To develop using the Jison parser you can follow the [http://zaach.github.com/jison/docs/#installation|guidelines for installation], or if you want to go about enhancing [https://github.com/robertleeplummerjr/jison/blob/master/ports/php/template.php|the template] (which is written in PHP and is where all the regex rules found in the JavaScript counterpart are 'imported').

!! Things to note
* When developing a Jison parser you need to have node.js installed and npm.
* node at time of writing is (v0.7.0-pre)
* npm at time of writing is (1.1.0-beta-7)
* Please try to be on latest of node and npm

!! How it works
The Jison parser is a [http://en.wikipedia.org/wiki/Operator-precedence_parser|bottom up reverse operator precedence parser].  This is an efficient means of analyzing (lexing and parsing) a string. To start building a parser you create a .jison parser definition (a '.jison' file) which is used in creating the jison output.  This template is parsed and inserted into a sort of empty JavaScript template.  In JavaScript, this template exists in many files, but in the PHP template, it exists as a single file.

When developing using Jison, generally you would edit the .jison file, and each time render the output of that file.  When development needs the PHP template, you would render using the Jison wrapper designed for that port (example: [https://github.com/robertleeplummerjr/jison/blob/master/ports/php/jison.js|php]).  With the PHP template, you would render the output of both a .js and a .php file at the same time, using a command similar to this:
{CODE(colors="shell")}
#cd /path/to/jisonfile.jison
#node /path/where/jison/is/installed/ports/php/jison.js jisonfile.jison
{CODE}
This code outputs 2 file in the directory where the jisonfile.jison is, jisonfile.js and jisonfile.php.

For examples in how to write a parser and how to interact with them, please see the following:

* [http://zaach.github.com/jison/docs/|Documentation]
* [https://github.com/zaach/jison/tree/master/examples|Jison Examples]
* [http://tikiwiki.svn.sourceforge.net/viewvc/tikiwiki/trunk/lib/core/JisonParser/|JisonParsers in lib/core/JisonParser]
* [https://jquerysheet.googlecode.com/svn/trunk/parser/|Tiki Spreadsheet (jQuery.sheet)]
Please note, although a somewhat large learning curve is needed to use Jison, the results are very stable, very fast, very scalable parser.  If you have problems trying to develop using the Jison parser, please [http://webchat.freenode.net/?channels=tikiwiki|join us in irc].

!! Pre-dogfood server
We have a ((tw:Pre-Dogfood Server)) with nextparserdev.tiki.org with ((doc:System Configuration)) keeping the new Jison Parser always on, so it's very easy to compare:
# http://nextparserdev.tiki.org/Overview  Tiki trunk in testing with Jison Parser
# http://nextdev.tiki.org/Overview  Tiki trunk in testing
# http://dev.tiki.org/Overview  Tiki 10.x

-=alias=-
(alias(Jison Wiki Parser))