It's been a very long time since I last posted, although I have been spending on average 2-3 hours a day cutting code since the last post, I did not feel I had made relevant progress until now.
In regards to language creation not much has progressed, I have been sidelined in working out the best way to create a syntax tree. I have switched to an EBNF style recursive descent parser, This was a touch reinventing the wheel, others exist, for c++ the boost libraries provide one. However having done this before in Java, I felt there were some things missing from these libraries.
I invented a new pattern (in a personal sense, I can't be arsed to see if I am the first), I like to call the visitor strategy pattern, an amalgamation of the visitor and strategy pattern. The main problem with the visitor pattern, is that you tend to have to bundle all the functionality for all the behaviours into one unholy and massive class. This makes code organisation a nightmare, and means you end up implementing way more than you need. It also tends to force a certain way of navigating the object hierarchies. Often they also manage state, and so can get quite confusing to know where you are.
In my version the visitors are very light weight containers of strategies, they effectively look at the object, and find the strategy to use against it. The strategy accepts the object, rather than the object accepting the visitor, the strategy will then do its thing, if we need to traverse the object hierarchy, it merely goes back to the visitor and asks it to visit the new object. This means each piece of logic is self contained, and can be extended or reused across different types. This has already proved useful, as my rule printer, navigates all nodes in the rule tree, whilst my rules evaluator only navigates a branch until it fails. On top of that the rule printer is remarkably small.
Strategies are extendable, and via wrapping I have also implemented rule histories, again something I believe unique to my vision of how these parsers work. The theory goes, that if a stateless rule is executed at a particular point in a token stream, (in this case token=character). then if that rule is executed at the same point, it should return the same result. By effectively caching the result we gain massive time savings, at the expense of a small amount of memory, (a freedom not given to those who originally had to write parsers). Recursive descent parser's can be notoriously slow on complex grammars, as they will often ask the question multiple times, hence the time savings. They are also prone to infinite loops, and stack overflows, this technique prevents the last from ever happening, asking the same question of a rule, whilst it is still evaluating at that point counts as a cyclic failure when asked additional times, until the original request has been satisfied. At some point I plan to fix that problem which is inherent in both the visitor pattern and recursive descent parsers which is that of their tendency to stack overflow. The removal of logic from the visitor should allow me to create execution queues instead of recursive calls, as a provided implementation.
Above is an example of the parser at work, I have, unashamedly, used ANSII colours to help work out what is going on. At the top, in ISO EBNF syntax are the rules, printed out by my rule printer. At the bottom is an evaluation of textual input use (me=that,and, (this=that)); this then results in the printing of the syntax tree, each node showing the full textual output of what made up those rules.
