Sunday, 6 September 2020

Flock Day 5

 So I managed to get a basic tokenizer running, it goes a bit beyond the tutorial, in that it is the startings of an LL(*) parser as opposed to the LL(1) proposed in the tutorial. For those not wanting to go into the deep specifics of parsers, the difference is how far we look ahead when reading text, an LL(1) parser will only evaluate on the next letter, whilst an LL(*) parser can evaluate as far ahead as it wants. I have actually written these before, and there is a Rules based evaluation pattern I built that I think will make parallel parsing very easy, in the new language. I am not confident enough in the way c++ manages its memory to bother until that point.

So it looks like I have decided to go with c/java style comments /* and // I have started tokenizing them already. I had a plethora of different options, # or -- or <!-- just for starters. I may go revisit that decision, but familiarity is important, and they are not the worst comments in the world.

I have also been thinking about memory management, generally the way memory is managed in modern languages is roughly equatable, and in my mind fundamentally flawed, it uses ownership semantics, which are hierarchical in nature. I have a thread, it starts a function, the function owns object A, object A owns Object B and Object B owns Object C, and so on, sometimes C can even own A, and we have some horrible problems trying to resolve what needs to be freed or allocated or passed. In reality there are only 2 owners of memory that are important, there is the global space, and the thread space. The global space is where we store our things accessible by all, they have atomicity constraints as multiple threads may have to access them at the same time. The thread space, is memory that belongs to a flow of execution, I use the word flow purposefully here, it can do what it likes with it, until it decides to move it to a different space (thread of global). Part of the problem is how do you clear data up when it is no longer needed. it is probably not good enough to wait till a thread ends its execution. However I am building a language that is moving out of the thread concepts, instead I have workers and flows, in effect I am creating a third space, the worker space. When a worker finishes, it clears its memory, rather than at the boundary of a function, we do it at the boundary of an execution. 


This is just working thoughts, what I don't want to do is just do a fuck load of memory copies.

No comments:

Post a Comment

Flock Day 8

 It's been a very long time since I last posted, although I have been spending on average 2-3 hours a day cutting code since the last po...