Storage system

First thing on the agenda is the storage system. I am currently saving everything in xml files (1 neuron = 1 xml file), grate for debugging, crap for real networks: reading/copying and moving 30.000 something files around is just to slow to work with (and that’s just a bare bones chatbot). I initially planned on using an sql db or something similar, but for practical reasons, this is not an option. Hence, it’s going to be binary, blocked flat files.

AICI demo

I would also like to extend the aici demo so that it shows how it can interact with the .net system by perhaps program it to copy files or something similar. Before I do anything to this demo though, the new storage system will have to work, cause debugging it is otherwise far to slow.  When this is done, there will probably be a new release.

Modules

next on the agenda will probably be the modules. These are partial networks, that can be imported into or exported  from an existing neural network. This way, you can share peaces of your work with others (like a set of frames or some actions, a thesaurus, …).

Designer improvements

From then on, I’m hoping to only do improvements/bug-fixes, mostly on the designer, so it can be moved to beta. These are the major areas that I will focus on first:

• There’s still a memory leak in the designer somewhere. This needs some serious attention. I thought I knew where it was, bit it isn’t there, so I have no idea at the moment what it could be.  Mmm…
• The code editor needs a custom control. I’m currently using wpf’s default listboxes for the editor (yes, it’s all listboxes). This was grate to get started, but you’ve probably already experienced some of it’s problems: it gets slow, real fast, only to blow up with a layout recursion exception (bugger). For me, it’s also a pain to get exactly how I want it (you constantly have to fight with it’s default behavior). Solution: custom control.
• The same is true for the mind maps and the flow editor (also all listboxes), which need custom controls.
• I will have to move the search functionality into it’s own thread, cause any small search at the moment freezes the UI. This can’t be off course.
• Finally there are lots of small things still doing strange things at times, but I guess that’s for the beta stage.

Thread sync system

There still is a major hiccup in the execution core: some instructions aren’t guaranteed to be uninterruptable, that is to say, some, like the instructions that change links, can be interrupted by other processors before finishing. This could result in data corruption. 

The current protection mechanism against data corruption also appears to slow down the core way to much (most time is actually spend waiting on a sync object here or there).  While trying to fix this, I pushed a bit to far, which resulted in the occasional loss of recycled id’s, in other words, deleted neurons are dropped incorrectly (you can usually see this in version 0.2 after a large input stream has been processed, the explorer might show some red slots).

The fix is a new thread syncing system on top of that provided by the OS, instead of trying to rely on the slower, clumsier locks  provided by the system, I need to create something new.  The design is mostly done, it just needs implementing, which requires some work.

AICI 1

I really like to get this demo working as soon as possible, so I will probably do a lot of work here.  There are some updates required to the system though, before this is possible:

• The wordnet importer needs some work. It currently doesn’t do a good job with respect to verbs, that is to say, their conjugations aren’t imported and linked properly.  This needs to be corrected so that it’s easier to determine the part of speech of a word in a sentence.
• Verbnet needs some serious work.  Not all the data gets imported or can be edited. I also need to write the neural network code to compare the data that was found against the known(imported) verbnet frames. And finally, I need to attach some action code to the frames, so that they actually do something.
• I also decided to rebuild the English grammar used by the AICI example, so this needs finishing.  I did this to force me to take a closer look at each and every flow item as to better understand later on where the filter code needs to be attached and where to attach the code that will build the result.

I guess by the time I have gotten this far, a lot more, currently unforeseen stuff will have been added, so for now, lets make this all for version 0.3. After that, I plan to do some more work on the visual side of things (not how the UI looks, but the image sensory interfaces and their UIs).