A split path represent the route that a result took, from start to now, expressed in the neurons that the path chose during all the splits.

As example, if only 1 split was executed to get to a result, and there were 2 neurons in that split, the path of the result only has 1 node and can have 2 possible values, those that were used in the split. In the real world though, a path usually has many, many nodes.

To record a split path, you first need to put a break point at a position where you know you will get many invalid results. That’s because a split path can only be recorded when the processors are still running, not when the result has already been calculated.

In the aici demo, it’s best to put a breakpoint in the ‘Stage 1.1’ code, in the if, second part (GetClusterMeaning(CurrentTo) == flow), if you drill down a bit further, you get the ‘Add split result’ instruction, which is basically the end point of the flow recognition  algorithm. (see picture below).

Put breakpoints on ‘Add split result’ instructions to find invalid split paths.

Once you have a processor that is producing an invalid result at your breakpoint, you can store the split path. Select the processor, open the context menu (right mouse button on a processor in the debugger overview) and select ‘Store split path’.  This will produce a new entry in the tree on the ‘Debugger’ page (see 3th image). The root item will be ‘path for x’ where x is the name of the processor. The children are all the neurons that were selected for the path during a split instruction.

I usually kill everything after I have the path, the other processors will produce invalid results anyway, and you need to start a new run before it’s  possible to actually use the path anyway.

So, finally we can start using the data to debug our network. The basic idea is to  use the path for finding the processors that are following the specified path, so we can take a closer look at the code. In short, a split path provide a shortcut: instead of manually stepping through all the code and individually deciding which processors to follow, we can jump to the ones we are interested in. These will be highlighted in green and will pause when they have reached a split that is selected in the path. Off course, we first need to select the path in the debugger, and at least 1 child node, otherwise there wont be anything to see.

 Processors that follow a selected split path will show up in green and pause whenever they reach a split that produces a neuron which is selected in the path.

 One last note perhaps, once you have the processor you want, you can use the ‘Kill all but this’ command to stop all the other processors. This way there is less distraction in the debugger.

In case you are looking for a mental image to visualize this whole concept, try this: Resonance. When a link gets activated (for instance, the one to the very first neuron), it creates a resonance that triggers one or more other neurons. This excitation in turn can cause another link between 2 neurons to be activated, causing more resonance and so and and on until the whole thing settles down.

I have absolutely no prove or idea that this is how it works in the real world, that’s just how my model can be interpreted. Truth be told, this is not how I conceived the thing (aka lets try to create a model that uses resonance), it was rather more like: I have 2 neurons, they are linked, how can I make something happen? Well, I can attach some code to the link and execute that. Cool. But, wait a moment, that looks like resonance…

and we just passed ‘culminate’, pfff

• There’s a complete new lockmanager running in the background. This is much more secure (thread-wise, that is) and a lot faster. It’s still a bit of a diesel though, it takes some time for it to get going, but once running, it should be pretty fast. The slow start is due to the storage mechanism (all xml files currently). This is the major drag on the entire system at the moment, and will be fixed next.
• Wordnet import has been seriously updated, a lot more info is retrieved, and it’s now also possible to import the entire db in one go (although not yet advisable, due to a memory bug in the designer, it still takes a major byte out of the hard disk and it simply takes ridiculously long).
• The thesaurus has been given a make over to allow for editing and filtering. He can now also display / edit non recursive relationships. Drag drop is also supported.
• The frame editor has been updated considerably: drag drop support has been added and frame element filters/restrictions have also been introduced (will probably be extended in the future).
• I changed the function of the ‘contains’ operator a bit. It now only checks the contents of a variable. For clusters/children, there are the new instructions (IsClustesteredBy, LinkExists, ContainsChildren, GetInFiltered,…).
• I have added the ‘not contains’ operator.
• New instructions:
• Arithmetic group (+-/*%): I finally caved on those. My original plan was to see how far I got without using any arithmetic in the neural code. I guess, this is as far as I got with that.
• Get-at group: get child at, get cluster at, get out at, get in at, get info at.
• Distinct
• get Incoming, get outgoing, get info, Get in filtered, get out filtered, Get info filtered
• Is clustered by, Link exists, Contains children
• Remove-at group: Remove child at, remove info at, remove link in at, remove link out at
• many, many bug fixes, updates and little improvements.

Aici 1

This demo is a small chat interface. Though, in it’s current state, it’s not much more than a framework. It can initiate a conversation, close it, ask for the name of the user and store the data. It’s not yet able to fully recognize a recurring user since I haven’t defined the neural code for this yet.

I will be explaining how it works and how you can expand on this functionality, shortly. For those who can’t wait and want to get a peek under the hood, here are some pointers to get started:

• There are 3 main stages:
• Flow recognition, which is basically the syntactical stage: check the word types and order. Project pages are:
• the flows (aici/flows),
• the code that is attached to these flows (aici/code/flow code)
• the code that recognizes the flows in the input (aici/code/flow recognition)
• Frame recognition, or the semantics stage. This is where we try to find meaning in the words. Project pages are:
• the frames (aici/frames)
• the code that is attached to the frame sequences (aici/code/frame seq code). The frames don’t have code (yet).
• the code that recognizes the frames in the flow results (aici/code/frame recognition).
• action execution, or the response of the network to the input. Project pages are:
• Actions: all the different actions that the system knows (not yet a lot, should be extended).
• Output: some common code blocks for rendering output. This is also used by the frame sequences, since they are also used to render data in a predefined format.
• Action helpers: code that the action neurons can use to perform common tasks, like stopping a conversation or controlling the timers. Timer callbacks are also stored here.
• The transition between the different stages can be located in aici/code/transition.
• More specifically, the ‘Transition’ neuron is used to go from the flows to the frames and finally to the actions. 
• An action is started using the ‘Execute action’ neuron as meaning for a link from a data cluster to the action that needs to be started.
• The project also contains some mindmaps that describe the inner data structures and functionality.

Time for the second demo overview: the Scanner.  It builds on most of the ideas found in the first demo but it goes way further, and actually does something very useful (although you wouldn’t say it at first).  It’s probably going to be a lengthy piece so I’m thinking of cutting it in 2 or maybe even 3 parts. Anyway, lets first start it up, either through the start menu shortcut (in the Demo’s sub folder, conveniently called Scanner demo), or by opening it in NND (File/Open, select the ‘My documents/NND/Demos/Scanner’ folder). Once the project is loaded, you should see a single text communication channel open (called Text sin), if this is not the case, go to View/Communication channels/Text sin and make certain that it is is selected.

Overview

Lets get a taste of what it does, so enter some text (or leave the one that’s already there) and press the ‘send’ button (or enter).

You’ll notice that it basically does the same thing as the echo demo: the input text is echoed back, except that it’s a bit slower. If you had the debugger tab open, you probably also noticed a lot more activity, so something more must be going on.

And indeed, if you take a closer look to the text sin channel, in the upper section, you can see the neurons that were send to the network (on the left) and those that were returned (on the right), which are different. This was not the case with the echo demo, it simply sent all the incoming neurons back out, as they were. In this demo though, we get back something completely different: TextNeurons, that represent the same thing as the int neurons that were sent as input (if you regard them as ASCII characters). Hence the name of the demo, it’s a scanner.

Converting a stream of ASCII chars into words, integers, doubles and signs is a pretty useful feature and that’s all this demo is capable of doing, but that’s only because I stopped there. You see, in the background, is a general purpose algorithm that converts an input stream of neurons into a single result cluster, using any and all of the flows that are defined in the network. This means that you can use the same algorithm with many different flow definitions. I have simply defined some to recognize words, integers and doubles. You could go further and add flows to find verbs, sentence subjects,.. (in fact, that’s what the AICI 1 demo does). You could even go further still and create flows for visual objects or audio fragments, the same algorithm can be used. Unfortunately though, the editor doesn’t yet support such types of displays for flows (will probably be added somewhere in the future though).

Details

So how is the translation actually performed? To explain this, let me first recap some of the basic concepts of neurons and flows:

• The different types of  data available to a neuron are: incoming and outgoing links, possibly one or more parent clusters, for clusters possibly 1 or more children and a meaning. And finally value neurons also have their value off course. This is important, cause when the translation process starts, this is all the available information.
• Flows are nothing more than clusters that contain flow items, which can be statics or conditionals (loops and options). These in turn can only contain conditional parts. They represent a single branch of the decision tree. Parts can again have the same data as flows: statics or conditionals. So if you are a neuron (a static, part, conditional or flow), you can always look up into your list of parents to see in which flows and parts it is used.

Now, if you recall from the first demo, an input starts by creating an IntNeuron for each ASCII value, which is linked to another neuron using the ‘Letter’ neuron (ID 109). These are all put on the execution stack and the processor starts (the Rules code on the Letter neuron is executed).

So both neurons are new and only have each other as links. This means that we can’t use links or parent/child relationships to resolve the first step, but instead must use something different. The only other thing that remains is the value of the int neurons, so this is compared against some constants to see if they are digits (0..9) alpha numeric (a..z+A..Z), spaces | returns, or something else.  This comparison results in the creation of 1 new neuron per input neuron: the result cluster, in which we store the integer. One of these clusters (or it’s duplicate, due to a split) will eventually store the end result.  This cluster is linked to one of 3 static neurons: Digit, Alpha or Space (signs like . or , are handled a bit differently, this will be explained later). Naturally, if the integers would represent color values, we would use other starting points than digit or alpha. In other words, this first part is variable according to the type of input and the required accuracy of the algorithm. As meaning, we use  the start of the ‘flow recognition’ algorithm, called ‘Stage 1.1’ and put the result cluster back on the stack.  It’s important to put this one back on the stack, and not the item we are looking for. That’s because the algorithm can perform numerous splits and we want the result to be duplicated not the searchable, cause the contents of the list are continuously modified and we don’t want the result of one processor to be modified by  another one (I had to learn this the hard way).

After this initial step, the actual recognition algorithm kicks in. This consists out of 4 stages, grouped by 2. Meaning that  stage 1.2 is executed immediately after stage 1.1 for each neuron (this is the same for stage 2,1 and 2.2), but at the end of stage 1.2 and 2.2 all the result neurons are collected into a single cluster. Only after the last link of the last item on the stack has  been processed, are all the result clusters put back on the stack, with links for the next stage.  This is done for allowing to group items together.

The different stages are:

• Stage 1.1 (search parts/flows): Find conditional parts or flows in the list of parents of the searchable. If there are multiple results, perform a split for each, after the result list has been filtered (these are the shortcuts). If there are no results and this is the only and last item still on the stack, the end result has been found.
• Stage 1.2 (sequence-combine and filter): Check if items are sequential (2 flow items declared after each other in the same parent list, which is a part or flow) and handle floating flows, which are allowed to appear anywhere in the input stream, but which break up the sequence of other items.  Different actions can be performed if the order of the items is not ok: try to solve further or exit without result. Results of sequential items are grouped together.
• Stage 2.1 (search conditionals): Find conditionals in the list of parents of the searchable if this is a conditional part, otherwise the stage is simply skipped. If there are multiple results, perform a split for each after the result list has been filtered (not yet completely implemented at this stage). If there are no results, there is an error in the flow definition.
• Stage 2.2 (process loops and sync-points): If there was a conditional found in the previous stage, check if this is a loop. If so, and the previous item is of the same loop, combine the results. Also start a sync-point (will be explained later) if this was defined on the conditional.

These 4 stages are repeated until there is only 1 result cluster on the stack that represents the end result of a flow which is no longer used in any other flows. This cluster is made the result of the  split for the processor it ran on.  Off course, because there were possibly many splits, there could be many results. These will all be presented in the split-callback cluster (you need to provide a code cluster to the Split instruction, which will be called when all sub processors are done). In this demo, the result is sent back to the sin that caused the input, in a normal situation thought, this will simply start another process, as is done in the AICI 1 demo.

During this whole process, the algorithm is capable of executing callback code (attached to the statics, conditionals, parts and flows) at certain specific moments in the code. This is where the magic happens. The following types of callbacks are possible (together with  their execution time):

• Flow code: this code cluster is executed when a flow has been recognized in the stream. The ‘Result’ variable (ID 1822) contains all the neurons that match the flow. It’s here for instance that the int neurons are converted to a single word using the CiToS (Cluster with ints to string) instruction.
• Filter flow code: this code cluster is called from stage 1.1  (or 2.1, but this is not yet completely implemented) just after all the next items were retrieved from the list of parents of the searchable (stored in the CurrentTo variable). It allows the flow item to determine if it is a valid result, given the current state of the network. This is done by checking the contents of a number of globals, like ‘Prev stage item’ (ID 1956), which contains the previously processed neuron.

In the next post, I’ll go deeper into the specifics of the algorithm itself, for as you’ve guessed by know, it’s a bit funky, and I don’t want to forget all the subtleties, since it’s definitely still a work in progress (there are many improvements still possible).

I also realized that the flow recognition algorithm simply was not yet mature enough to be used in even the most basic situations. Not just because it was not yet able to process the more complex flow structures, but also because I made a basic design error that I did not yet know off: always put the result cluster on the stack (‘from’ part of link) and the item-to-search in the ‘to’ part. Getting this algorithm ready definitely took the most effort, but it was a grate catalyst to improve the debugger and the execution core. Here’s a (non exhaustive) list of new/changed things:

• New instructions: GetClustersFiltered, GetChildrenFiltered, Freeze
• Added ‘Stop all processors’  and ‘kill single processor’  commands
• Added import of VerbNet data
• Updated Frame editor so that it is able to edit VerbNet data, this is a work in progress, not all data items found in verbnet can already be edited.
• major update to the scanner demo
• Added the start of the AICI 1 demo, which will/should soon become able to parse the English grammar (it’s already able to parse the verb ‘to be’ ex: you are, I am, is he, he is, they were,.. (silly I know, but a start is a start!)
• Updated flow editor with overlay items, which display the presence of certain (useful) links, like code that is attached to the flow item.
• Added a new dialog: ‘Overlay editor’ in the Tools menu for editing the overlays that are defined in a project. This means that overlays can eventually be used everywhere.
• new debug feature: split paths, which allow you to track previously recorded paths of processors.
• new debug feature: attached neurons, which allow you to track down data changes in threads that should not be allowed to change the data.
• Extra info about the static neurons (like the display title and the description) is now stored globally, outside of the project so that I only have to update 1 thing, and not all the projects whenever some documentation needs to be updated. This does mean that you can no longer store any description info for these statically defined neurons.
• Updated the WordNet sensory interface so that it generates POSGroup objects, which ‘group’ all objects together, that have the same text and part of speech. This saves a grate number of unneeded splits while processing the English grammar.
• Made project saving/opening multi threaded so that the UI doesn’t appear frozen.
• Many, many small bug fixes all over the place.

Perhaps a final note, best to uninstall any previous versions.  I haven’t yet tested how the installer works when it overwrites a previous installation, so it might screw things up.  And off course, you can get the new installer from here.

The engine appears to be stabling out, although it is still acting fishy on single core machines, where there are errors I don’t have on my multi core dev system, so I need to move to a different machine to test this out. The designer is also still very much lingering behind the engine when this is processing at full speed, but the UI should remain responsive now.

I have also included an extra demo project called ‘Scanner’. It is able to transform an input stream containing integers, representing characters, into words and numbers (ints and doubles).  This doesn’t seem much, and it isn’t, except that it is doing this using a couple of flows and a general purpose algorithm (processing is still slow, mostly because the UI is trying to catch up).  This was the guts of the older ‘English language definition’ demo, which I have split into 2: the scanner and the language definition, which is no longer able to do any processing (all code removed). It’s an example of a more complex flow.

The scanner demo also has the number scanning problem fixed (numbers longer than 2 came out with multiple results). This appeared to be caused by deleting a couple of neurons to many (in the scanning algorithm).  I had already experienced the dangers of deleting neurons I thought were no longer used, but which were because of the splits. I will probably have to implement some sort of a garbage collection system to clean up unused neurons (but that’s for later).

Anyway, here’s the latest download (best to remove previous installation before installing this one).

Set up

Before we send some input to the network, we need to set up the designer so that it is ready to debug:

1. We need to  put a breakpoint  on a statement. You can do this in a code editor. The very first code block that gets called in this network is the rules code on the ‘Contains Word’ neuron, so simply double click on the ‘Code: Contains Word’ node in the Projects tab.
   To set the breakpoint on the first statement, expand the ‘CheckStartTextBlock – space insertion’ code block and click in the little circle on the ‘if’ statement. This should make it turn red.
2. The designer also needs to be set in design mode.  You can do this by the drop down box on the main toolbar or on the debugger tab’s toolbar. There are 3 possible modes:
   • Off: no debugging is possible.  This mode runs faster.
   • Normal: The debugger stops whenever a breakpoint is encountered. When a breakpoint is encountered or a processor was paused, it is possible to inspect values.  This is how most other debuggers provide debug capabilities.
   • Slow motion: In this mode, the debugger automatically pauses and continues on every statement, creating a movie of the execution process. You can use the throttle bar on the left of the combo box to select the speed, which is updated real time. It’s possible to pause and continue this movie using the pause and play buttons on the debugger tab’s menu.  I have included this type of debugging mode because I believe it is sometimes useful to see the path that is taken to a specific point of interest. With this feature, you don’t have to keep pressing F6 to advance.
3. It’s also possible, but not required, to let a processor stop when an error was encountered. This is done by activating the ‘Break on error’ button. This only works when the error happened on a processor in debug mode.

Note that it is not possible to switch between different modes while a processor is running. You can only specify the debug mode for newly created processors.

An overview of all the breakpoints in the project can be found on the right side of the debugger tab. Currently it simply lists all the breakpoints, but new features should be added to this list shortly.

Start a processor

To get started, we need to send  some data to the network through it’s text-sin (sensory interface), so make certain that the ‘Echo channel’ is opened (a communication  channel is the visual interface for a sin). Go to View/Communication Channels/Echo channel, and make certain it is checked (and that the tab is selected).

Once the channel is open, type some text in the input section and send it to the network by pressing enter (or the send button). You should see a neuron appear in the left upper screen, which represents the input event (the neuron that will be solved by the processor). Your text will also appear as a string in the centre dialog screen. There will also appear an object in the centre screen of the debugger tab. This represents the processor that was started and which is handling the input event.

The processor overview contains 2 numbers. The first number represents the name of the processor. This value can be changed and is used to identify it between multiple processors. The second number represents the number of neurons that are left on the stack + the current neuron that  is being solved.

The buttons represent, from left to right:

• a toggle button that can be used to open and close the detailed view for the processor. 
• An indicator that is selected when the processor is paused. This is used to inform you on the run state, it’s not an interactive button.
• An indicator that lets you know that the processor is still running or not. When this is no longer selected, an irrecoverable error occurred in the processor and it is actually dead (in other words, you’ve got a problem when this is no longer selected).

Once a processor is started, the middle square in the right side group on the status bar will be blue.  This will remain so for as long as the network has still a processor running.

Detailed view

If you press the first toggle button, a detailed view of the processor should open in a new tab (using the same name as that of the processor).

This tab is divided into 3 sections (from left to right):

• The content of the execution stack.  This stack contains the neurons that have to be solved by the processor.  You can add and remove neurons from/to this stack using the Pop and Push instructions (The first neuron is automatically added). In our demo, there is only 1 item currently on the stack, that’s the one being solved. This item displays  a square in the stack UI element.
• All the neurons that were assigned as meaning to the links starting from the solvable neuron. The top one will be executed first.  A special case is the ‘Actions’ neuron, which will only be solved when all the others have been, no matter where it is found in the list.
• A stack containing all the processor frames.  A frame represents a single cluster’s code that is being executed by the processor. Since some statements call other clusters, like conditional statements, code blocks,… a processor will usually have a number of frames active. This provides an exact view of the processor’s current execution location.
  The following info is provided for each frame, from top to bottom
  • The code, in textual form. Each code unit can easily be selected separately for debugging purposes (check out the context menu on each item to see what you can do with it).  It’s not possible to edit in this window though (you have to open the editor for this).
  • The name (or id when no name is defined) of the neuron that defines all the code.
  • The relationship between the neuron that defines the code and the actual cluster containing the code.  This can be:
    • Children: in this case, the neuron that defines the code, is the cluster that contains the code.
    • Rules: the neuron that defines the code has a link with meaning ‘Actions’ to a code cluster.
    • Actions: the neuron that defines the code has a link with meaning ‘Actions’ to a code cluster.

The detailed view is great to have an overview of where we are in the processing stage, but it isn’t very useful to  debug a program flow. It’s better to use the code editor for this because it provides a better overview of the connection between the statements.  To do this, open the editor containing the code and make certain the the processor in the debugger tab’s center screen is selected.  This last action is very important for the following reason: a detailed view is opened for a specific processor, so a single tab always links to a single processor.  Code however is not specific to a processor, it can be called by many processors. So it needs to know the processor context for displaying debug information.  It uses the processor that is selected in the  debugger overview for this purpose.  This has a nice  side effect: when you have multiple processors running, you can quickly switch between active processor and view where each one is in the code. Note that the code editor will put a red square around the statement that will be executed next, to indicate execution location.

Do the debug

Code flow

We are ready now to start debugging the network. The first thing you can do is walk through the code using the following commands:

• Run (F5): continue execution until the next breakpoint or until the processor is finished (only active when paused).
• Next step (F6): execute the next step only and  pause again (only active when paused). Note, when  you do a step next on a split instruction that creates multiple sub processors, all sub processors will also wait on the next statement, they wont run wild, but wait until you tell them what to do, which is very useful for inspecting stuff.
• Pause: stop execution and wait until the Run or Next step command have been given (only active when running).
• Stop: Ok, this sounds silly, I know, but I haven’t yet implemented the stop.  I’ve always been able to stop in other ways (I’m running the debugger inside another debugger which is great for creating a schizoid coding mind). If you feel an urgent need for this command, let me know, otherwise, it’ll get in there,  I just have absolutely no time frame for this.

Inspecting values

When a processor is paused, it is possible to inspect the value of any item that returns a result, this includes: variables, globals, result statements, bool expressions and ByRefs. You can do this by selecting the item you want to inspect with the mouse, and pressing F7 or through it’s context menu (Inspect value). This opens a dialog with the debug info of the result.  This can be empty, 1 or multiple neurons.

Debug info for a neuron contains the following info:

• All the incoming links, where they point to and the meaning of the link (note that the meaning UI element still has to be changed into a debug neuron).
• All the outgoing links, where they point to and the meaning of  the link.
• All the clusters that contain the parent node.
• When the parent node is a cluster, all it’s children.
• When the parent is a cluster, the meaning of the cluster is depicted, in brackets, after it’s name.  This is also a debug UI element.

The debug info is depicted recursively for all neurons. This type of debug visualization is used in multiple places throughout the designer.  You might have noticed that the echo channel uses this UI element to depict the incoming and outgoing neurons. The detailed view of a processor also uses it to depict most of the neurons.

Watches

A final feature of the debugger is watches.  These allow you to observe the content of variables and globals in a list for a single processor or across all processors for those that are paused. 

This feature is best experienced using the ‘English language definition’ demo since this performs some splits while processing text input. I have put a breakpoint in the ‘Code: Stage 1.1’ page on the second if statement  (first child if in the left-side path of the only root if) and ran it until ‘Found == Sentence(flow)’ to get the screenshot on the left.

To add watches, drag a variable or global and drop it in the left part of the debugger tab.  This will add it at the bottom of the list. If you are dragging it from a code editor, it is best to hold the ctrl key pressed, so that the item stays at it’s original position.  Note that it’s currently not yet possible to drag from the toolbox.  This is a small bug that still needs fixing. Also note that it’s not yet  possible to remove variables (except by editing the designer file).  This command will be added very soon (just goes to show how fresh the debugger still is).

By default, a list of watches with their values for the selected processor is depicted.  The left side is the name of the variable (or it’s id if no name has yet been assigned). To the right of the name, the content of the variable is displayed. This can be empty, 1 or more neurons, all of which are depicted using the debug UI element for neurons.

If you switch to variable view, the left section of the debugger tab will only display a radio button for each watch. The middle section, which contains processor info, will now display the contents of the selected watch for each processor. 

Also note that, when a processor gets split up into multiple sub procs, the view will switch to a tree.  The root node shows the number of processors contained by the node. If a new input event is sent to the network before the previous has been processed, it is added as a list item, at end of the list. If sub processors split up, more sub nodes are created as children of the nodes that triggered the split (tree structure). This allows you to see how processors are related to each other.  At the moment you can only see the current state, through the tree/list structure.  In the future, an extra view might be added that shows a line view over time to show when and how may processors did a split and when they died out, although that’s just an idea at the moment, so don’t put your hopes up to see it any time soon, there are still far more important things to do.

Errors and warnings

If there are any errors generated by the code, either through the Error or Warning instruction or because of an error in the code, you can view exactly where it occurred. All messages are stored in the log tab. When they are blue, you can double click on them. This will open a code editor with the statement selected  that caused the log item (note: if it is somewhere in a sub section, this is not expanded automatically) Because you can use the same statement in multiple locations, only the first few will be selected, this is to take care of some problems with WPF’s standard controls (will be fixed in the future).

As you can see, it’s all still fresh,  but functional. The debugger has already become a central point of usage for me and I suspect this will only increase, so this will get some extensive testing early on. There are plenty of things that still need adding like conditional breakpoints, counts on breakpoints, enabling-disabling of breakpoints, more commands on the breakpoints-list (like clear, enable/disable all,…),… These things will probably be added as needed.

I am certain there is plenty more to say about NND’s debugger, I just can’t think of anything  anymore, so I guess I’ll leave it by this for today. It’s already turned out a long enough post.

N2D is still in proto type stage, so there are plenty of idiosyncrasies to work around. Here are some tips and tricks that might make things a bit easier.

• When there is lots of code visible, things can slow down fast.  To avoid this, use the drill down/up arrows on some statements like conditional statements to close as much as possible.  If you are drilling down and need to see more code, you can always open sub items into a new code editor.  This will speed up things dramatically. You can do this on conditionals, their parts and code blocks
• Be careful with copying items around (or using shift drag). The editor displays items that are used in multiple places, but you can still be surprised when you change something to an argument only to see it changed it multiple places. The same goes for breakpoints: if an item is used in multiple places and it has a breakpoint defined, it will break everywhere the statement is used.  Keep this in mind while using copy or shift-drag.
• I use the sync with explorer command (F4) a lot. This was mainly because the copy paste system wasn’t working yet.  But it’s still useful I think: it’s a quick route into the explorer.
• The Toolbox is really useful to build code.  If you select the ‘Instructions’ section, you get a good overview of all the available instructions, grouped by functionality.  To quickly collapse and expand the groups, you can use the context menu of the toolbar.
• It’s possible to add items to the toolbox, although this currently has to be done by hacking the “XXXdesigner.xml” file.

Final note

Ok, this turned out to be a longer post than originally planned.  As you can see, it’s already a pretty extended little toolset, although there are still a number of shortcomings in the editor that still need to be worked out, more specifically:

• CPU and memory usage are a problem resulting in a very slow view when there is lot’s of visible data. This needs be solved by a custom control.
• Keyboard entry functionality, similar to the flow editor must still be implemented, this should speed up code editing significantly, compared to the all drag-drop or copy-paste solution.
• Clean up the views some so that they are clearer, compacter,…

Next up will be the debugger, I guess. 

All the different statement types, the general neuron types, some common static neurons, all the operators and the instructions can be accessed from the toolbox. You can easily drag and drop them on the editor.

Assignment

An assignment is used to assign one or more neurons to a variable. After the operation, the variable will reference the result of the right part of the assignment. The left part must always be a variable: regular or global, but no system variables are allowed.  What exactly is assigned to the left part, depends on the type of the right part.

• If it is another variable, the contents of the right part are assigned to the left part.
• It can also be a result statement, in which case the statement is executed and it’s result is assigned to the variable.
• ByRef statements are also resolved: the content of the ByRef statement will be assigned to the variable. This is useful if you want to assign a variable or a result statement to a variable and not it’s contents or results.
• In all other cases, the right part is assigned to the left part, so if you put a code block in the right section, it will not executed, instead, the variable will point to the code block.

(Result) Statement

A statement is used to call an instruction and provide all the parameter values for this instruction. You can call any type of instruction with a statement, even those that return values, although that’s not very useful cause any result values are lost when used in a statement.

You select the instruction using the drop down box.  The instruction itself is also just another neuron, that the statement links to.  This has some consequences.  For instance, if you want to find all the statements that use a specific instruction, just check the incoming links on the instruction.

Most instructions require 1 or more arguments. The values for these arguments are displayed in a list that comes after the instruction. This is a regular drop list, like a code list. Each  instruction should have a little description (you can see/edit this in the description tool-frame if you select the instruction in the toolbox).  This description should say all the required arguments and any possible result values.

A result statement works similar to a normal statement, except that it return a value and tend to be used as part of another statement, like an assignment, a Boolean expression or as an argument value.  Result statements are depicted using the same template, but with a slightly different color to make a distinction.

Code block

A code block is used to group other statements together. This allows you to reuse common functionality.  It is a bit similar to a function in traditional programming languages (like C, Pascal,..), except that there are no arguments allowed and it doesn’t allow for local variable values (scope locality), see Variables for more info on this one. Internally, a code block stores all the statements in a cluster that is linked to the statement.

It is possible to see the statements of a code block inline, by using the expand button in front of the name.  Note that this can slow the editor down dramatically.  Better is to open the code block in a new editor frame (make certain that ‘Statements‘ tab is selected).  You can do this through the context menu’s ‘View code’ command (can also be triggered from the main toolbar). If you want to reuse a code block, you can simply drag it from the explorer frame to the editor.

Even though you define parameters on code blocks, it’s still possible to pass along argument values.  Because the content of a variable isn’t reset when a code block is called, you can create some Variables that function as parameters, fill them before you call the block and access them inside the subroutine.  The same can be done for result values, you can return as many as you want, simply  define some variables and assign a value to them in your code block.

Conditional statement and it’s parts

A conditional statement is used by the processor to perform execution jumps (if, case, while, foreach,…).  Each conditional should contain at least 1 conditional part, but can have more. This depends on how it is set up. You add parts by dropping them in the ‘Children’ section of the UI element (which moves to the right).  You can also move them around, just make certain that the entire part is selected, and not just the ‘conditional’ section. When correctly selected, the part will have a selection border.

Visually, a conditional statement is always represented as a horizontal split in the vertical code branch.  When there is a repetition in the statement (for-each, looped, until, case-looped), 2 vertical lines are also visible, making the conditional statement boxed in. This is an easy visual queue to identify repetitions (so you can have a bit of a birds eye on the code to see the structure, without the details).

There are currently 5 different types of conditional statements (which can be selected from the drop down box): these are:

If statement

An ‘If’ statement can be compared with a structural language’s if statement. It can contain multiple parts, with only the last one allowed to have no condition. The first part who’s condition evaluates to true will have it’s code executed, all other parts are skipped from then on (so there is no fall through possible like in C’s case statement). After all the code in the part has been executed, the next statement after the if is performed, so there is no repetition in this type of conditional.

Adding statements to a part is done by dropping them in the ‘children’ section of the UI element. Because a condition must evaluate to either true or false, a Bool expression tends to be used, but a result statement can also be used. Simply drop an item in the ‘condition’ section.

Case statement

The case statement is, like the ‘if’ statement, very similar to it’s structural equivalent: the content of a value is compared against a number of different possibilities.  Each possibility is provided through the ‘condition’ of the part, which can be a static or another statement that returns a result. Only the last part can be empty, which will be executed if none of the previous values matched the variable.  An empty part is not required though. There is no fall through possible (unlike the switch in C).

When you select this type of conditional statement, the UI element will display an extra drop target for you to specify the variable  who’s content needs to be checked.  This has to be a variable (or global).

Looped statement

A looped conditional is comparable with  the ‘while’ statement, but with a twist: it is possible to define multiple parts.  It’s actually a bit like C’s while and if combined. The loop will run for as long as the condition in one of the parts evaluates to true.  Like the if statement, this part’s sub-statements will be executed.  The last part doesn’t require a condition, in that case, the loop will run for ever and will only stop when a break or one of the exit instructions is called. These instructions will always terminate a loop. If there  is no ‘empty part, the loop will stop when none of the conditions evaluates to true.

Until statement

The until conditional is similar to the looped variant in that it is repetitive, but it only allows 1 part that must always have a condition which is evaluated after all the sub statements have been executed.  When the condition evaluates to true, the repetition is stopped. This type of conditional is very similar with C’s do until loops.

Case looped statement

This conditional is a looped statement, but instead of using an ‘if’ kind of testing style, a case is used. This means that you need to provide an extra case item variable, like the case statement. The statement will loop for as long as a condition in one of the parts equals to the content of the case variable. The conditions in the parts should be static values, result statements, ByRefs, bool expression, variables or globals (so anything that can return a result). You can also specify an empty condition in the last part which will be activated if all of the other parts were different than the case variable.  This results in a never ending loop, which will only stop when the break or one of the exit instructions is called.

The case looped statement is a bit of a strange beast.  It’s origins can be traced back to Gene, where it made a lot of sense for parsing or generating stream.  Id don’t know if it’s useful in this context, so I simply left it in, you never know where this might end up: could be the dumpster, could be some cool trick.

For each statement

This type of conditional statement can probably best be compared to C#’s ForEach loop: it executes a peace of code for each neuron  in a list. Only 1 part is allowed. It’s condition should be a statement that returns some neurons.  This can be a variable or global that contains multiple items, or a result-statement that returns a number of items, like the one that returns all the children of a cluster.

When this type of conditional is selected, an extra drop target is visualized, which should reference a variable (or global) that will receive the current item from the list, as it’s content. This allows you to reference the loop item from within  the subroutine.  After the loop is done, this will still reference the last item in the list.

Bool expression

A Boolean expression is evaluated to the ‘True’ or ‘False’ neurons. It can be used as a conditional part’s condition. The left part is compared with the right part, using the specified operator. This statement type is probably recognizable from more classical programming languages.  There are a couple of important twists to this type of statement though:

First of all, because the operator is also a neuron, it can just as well be another result statement, or variable.   As long as you have something that can be evaluated to 1 or more operators, it’s ok. And this brings us to the second wrinkle:

Because variables, globals and result statements can return a list of neurons, so that the left, operator and/or right sides can have multiple items, the expression evaluates the lists using the following algorithm:

• If the left part has no result (no neurons) and all the operators are all ‘!=’ and the right side is not empty, true is returned, otherwise false.
• Otherwise: for all items in the left part, compare each item in the right using every operator.

Note, when there are multiple operators, it is like having an ‘and’ operator for conditions that work with the same left and right side, but different operators.  This will probably not be used a lot (I haven’t yet).  It’s just a result of the structure, so I let it be available.

All the operators are represented by statically declared neurons (so you can’t add your own). Most are recognizable from classical programming languages like C(++). You have:

The ‘Contains’ operator is a bit different. First of, there is no equivalent in traditional programming languages and it  requires a little more information to work properly: it needs to know which list to search: in the child list (in which case the left part should be a cluster),  or in the clustered-by list, which contains all the clusters that contain the neuron (this list is available in every neuron type, so the left part can be any type).

There is no operator precedence defined as in most traditional languages, since it is the structure of the neurons that defines the order.  This is visually verifiable by the statement’s container: first the left part is evaluated, next the operator and finally the right part.  If left or right has a sub bool expression, this is recursively repeated.

Search expression

This type of statement has already become deprecated. It’s been surpassed by the instructions, which provide more functionality and flexibility.  Nothing more to say about this, I guess.

Variable

A variable is a neuron that is able to store a reference to other neurons in the context of a processor. That is to  say, the content of a variable is only valid for as long as the processor running the code is still alive.  The contents of a variable can also be different for each processor. It will also be reset after the processor has completed evaluating all the links on a neuron and when a split callback function is called (scope locality).

A variable doesn’t have an initial value by default. It is possible to provide one though.  You need to unfold the drop location to make it accessible. This value can be another result statement, variable, global byref, or bool expression, in which case it is first resolved. In all other cases, the value is treated as a static.

Global

A global is similar to a variable, except that it has a different scope.  Globals are able to retain the same value for as long as the processor is alive while normal variables are reset at different times during the execution process of a processor. This is useful to pass along data generated during the execution of one link to another one.

Globals are also able to define what should happen during a split operation.  Possible values are:

• Empty: clear all values from the global in the new processors
• Duplicate: create duplicates from the content of the globals. A duplicate is a neuron with the same incoming and outgoing links, same value (in case of Int-, double- and Text-Neurons) and same children, but a different ID and not clustered by any item (so the clustered-by list is not duplicated, cause the clusters don’t get new neurons added because of a split).
• Copy: simply copy the content of the global to the new processors.

ByRef

The ByRef statement can best be compared with C’s ‘&’ operator, but instead of getting the address of the memory location, in a neural network, it prevents a statement that return a result (like a variable, result statement, bool expression) from calculating that result, but instead returns the statement itself.

Although this statement probably wont be used that much, it is very important for the split instruction to work properly, since this requires a variable neuron as one of it’s arguments.  Often, this statement is the only way to provide this value.  There are other situations where this statement can be used, but less often.

Lock

(Added 11/July/2010) The lock is used to combine a number of statements into a single  unit of execution, relative to 1 or more neurons and/or links. This statement can best be compared with C#’s lock keyword. When you lock a link, you need to specify from and to, but not the meaning, so each link lock actually requires 2 neurons to lock.

When you lock a neuron, the entire object is locked: incoming and outgoing links, value, parent cluster list and children. When you lock a link, only the incoming or outgoing section of the neuron is locked. This is important to keep in mind, if the locked items are operated on from other processors.

Don’t use the ‘duplicate’ instruction or perform any splits (with the split instruction) inside the lock. Although technically possible and allowed, it is not advised since this can and probably will cause deadlocks. Any object that needs to be duplicated during a split and which references the locked object, will cause the network the hang in a deadlock (although you can still terminate all the processors from the designer).

This concludes the overview of all the different statements.  Up next, some tips and tricks.

Before you get started with this post, might I suggest you to check the Echo words demo explanation for a more general introduction on how to use the code editor. This provides a good introduction for most of the general concepts. This post will deal with some more technical details.  Because of the length, I decided to split it into 3 posts:

• editing techniques: that’s this post
• all the different code statements
• and: tips and tricks.

Editing techniques

Code editing is currently based on a drag drop paradigm in which you drag statements to the editor and drop them at the appropriate location.

• Drop locations are indicated using a gray, rounded border, containing the name of the drop target, like ‘Args’, ‘Children’,…
• To add statements at the bottom, simply drop them somewhere in the white space below or at the side.
• To insert items, drop them on the little black line above the statement.  The previous statement will be moved down.
• When you drag an item and drop it somewhere else, it will be moved, except if the ’shift’ key is pressed, in which case the item will remain at it’s original position and will be copied to the drop target (no duplicate, but the same neuron is referenced. This is important!)
• You can also copy and paste items. During the copy process, the id’s of all the selected items are placed on the clipboard.  A paste will get the id’s from the clipboard, transform them into neurons which are put in the selected place.  You paste in drop targets, simply click in the drop target (there is no visual queue yet to indicate that the drop target is selected, this still needs some work) and paste. Note that copy-paste doesn’t yet work across multiple applications.
• If you press delete while an item is selected, it’s reference will be removed from the editor.  If the underlying neuron is no longer referenced anywhere else, it is removed.  All neurons that it references which are also no longer referenced are also removed (cascading delete).  This is the most used deletion method in code editing.  This means that when you delete a statement, all the parameter values that are no longer used, are also deleted. Or, if you delete a code block, all the statements that are no longer used anywhere else, are also deleted from the network.
• If you want more deletions options, use the ‘Delete special’ (ctrl+del)  command.  This shows a dialog which allows you to specify the deletion method you want:
  • simply remove the reference, but always leave the neuron in the network
  • Remove the reference and when  no longer used, delete it (as with the normal delete)
  • Always delete the neuron.Referenced neurons can be
  • left alone: they are not deleted
  • deleted when no longer referenced
  • always deleted
• All neurons can have a ‘display title‘. When the designer finds one for a neuron, this is displayed when possible.  You can easily change this name by pressing ‘F2′ or through the context menu of the selected item.  This is very useful for variables and globals.
• Most statements have a little circle in the front of the image.  This is a toggle button to enable/disable a breakpoint on the item. This is part of the debugger integration into the editor (more on that later).
• It’s also possible to change a statement from one type to another one through the context menu.  A warning though about this command, you might loose data with this command, that can’t yet be undone (no proper undo support yet for this command).
• If you simply need to move a statement up or down, you can also use the context menu, which has a submenu for moving items around.

The problem, you see, is the fact that N2D is a Designer, running on top of a neural network, accessing it’s services.  So when you are designing the network, it is also running.  This means that all the stuff you type into the text-channels, will also be stored in the project. Something you don’t want while performing tests.

To overcome this issue, N2D can use a special directory as a sandbox in which it can copy projects that need testing. The sandbox project will be started in it’s own designer so you can play with it, with all of it’s features (including neuron streaming) without messing up the actual data.

To run the current project in a sandbox, use the ‘Sandbox‘ toolbar button or menu item Debug/Sandbox. This will save the project, copy it over to a temp directory (selectable in the ‘Tools/Options‘ window) and start a new session.

When N2D is started in Sandbox mode, it will show this in the lower right corner, like so:

To start the demo, go to {Windows start}/N²D/Demos/Echo words if N²D isn’t already running, otherwise, do File/Open and browse to the folder {My documents}/NND/Demos/Echo. Note that a project is opened by selecting a folder, not through a file (this might still change in the future, if required).  If the project has already been opened, you can reopen it using File/Recent projects. Once opened, you should see something like this:

Note that the echo channel (a text channel) is shown by default. This is because I saved the project with the channel opened: visibility of a channel is stored, not for other items at the moment.  You can close this window by right clicking on the tab and selecting the close command. You can also control which channels are visible through the menu: View/Communication Channels. The designer will
create a communication channel for each sensory interface in the network.

The Text channel contains the following items:

• In the bottom: a Text box to enter text you want to send to the network
• with a toolbar on top of it containing
  • buttons to control the visibility of the top row items,
  • a toggle button for turning the speakers on/off,
  • a combo box to select the way the sin splits up the input strings into neurons
  • a button to send the text to the sin.
• in the middle is a list box containing the current conversation: all the text you sent interleaved with the answers of the network.
• and finally the top row contains 2 listboxes which contain visualizations for the neurons that were processed as input by the sin (left one) and received for output (right part).  Note: selecting a line in any of these 2 lists will also select the corresponding line in the conversation section and visa versa.

If you make certain that ‘Speaker‘ is pressed and your loud speakers are turned on, you should here ‘Type here’ once you press the ‘send‘ button (and haven’t changed the input text). Some items should also appear on the screen, something like:

What just happened? Well, a whole lot really:

• The text channel displays the string ‘Type here’ in the central dialog overview.  This is done by the designer.
• next, it sends the string to the Echo channel’s backing sensory interface which splits it up into 2 words: ‘type’ and ‘here’.  This is because it is in ‘Use Words’ mode.  There is another echo demo, called ‘echo letters’ which is programmed to ‘Use letters’ by the way.  If you want, you can open this and try it to see the difference. In the ‘echo words’ demo, the 2 words are converted into text neurons (by searching a dictionary of already created text neurons or by creating a new one, which is stored in the dictionary. These 2 text neurons are linked to a newly created neuron which represents the input event.  This conversion is done by the sin itself.
• The sin also raises an event to let the channel know about the neuron it just created as input value which is displayed in the top left list.  A processor is created and the neuron representing the input event is pushed on it’s stack, which starts of the network. Up until now, we haven’t really used the neural network yet, everything was done using traditional code. This changes once the processor is started.
• The processor ‘executes‘ the code attached to the links between the ‘input event’ neuron and the text neurons (how this code gets there and how it looks like will be dealt with later).  This is the neural network doing it’s thing which will eventually execute several ‘Output‘ instructions, sending neurons back to the sin.
• When the sensory interface receives these neurons, it converts them back into a string and raises an event. And so we get back into the world of regular code.
• The communication channel, which receives this event, displays the text in the conversation dialog and the neuron(s) that represent the string in the top right list box.

Ok, but what exactly happened in the neural network itself? Truth is, not much as you will see. It’s time to show some code. If you select the ‘Project‘ tool-frame, you will notice 3 items: a mind map (more on that later) and 2 code items, each one is simply a reference to a neuron in the network. To open them, double click on the item.  Let’s start with ‘Code: ContainsWord(in)‘.

Mmm, interesting. All GUI-like is probably the first thing you’ll notice. Indeed, there is currently no textual method for entering code, as more traditional development environments tend to use.  I choose this way for 2 reasons: first, it’s faster to develop (at least, in WPF that is) and it lowers the entry barrier since you can visually put items together instead of having to conceptually create a peace of text in your mind, requiring a grater level of knowledge. In the long run though, this technique tends to be slower than writing code and it also somewhat limits the amount of code displayed, but be patient, there is a solution for this.

Lets first go over the 2 lower tabs.  These represent the 2 different code lists that can be assigned to the neuron: the ‘Rules‘ and ‘Actions‘ list. Note that these lists are only created if there is code, for as long as you don’t drop any objects on the design surface, there isn’t actually a code list assigned to the neuron. The bottom tab-strip also has a context-menu which allows you to toggle wether this neuron is shown in the project tree (under the currently active folder, or the root if there is no folder). You can also create new code clusters from the sub toolbar, but remember these are code clusters (which can also point to code clusters), not neurons with code clusters assigned to them.

Both lists are called at different times, and serve a different purpose. Code in the Rules section, is called when this neuron is used as the meaning for a link that is currently being processed by the processor. The Actions code list is called by the processor when this neuron was processed by the processor and all other links have already been handled. Usually, you don’t program on the last tab (actions), it is provided to ‘view’ the code that was attached to it by the sin that generated it (see further). Note that there are other tab possibilities, but these have no importance for now and will be explained later on.

So what does this actually mean for our example: well, the ‘ContainsWord(in)‘ neuron is used by the sin as the meaning for the link between the generated neuron and each word in the input string (this is pre defined), so for each word in the input text, it will call this little peace of code.

As you can see there are only 2 statements in this code list. The second statement calls an ‘Output’ instruction which will send the content of the ‘CurrentTo‘ variable to the content of the ‘CurrentSin‘ variable, in other words, this is the actual echo. Both variables are system defined and are filled in by the processor, they can not be modified, unlike normal variables. You’ll notice that they appear to be multiple variables stacked on top of each other.  This is to indicate that the same neuron is used in multiple places.  It’s a visual cue to indicate if a code item is used somewhere else or not (ahh yes, unlike regular code, these are neurons, all linked to each other, there is nothing preventing you from literally using the same statement in different places). That’s it for doing the echo really.  The rest of the code is simply to make certain that the sin will put all the words in 1 line with spaces in between them. This is spread out over 2 locations: the first statement and the ‘Input actions‘ code list of the sin.

The first one, is a ‘Block‘ statement, which can best be compared to a ‘Function call’ except that there are no parameters or return value allowed. To view it’s content, use the drop down arrow for an inline visualization, or ‘View code‘  in the context menu to display the code in a new window. You should see something like this:

Note that the code is located in the ‘Statements‘ tab this time if you opened a new view. That’s because a code block has an extra known code cluster attached to it: the code for the block.  It can also have a rules and actions list, but these are not used in this example (and probably will rarely be used this way on code itself).

Ok, and what exactly does this code do? To explain that, perhaps a note first on how a text sin converts output data. Whenever it receives a neuron, it is converted to a string which is than sent as an event to the outside world.  Off course, in our example, we don’t show each word on a separate row, instead, we show the entire input statement on a single line with spaces between the words.  This can only be done if the sin knows where a sentence starts and where it ends, otherwise it simply sees a stream of output data. So to accomplish this, there are 2 special neurons that it recognizes: ‘BeginTextBlock‘ and ‘EndTextBlock‘. When these are sent to a text-sin, they are not converted, but instead, BeginTextBlock will make it accumulate all the strings until the EndTextBlock neuron is received. So that’s what this little bit of code does: it checks if this is the first word being processed, if so, the ‘BeginTextBlock’ neuron is sent and we store a switch so we know of this event.  If it wasn’t the first one, we sent a space to the sin, so we don’t concatenate the words. The ‘EndTextBlock’ is handled in a different way, we’ll get back to that later on.

First a bit about the different statements that were used in this little code block.  The top most one is a conditional statement, set to ‘if’.  N²D takes conditional statements even further than Gene and
completely unites all different conditionals into 1 and the same statement type, with an option to switch between them.  So a Do-while, While-Until, For-Each, For, if, case, and case-loop (not possible in other languages) are all combined into 1 statement, with a single neuron that switches between the 2. This approach is taken because of it’s ease in coding: it becomes a lot easier to change an if statement to a loop with multiple if’s: simply make it a loop (yes, you can have multiple conditions in a while loop). To design this bit of code, you can simply drag a ‘Conditional statement’ to the designer and drop ‘Conditional parts’ in the ‘Children’ area to the right. A conditional part forms a single trunk of the evaluation tree. It contains a list of code items (which can also be dropped on it’s respective ‘Children‘ drop area) and possibly a condition that determines if the part’s code is executed or not. This condition should evaluate to the ‘True’ or ‘False’ neurons.  Usually this is done by a Boolean expression (as has been done in this example), but this is not required, it could just as well have been a result statement that returns ‘True’ or ‘False’.

A word perhaps on the little arrows found on the variables to the right of the name.  These allow you to assign a default value to the variables.  In our example, we can’t use this cause all of them are system variables who’s values are controlled by the processor.  Regular values however don’t have a value the first time they are accessed in the context of a processor.  You can use this little drop target to assign a default value.  This can be a constant, or another statement that can be resolved to one or more neurons.

Another important fact about these variables is that they can point to 1 or more neurons.  They actually function as buckets.  This is very useful in that there is only 1 known type: a list of neurons.  This list can contain 1 item or more.  Often times, you will use a split to resolve multiple values to 1.

The last part of this demo can be found in the ‘Code: Echo channel‘ Project item, which is actually a reference to the text sin.  If you open the ‘Input actions’ tab, you’ll see the final 2 statements for this demo. This tab represents a code cluster that the text-sin attaches to each input neuron that it generates so it can be executed after all other links have been.  In our example, we do 2 things: let the sin know the sentence is closed so the string can be displayed and delete the input neuron, so we don’t let the network grow indefinitely. By default, all data is always stored, if you don’t want this, like in this example, you will need to manually delete the neuron.  All of it’s links and stuff will be cleaned
up automatically.

The notion that the code is spread out over different locations and called at different times is an important point: a neuron network can not be programmed in a monolithic fashion as a single code block. Code is always divided into small chunks, which run at different times, in an undetermined order.  This is where a lot of it’s power comes from.  Once you grasp this, you are a long way into designing something truly beautiful.

Gosh, dear all mighty, this was a climb indeed. I’m glad you were able to get till here, all the way to the end. Even though this is a small demo, it packs a punch when it comes to new ideas. There are plenty more areas to cover, but I am certain that once you become more familiar with the basic concepts explained in this demo, the rest of the ride will be downhill. More demo explanations to come soon…

Viewing and importing thesaurus data

N²D has access to the English version of the WordNet database through a sensory interface (input/output port). Apparently there are WordNet database for other languages, but these are not yet available. To open the view for the WordNet data activate the menu item: View/Communication Channels/WordNet (note: opening for the first time can take a little time as the database needs to be opened, should get fixed somehow in the future).  This should show something similar to (although with less text):

You can search for words in the database.  All known meanings will be displayed. You can also select to show a specific relationship type with the drop down list.  If ‘As Part’ is pressed, all entries that contain the specified text will be shown.  You can either import a single meaning separately or the whole lot, including all the relationships (advised) and see the network grow.  The system should be able to do multiple imports at the same time (you’ll notice that a single import can take a long time, depending on how many relationships there are for the word). While the network is learning, the green box in the right bottom corner is filled.

You can browse through all the imported items from the thesaurus view.  You can select the type of relationship you want to see.  All root tree items represent root neurons for the specified relationship. With the text box, you can search for words in the tree. All neurons can be dragged to other locations (like mind maps for instance, see further).  It is also possible to add items to the tree by dropping them on the level they should reside.

Creating Mind maps

Mind maps allow you to draw neurons in a free form manner, similar like vector graphics. There are multiple ways to create new mind maps, you can:

• use the menu: Insert/Mind map
• use the toolbar button (4th from left)
• or when on the ‘Project’ tool window, use the local toolbar button.  From here, you can also create sub directories, change the name of the project items (F2),…
  Note: Mind maps (and all other project items) are added to the currently selected folder or the folder that contains the currently selected item.

Most of the mind map commands are currently only accessible through it’s context menu. For creating new items, you can also use the toolbox. You can add different types of neurons or change them to another one.  Watch out with changing the type though, cause this might loose data sometimes (for instance, when you change a cluster into a regular neuron).

If you select a neuron, you can use different ways for showing ‘related‘ items, like children, incoming or outgoing links and clusters to which the selected item belongs. You can also manage to which clusters it belongs.  If you have multiple items selected, you can quickly create a cluster for them through the ‘Make cluster‘ command.

The easiest way to create a link between 2 items is by selecting them both (the first will be ‘From‘, the second ‘To‘) and activating the ‘Link‘ command. This will show a dialog with From and To already filled in, so you only have to select the meaning of the link.

As a side note, ‘Sync with explorer‘: a very useful command, accessible from the main toolbar, allows you to quickly find the selected item in the explorer.  This can be used from most places in the designer (all items that have a neuron as backing).

Drawing

Yes, N²D is also a drawing application, although some further evolution is clearly needed in this area.  Drawing capability is provided so you can work with the visual sensory interfaces. To create a new image channel, activate the menu command Insert/New Image channel.

Perhaps a word about the difference between a sensory interface and a communication channel: the underlying network has sins, which are also just neurons and have no ‘functionality‘. N²D, the designer, creates a ‘communication channel‘ for sins so you can work with them. If you use the network in a different application, the communication channel might look completely different, but the sin is still the same.

An image channel currently has 2 sections: the left part to manage/import multiple images (which is not yet fully functional) and a right part that will eventually contain 2 sections: an input and output section.  The output is currently not yet visible, but is supposed to function as the output part of the sin, so you can see images/videos that the network generates.

The input section works as an ink canvas, so you can use a pen table, and apply pen pressure (this can be toggled). The shape, size and color of the pen point can be selected. It is also possible to switch to a gum or selector.

Commenting

All neurons and some other types (like mind maps and frame editors) can have a description.  This is a FlowDocument (rich text format), so you format the text. Spell check is also provided. The description of the currently focused item is shown, you will have to make certain that the ‘Description’ Tool window is visible.

Standard text editing functionality is present:

• Select the font type
• and font size
• Bold/Italic/underline and Outline
• Bullets and numbering
• Spell check

As a side note: The selected item is a NeuronCluster, so it can have child neurons.  The explorer in this screenshot, is set to display it’s side panel which contains all the children of the selected cluster. Just like from the explorer, you are able to drag items from the left panel.  It’s also possible to drop neurons on the panel, which will add them as children to the selected cluster.

Creating frames

Frames are a form of compound objects (a group of neurons that form a single unit) which can be used to translate input from one form to another and to connect code to these compound objects. They are created in a similar way like mind maps, and can also be managed from within the project tool window.

To create a new frame, you basically push the toolbar buttons in sequence:

• The first one will create a new frame (a frame editor can contain multiple frames,usually they are related to each other somehow).
• Next, you create a frame element for the selected frame (make certain there is one selected, there are still some bugs in this area). You can also drop items to be used as frame elements.
• The third button creates a new frame evoker (all neurons in this list are put in a cluster ‘Evokers’, so you can easily do searches on them, more on that later). Like frame elements, you can also drop already existing items in there.
• On the last tab, you can edit all the sequences for the frame.  Use the 4th toolbar button to add a new sequence, make certain it is selected and use the buttons to add/remove or move up and down the frame elements from the first tab to build a sequence.
• The last toolbar button is used to display the ‘Import from FrameNet‘ dialog, with this frame editor already selected.

Importing frames

If you don’t feel like creating your own frames, you can always see if they have already been included in FrameNet, which you can import, either from the menu Tools/Import from FrameNet or from the
toolbar of a frame editor.

You can search FrameNet using the textbox in the upper right corner. The first time you select a row in the upper list might take a little time, this is because he is loading the WordNet database (which takes a little time). This is required to provide a mapping between FrameNet and WordNet data (not provided by default). Some mappings have already been created but most still need to be defined.  Each time you display this dialog and change some mappings, these will be saved.

There are many more things to discover in the designer, but I’m all out of words for today, so I’ll continue this on another occasion.  In the mean time, lot’s of fun playing with your new toy.

The whole thing is written in C#, the designer in WPF.  So for the time being it is strictly windows stuff (XP or vista). The core neural network and sensory interface libraries should be portable to Mono but
the designer will be a tat difficult. I don’t know yet how to tackle this one.

With regards to processors I can be simple: the more the better.  This is a massively multi threaded application, so the more cores the faster it should run.  I only have a dual core here, so I haven’t had the change yet to see it running on lots of threads simultaneously, but I would love to find out.  As I am writing this, I am sure there will still be a lot of bugs to filter out.

So to recap:

OS: Win XPSP2/Vista
Platform: .net/WPF
Hard Disk: + 450 MB
CPU Cores: as many as you can find