Talk:Nsd.c

Overview

The NodeSpace Daemon needs to be outlined clearly here so that development can continue bit-by-bit along with the other work threads such as MediaWiki extensions. The basic idea is that it's an applicational environment for workflow management. It is a wiki-like system of dealing with objects making up the work, and all the objects make up a distributed object space shared across and supported by the users. There are three main abstraction layers, Logic, Interface and Network which are described below.

Logic

This is the organisational environment used by the other two layers and by all work defined in the system. It is a means of describing processes and scheduling instances of the processes based on the descriptions. These descriptions are just nodally reducible sets of connected nodes, which involves all the basic nodal relationships such as root, next, prev, this, parent, class. All such sets are instances which undergo change. The change is not only in state, but also in connectivity - not just the change of the current focus which is inherent due to nodal reduction, but also "indeterminately" due to the instances decision-making processes. This allows workflow (executional control flow) to be determined dynamically at runtime if necessary, and also allows threads to be hooked in to events to avoid wait-states.

The Logic layer can really be summed up as being the nodal reduction environment which offers an object-oriented paradigm involving a method of recursive templating (class/instance) and an multi-threading-like executional model (recursive multiplexing). This behaviour is encapsulated in the Root node where every quantum of executional focus starts.

Network

The network layer handles all communication and storage of nodal data using various protocols and formats. Communicating or storing nodal information (as for any runtime information) is a process of serialisation and deserialisation. Nodal information is very easy to represent in any of the widely used formats such as XML, RDF or JSON. I've chosen to use RDF/Notation 3 for our second serialisation implementation because it is very easy to write in C and is most closely related to the nodal data structure (the first was an arbitrary wiki-like format, see serialise.c). The network layer must also handle non-nodal storage and communications work which can be broken down to mean the same thing as nodal work except without the serialisation stage (i.e. it's just the moving around of pre-built "packages" of information).

A number of protocols and storage media must be interfaced to the nodal environment, the most important being TCP/IP, Local FS and DHT. Other protocols will also be supported later such as email and SSH.

The Logic layer handles the breaking down of storage and communications jobs into smaller parts and scheduling these parts based on their size, the job priority and the performance statistics of the protocol(s) to be used for the work.

Interface

Applications these days are all based on the idea of templates designed to add layout and style to the processes and information making up the network content. The templates used in the rendering of various aspects of a system undergo collaboration and refinement in the same way as multiple languages are supported in a wiki. This same method also applies to different binding functionality to different programming languages or hardware architectures and to sets of constraints or properties defined by groups or organisations.

There is a lot of work involved in building an interface from the ground up, although that is what we eventually hope to do using SDL. For now though an easier option is to use HTML templates and use the browser as the interface. This would not be the usual crippled request-response based situation though, not only is the browser local to the daemon, but also it would use the 1x1 pixel SWF used by Livelets to integrate the browser DOM dynamically.

Generic Organisation

It was noted above that the Network and Interface layers make use of the Logic layer; in fact they're defined as workflow systems described in terms of the structure the Logic layer interprets and executes. Generic organisation is the idea of taking the largest set of functionality which is common to all the workflow systems which are required to fully model the Network and Interface layers and their components. This process yields a very definite organisational system because all the components being compared to find the commonality are already defined in terms of the Logic layer.

The behaviour of all the fundamental organisations has one general theme; they all involve the ongoing maintenance of persistent non-nodal representations of particular subsets of nodal content. These non-nodal representations are defined in terms of recursive templates of symbols such as unicode or binary. The ongoing maintenance aspect can vary in some ways such as whether its schedule-based or event-based, and it may rebuild the entire state each time, or just updated changes, but the point is that this general representation theme is re-used at all abstraction layers of an application or even a high-level organisation using concepts such as cashflow, budgetting or stock management.

Resource

This common behaviour is encapsulated in a single node called Resource which is the general concept of a dimension of resource (meaning that it's a numerical value which specifically represents a state between empty and full). It has input and output methods for supplying or consuming the resource and it may have maximum or minimum constraints. Part of this general node is also to accumulate and provide an interface to information on performance of the processes because higher layers of abstraction require such information to decide how many pieces to break the work into and what kind of protocols and schedule cycles are appropriate for the work.