Namecoin SPA

Legacy: This article describes a concept that has been superseded in the course of ongoing development on the Organic Design wiki. Please do not develop this any further or base work on this concept, this is only useful for a historic record of work done. You may find a link to the currently used concept or function in this article, if not you can contact the author to find out what has taken the place of this legacy item.

I'm making a Single Page Application which exists entirely inside the Namecoin network's name:value storage system. Values can only be a maximum size of one kilobyte, so the boot-strapping has to be very modular. In fact it turns out that the current version has a bug that prevents values from being updated if the current value is greater than 520 bytes! There's discussion about raising the limit at some point to around 9KB, but incurring steeper transaction costs for updates greater than 1KB.

What it actually does

This is the current version of implementing the viewer onto the unified ontology. It is a web-browser based Single Page Application which resides within the namecoin peer-to-peer network. This application is essentially providing the network layer for the Nodal interface using corMVC project, and then continuing work on the development of the foundation ontology in this nodal environment.

First is jQuery which gives us an excellent framework for asynchronous (event-based) communications with the network-layer and powerful DOM-handling, CSS and GUI components making high-level aspects of in-browser application building much simpler. The Single Page Application pattern is also really well suited to jQuery and the nodal system.

Second is Namecoin which gives us a huge amount of required functionality in one package:

• Currency - it doesn't matter that Bitcoin is the more popular one, the main thing is that there's reliable means by which value can be stored and transferred between application users which can easily be exchanged for other currencies.
• Names - it gives us an internal DNS system which is completely independent of external services or authorities.
• P2P - these names can be used to store our data and already has a large critical mass of users who are really dedicated to keeping the content alive and well!
• Privacy - the Namecoin system already has robust methods by which we can lock content down to accessibility by only those having the private keys.

Notes

• The code uses very short variable names, minimal white-space and no comments because each node is limited to 520 bytes
• Those that are spread out and commented are in-progress
• Initially I'll run it using files instead of nodes to avoid unnecessary traffic and cost of namecoin updates and to assess "hosting" cost

The HTTP Server

The system uses it's own tiny HTTP server which is only about kilobyte of Perl code spread across a few nodes. There's no point using a real web-server package, because a normal web server is designed to serve files and map URLs to file names and directory paths, but this application has no need for that. Also even the lightest web servers like Lighttpd or NanoWeb are designed to server many concurrent users, whereas in this applications we're serving only the local user.

Our server has one simple job, which is to take the node name (first sequence of alphanumeric characters) in a GET request and return the content of the namecoin domain of the same name. And if no name was supplied, it returns a default HTML document composed of other namecoin domain's content.

Another purpose of the web server in the usual web-application environment is to provide the dynamic back-end behaviour of the site that ties all the requests together into an application and populates the page templates with dynamic information from the server. But in our case the entire application is written in JavaScript and runs in a single initially served HTML document, so the server is required only for retrieving the node content from the P2P network, not for generating any dynamic content or performing any other program logic.

Single Page Application

The URL format (http://localhost/#node/action) is really key to its operations. The only parts that ever change are the name of the node being requested (which is a namecoin domain name but with a prefix of "od_" added internally to avoid naming conflicts within the namecoin network) and the action part. Both of these parts come after the hash (#) which means the server is never requested for any URL change apart from internally by Ajax, yet all the global content in the network still has its own specific address. See Single Page Application for more details about this aspect.

Remember that the "hash fragment" in a URL is used to refer to information within the current document (or application) and hence doesn't involve a server request. But in this case, on the event of the hash fragment changing, an Ajax request is made for the content of the new node so that the application can rebuild (i.e. re-render, not re-load) its view within the context of the newly requested node.

Node

The general idea of a node is to have something similar to an object except that it's persistent and supports continuation (it can carry on from where it left off if something's required that takes time to occur, or can determine another node from where execution should carry on).

Nodal structure

Here I've described the nodes as they currently stand, the content is in the namecoin domains of these names, and is also in our subversion repository where they're a bit easier to work on and test. The first node is od_perl which is the initial boot-strapping node that defines and launches all further functionality including an HTTP server listening on port 2012 and the HTML document containing the application to be served to the local browser.

od_perl

This is the first node which declares two functions, the first called "d" (for data) takes a namecoin domain name and returns the content of its data value field, or "404" if no such name was found. All the names that make up this system start with the prefix "od_", so this is added automatically by the d function.

The second function called "c" (for code) extends the first function by taking one or more namecoin domain names and treating their values as the content of functions that it declares. After d and c are declared, it then calls c with "od_http" as the parameter to declare the contents of that domain as a function, and then calls that function.

sub d{$_=shift;$_=`namecoind name_show d/od_$_`=~/"value"\s*:\s*"(.*?[^\\])"/s?$1:"404";s/\\"/"/g;s/\\n/\n/g;$_}
sub c{eval"sub ".($_=shift)."{".d($_)."}"while$#_>=0}
c("http");&od_http;

od_http

This is a small server loop which listens for HTTP requests on port 2012, it declares functions from two other domains, "od_send" and "od_recv" which are called within the loop. The od_recv function is called when the server loop detects that input needs to be processed, the result will always be a domain name which is then passed through the d function to get its data content, and this value is sent back to the client as an HTTP response message by od_send.

use IO::Socket;c("send","recv");sub w{wait}$SIG{CHLD}=\&w;
$s=IO::Socket::INET->new(LocalPort=>2012,Type=>SOCK_STREAM,Reuse=>1,Listen=>10)or die"$@\n";
while($c=$s->accept){next if$p=fork;die"fork:$!\n"unless defined$p;od_send(&od_recv);exit fork}
continue{close$c;kill CHLD=>-$$}

od_send

This composes the passed content into an HTTP message and sends it to the client.

select$c;$|=1;$l=length$_;
print$c"HTTP/1.0 200 OK\r\nContent-Type: text/html; charset=UTF-8\nContent-Length: $l\n\n$_\r\n\r\n";
close$c

od_recv

Reads in the GET request text and extracts just the valid domain part of it if any. If no domain is requested, then an HTML page is constructed from "od_html" (an HTML DOCTYPE definition and opening element), "od_head" (meta tags, CSS and useful JS frameworks) and five JavaScript code fragments; od_data, od_code, od_json, od_loop and od_node.

The body of this default HTML document contains just a single <script> element composed of the of the five domain's content which are all JavaScript code fragments. Each code fragment declares a global function; $d, $c, $j, $l and $n respectively.

$i="";while(<$c>){last if/^\r\n$/;$i.=$_}
$_=$i=~/^GET \/([a-z0-9_]+)/?v($1):d("html").d("head")."<body>
<script type=\"text/javascript\">".d("data").c("code").c("json").c("loop").c("node")."</script>
</body></html>"

od_html

This defines the DOCTYPE definition, and opening HTML element used for all content within the interface.

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en" dir="ltr">

od_head

This is the HTML head element for our page including jQuery and jQueryUI which are practically prerequisites for any kind of web application these days.

<head>
   <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
   <script type="text/javascript" src="http://code.jquery.com/jquery-latest.min.js"></script>
   <script type="text/javascript" src="http://code.jquery.com/ui/jquery-ui-git.js"></script>
   <link rel="stylesheet" href="http://code.jquery.com/ui/jquery-ui-git.css" media="screen" />
</head>

od_data

This defines a JavaScript function called $d which performs the same role as the d function did in od_perl, i.e. to read in the data content for the passed namecoin domain name. However, this time we're in JavaScript context so the name needs to be requested from the od_http server using Ajax and passing a function that should deal with the result when it returns, i.e. Continuation-passing.

function $d(n,f){$.ajax({type:"GET",url:"/od_"+n,success:function(d){f(d)}})}

od_code

Again this is analogous to od_perl in that it defines a $c function that passes the request to d and then declares the result as a JavaScript function. The resulting function is put inside $c so that there's no possibility of conflict with other function names in the JavaScript global scope, and we so don't need the "od_" prefix on any of them. The $c function name is assumed to be unique across all scopes, allowing the declared functions to be accessed globally.

function $c(n,f){$d(n,function(d){eval("$c."+n+"=function(){"+d+"}");f()})}

od_json

This function declares another global function called "$j" (for JSON) through which domain items that hold relational content will be accessed. Like the previous two functions, it accepts a domain name and a callback function to deal with the resulting value (obtained via $d), but the result is treated as JSON data and stored as an object inside $j. The $j function looks in a property called "map" for the properties it will store for the node, the reason being that the JSON is then compatible with the namecoin network's method of encoding relationships between domains. If the JSON was not valid, or if there was no "map" property, an empty object will be stored. The resulting object is then passed to the processing function to be handled.

This level of functionality now gives us the ability to define an ontology, i.e. a triple-space of nodes with a reasonable number of relationships to other nodes (remember the JSON description must currently stay under 520 bytes). The node is available to the JavaScript environment as a persistent closure, i.e. a context which is stored when not in use, and when in use, is an executable scope containing variables and functions. When the functions execute, the this value will be set to the closure.

function $j(n,f){v=$v(n,function(v){$j[n]={};
try{j=$.parseJSON(v)}catch(e){j={}}
x=$j[n]={};if('map'in j){for(i in j.map)x[i]=j.map[i]}})}

// check whether it has data or code nodes and if so, convert then to actual data/code with $d and $c

// set its name so its accessible to its context
x.name=n;

// we add this method so that references to $j items resolve to their proper name when used as array keys etc
x.toString=function(){return this.name}

return x;

Notes to add

• $j items are just objects (not closures)
• named items with empty value refer to a node class that should be instantiated
• named item of name init should be declared as a function to be called on instantiationg
• named item with a value is a property
• named items repeated create lists - e.g. multiple instances of one kind of class

od_loop

This makes sure that the number of current JSON objects inside $j is kept to a maximum by disposing of the ones that are stale (most seldomly accessed). This is done by adding some properties to the objects in $j forming a closed linked-list, or loop, that has one position considered to be the start (and the previous, to be the end).

All newly created nodes (which are created simply by requesting items that aren't yet loaded or have been previously deleted - i.e. "lazy loaded") are inserted into the start point, becoming the new start, and if the total number becomes more than the maximum number allowed, then the one in the last position is deleted.

All the items within $j should now only be accessed via a $l global function which od_loop defines. The actual items are still stored within the $j function object, but they're accessed via $l.

The $l accessor function moves the $j object being accessed to the head of the list to ensure that only the objects with the least activity are ever deleted. Note that there is not really a list of items making up $l, i.e. there is no array like there is within $j, the loop accessed via $l exists entirely in the form of next and prev (n, p) relationships between the $j items.

A unique string is defined in the $w global to represent a unique return state meaning that execution of this context must wait until data is retrieved and that any function receiving this special value must return immediately, and its parent will do the same. It is assumed that the original source of the "wait state" (probably $l) will have ensured that this closure will be re-established when the required information is available. This isn't a very robust way of defining a unique wait state, but will do for now.

$w="_\x07";
function $l(x){

  // if item exists...
  if(x in $j){
    x=$j[x]; f=$l[0]; l=f.p;

    // unlink the item from its location in the loop
    x.n.p=x.p; x.p.n=x.n; x.n=f; x.p=l;

    // return the item to the caller
    return x;
  }

  // retrieve it via $j and send a function to handle it when it arrives
  $j(x,function(x){
    if(0 in $l){

      f=$l[0]; l=f.p;

      // insert the new item (a closure) into start of the loop removing last if > max
      x.n=f; f.p=x; 
      if(1000 in $l){ x.p=l.p; l.p.n=x; delete $j[x]; }else{ x.p=l; l.n=x; }
    }

    // this is the first $l items - make it a one-item loop
    else {$l[0]=x;x.n=x.p=x}

    // return execution to the closure that originally called $l
    $n.call(x)

  }
  
  // return the "waiting" flag
  return $w;
}

• see also understanding delete in javascript

od_node

• redoing this section soon - $n instantiates the $l objects as closures in the current closure (or root if none specified) and executes the init function. Any null valued associations in the context are instantiated as closures and their inits called, and so on recurslively. This creates a prototype-equivalent of a multiple-inheritence OO system.

This is the last of the five domains that declare global functions ($d, $c, $j, $l and $n) and that are read in as a JavaScript code fragment. Recall that these fragments are inserted directly into the body of the HTML document, and that the insertion is initially done by the od_recv function in the Perl server. All functionality from here on will be retrieved using $l so that it becomes part of the relational node structure.

The $n global function defined in this od_node fragment, is used to pass executional focus to a node in the $l loop and to allow that focus to flow in accord with the nodal relationships. The $n function allows this execution to be formed into hierarchical continuations that can execute smoothly regardless whether some operations must wait before execution can continue, such as when a node is requested that needs to be retrieved from the network, or when a user's response is needed.

After od_node is executed, all further program execution will be controlled by the relationships between nodes. It does this by treating a node as a reducible recursive queue.

If the $n function is called without any node name being specified, then by default, the od_root node, which begins initialising the nodal aspect of the application, is used.

If some executing code requests an object using $n and the object is already loaded, the reference is returned and the function execution continues on. However, if the object is not loaded, then the function returns null and execution should stop and any calling function will return null upon receiving a null - these nulls propagate all the way back to ensure a clean halt to the entire executing reduction structure. The same thread will continue again when the missing function has loaded because the the $n will ensure that the context from which it executed will be re established when the requested node has been loaded.

So in addition to defining $n, od_node's job is also to define the program continuation mechanism using relationships between nodes. Each node that gets focus may have an associated function to execute which will then be called with this set to that node entry within the $j structure. Functions must not rely on local scope across calls to $n because execution can be broken at that point by the requested object needing to be loaded.

The main purpose of od_node is basically to create a jQuery event-based version of the nodal reduction recursive toroidal execution algorithm developed in 2005. Here's the main reduction function written in C that od_node will be replicating.

// Moves "this" to node's focus if exists then rotates loop and executes/reduces the focus item void nodeReduce() { if(this = nodeLoopRotate(parent = this)) { // Move "this" to the focus in the node's loop and rotate nodeSetValue(this, nodePARENT, parent); // Update the parent association if(code = *nodeState(this, nodeCODE)) code(); // nodeCODE value is a pointer-index, execute as function-reference if non-zero } }

After od_node has defined the $n function, it then calls it with no parameter so that the default node is retrieved and executed.

$n=function(n,f){

  // todo

}
$n()

od_root

This is the default node that's retrieved and executed by the $n global function if no parameter was supplied to it. The job of od_root is to initialise the nodal aspect of the Single Page Application environment.

od_view

This represents which is a view from within the context of the node specified in the hash fragment of the current URL (or to put it in more common terms: it renders the website at the current location). The od_view node also hooks an event handler into the hash-changed event to cause the page to re-render in the new context.

od_view must also take the next alphanumeric portion of the hash-fragment and call the corresponding method in the node's closure, which in turn interprets any remaining part of the hash-fragment after the method name.

...todo...

// NOTE - view is a node so it needs to be in JSON format specifying its relations and pointing to another $j item for its code content

code:

h=window.location.hash.substr(1))

// process node part of fragment

// process method part of fragment

$(window).bind("hashchange",$n('view');

Nodal code conventions

The idea of this system is to try and set up a simple and familiar environment for code-execution by making the context in which a node's code executes a standard JavaScript closure, but some conventions are unavoidable since the code needs to work in a way that supports the persistence and continuation aspects of the nodal environment.

The first important issue is that continuation occurs by re-executing the same closure's code function which will execute from the start, not at the point it left off last time. This means that all the effects that the function has on its closure should be based on assessing the state of the properties in that closure, not on the local function scope which will not persist through a continuation. In simple terms, this just means before doing anything, check the environment (the closure referred to by this) to see what needs to be done.

The second important issue is that referring to other nodes is always done through the $l global function, and must be written in such a way as to return immediately should the unique $w (wait state) be returned instead of an object (a reference to the requested node's closure). Here's how this might be done for a retrieving a single node, which will result either in x being the new node reference, or bailing with a wait state for later continuation.

if( ( x = $l(n) ) == $w ) { return $w }  // we can't just do "x = $l(n)", we have to test for a $w result

See also

• Bug - cannot update a value if current value is greater than 520 bytes
• Common vision - interesting note on using vision as root node
• Namecoin
• Single Page Application
• Closure
• jQuery
• P2P
• Continuation
• Continuation-passing style
• Dynamically load js and css files - allows nodes to add their own css and js
• Nodal interface using corMVC