Difference between revisions of "Namecoin SPA"

From Organic Design wiki
m
(Change source-code blocks to standard format)
 
(99 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 +
{{legacy}}
 
I'm making a [[Single Page Application]] which exists entirely inside the [[Namecoin]] network's name:value storage system. Values can only be a maximum size of one kilobyte, so the boot-strapping has to be very modular. In fact it turns out that the current version [https://github.com/namecoin/namecoin/issues/3 has a bug] that prevents values from being updated if the current value is greater than 520 bytes! There's discussion about raising the limit at some point to around 9KB, but incurring steeper transaction costs for updates greater than 1KB.
 
I'm making a [[Single Page Application]] which exists entirely inside the [[Namecoin]] network's name:value storage system. Values can only be a maximum size of one kilobyte, so the boot-strapping has to be very modular. In fact it turns out that the current version [https://github.com/namecoin/namecoin/issues/3 has a bug] that prevents values from being updated if the current value is greater than 520 bytes! There's discussion about raising the limit at some point to around 9KB, but incurring steeper transaction costs for updates greater than 1KB.
  
−
'''od_init''' is the first node which declares two functions, the first called "v" takes a namecoin domain name and returns the content of its value field, or "404" if no such name was found. The second function called "d" extends the first function by taking one or more namecoin domain names and treating their values as the content of functions thats it declares. After the two functions are declared, it then calls "d" with "od_http" as the parameter to declare the contents of that domain as a function, and then calls that function.
+
== What it actually does ==
−
{{code|<perl>sub v{$_=shift;$_=`namecoind name_show d/$_`=~/"value"\s*:\s*"(.*?[^\\])"/s?$1:"404";s/\\"/"/g;s/\\n/\n/g;$_}
+
This is the current version of implementing the [[viewer]] onto the [[unified ontology]]. It is a web-browser based [[Single Page Application]] which resides within the [[namecoin]] [[peer-to-peer]] network. This application is essentially providing the network layer for the [[Nodal interface using corMVC]] project, and then continuing work on the development of the [[foundation ontology]] in this nodal environment.
−
sub d{eval"sub ".($_=shift)."{".v($_)."}"while$♯_>=0}
 
−
d("od_http");&od_http;</perl>}}
 
  
 +
First is [[jQuery]] which gives us an excellent framework for asynchronous (event-based) communications with the network-layer and powerful DOM-handling, CSS and GUI components making high-level aspects of in-browser application building much simpler. The [[Single Page Application]] pattern is also really well suited to jQuery and the nodal system.
  
−
'''od_http''' is a small server loop which listens for HTTP requests on port 2012, it declares functions from two other domains, "od_send" and "od_recv" which are called within the loop. The ''od_recv'' function is called when the server loop detects that input needs to be processed, the result will always be a domain name which is then passed through the "v" function to get its value, and this value is sent back to the client as an HTTP response message by ''od_send''.
+
Second is [[Namecoin]] which gives us a huge amount of required functionality in one package:
−
{{code|<perl>use IO::Socket;d("od_send","od_recv");sub w{wait}$SIG{CHLD}=\&w;
+
*Currency - it doesn't matter that [[Bitcoin]] is the more popular one, the main thing is that there's reliable means by which value can be stored and transferred between application users which can easily be exchanged for other currencies.
−
$s=IO::Socket::INET->new(LocalPort=>2012,Type=>SOCK_STREAM,Reuse=>1,Listen=>10)or die"$@\n";
+
*Names - it gives us an internal DNS system which is completely independent of external services or authorities.
−
while($c=$s->accept){next if$pid=fork;die"fork:$!\n"unless defined$pid;od_send v od_recv;exit fork}
+
*P2P - these names can be used to store our data and already has a large critical mass of users who are really dedicated to keeping the content alive and well!
−
continue{close$c;kill CHLD=>-$$}</perl>}}
+
*Privacy - the Namecoin system already has robust methods by which we can lock content down to accessibility by only those having the private keys.
 +
 
 +
== Notes ==
 +
*The code uses very short variable names, minimal white-space and no comments because each node is limited to 520 bytes
 +
*Those that are spread out and commented are in-progress
 +
*Initially I'll run it using files instead of nodes to avoid unnecessary traffic and cost of namecoin updates and to assess "hosting" cost
 +
 
 +
== The HTTP Server ==
 +
The system uses it's own tiny HTTP server which is only about kilobyte of Perl code spread across a few nodes. There's no point using a real web-server package, because a normal web server is designed to serve files and map URLs to file names and directory paths, but this application has no need for that. Also even the lightest web servers like Lighttpd or NanoWeb are designed to server many concurrent users, whereas in this applications we're serving only the local user.
 +
 
 +
Our server has one simple job, which is to take the node name (first sequence of alphanumeric characters) in a GET request and return the content of the namecoin domain of the same name. And if no name was supplied, it returns a default HTML document composed of other namecoin domain's content.
 +
 
 +
Another purpose of the web server in the usual web-application environment is to provide the dynamic back-end behaviour of the site that ties all the requests together into an application and populates the page templates with dynamic information from the server. But in our case the entire application is written in JavaScript and runs in a single initially served HTML document, so the server is required only for retrieving the node content from the P2P network, not for generating any dynamic content or performing any other program logic.
 +
 
 +
== Single Page Application ==
 +
The URL format (http://localhost/#node/action) is really key to its operations. The only parts that ever change are the ''name'' of the node being requested (which is a namecoin domain name but with a prefix of "od_" added internally to avoid naming conflicts within the namecoin network) and the ''action'' part. Both of these parts come ''after'' the hash (#) which means the server is never requested for any URL change apart from internally by Ajax, yet all the global content in the network still has its own specific address. See [[Single Page Application]] for more details about this aspect.
 +
 
 +
Remember that the "hash fragment" in a URL is used to refer to information ''within the current document'' (or application) and hence doesn't involve a server request. But in this case, on the event of the hash fragment changing, an Ajax request is made for the content of the new node so that the application can rebuild (i.e. re-render, not re-load) its view within the context of the newly requested node.
  
 +
== Node ==
 +
The general idea of a [[node]] is to have something similar to an ''object'' except that it's persistent and supports [[w:Continuation|continuation]] (it can carry on from where it left off if something's required that takes time to occur, or can determine another node from where execution should carry on).
  
−
'''od_send''' composes the passed content into an HTTP message and sends it to the client.
+
== Nodal structure ==
−
{{code|<perl>select$c;$|=1;$l=length$_;
+
Here I've described the nodes as they currently stand, the content is in the namecoin domains of these names, and is also in [http://svn.organicdesign.co.nz/listing.php?repname=tools&path=%2Fod%2F& our subversion repository] where they're a bit easier to work on and test. The first node is ''od_perl'' which is the initial boot-strapping node that defines and launches all further functionality including an HTTP server listening on port 2012 and the HTML document containing the application to be served to the local browser.
−
print$c"HTTP/1.0 200 OK\r\nContent-Type: text/html; charset=UTF-8\nContent-Length: $l\n\n$_\r\n\r\n";
 
−
close$c;</perl>}}
 
  
 +
=== od_perl ===
 +
This is the first node which declares two functions, the first called "d" (for data) takes a namecoin domain name and returns the content of its data value field, or "404" if no such name was found. All the names that make up this system start with the prefix "od_", so this is added automatically by the ''d'' function.
  
−
'''od_recv''' reads in the GET request text and extracts just the valid domain part of it if any. If no domain is requested, then an HTML page is constructed from three other domains, "od_html" (an HTML DOCTYPE definition and opening element), "od_head" (meta tags, CSS and useful JS frameworks) abd "od_js" which is the next stage of the applications execution.
+
The second function called "c" (for code) extends the first function by taking one or more namecoin domain names and treating their values as the content of functions that it declares. After ''d'' and ''c'' are declared, it then calls ''c'' with "od_http" as the parameter to declare the contents of that domain as a function, and then calls that function.
−
{{code|<perl>$i="";while(<$c>){last if/^\r\n$/;$i.=$_}
+
<source lang="perl">
−
$_=$i=~/^GET \/([a-z0-9]+)/?$1:v("od_html").v("od_head")."<body><script type=\"text/javascript\">".v("od_json")."</script></body></html>";</perl>}}
+
sub d{$_=shift;$_=`namecoind name_show d/od_$_`=~/"value"\s*:\s*"(.*?[^\\])"/s?$1:"404";s/\\"/"/g;s/\\n/\n/g;$_}
 +
sub c{eval"sub ".($_=shift)."{".d($_)."}"while$#_>=0}
 +
c("http");&od_http;
 +
</source>
  
 +
==== od_http ====
 +
This is a small server loop which listens for HTTP requests on port 2012, it declares functions from two other domains, "od_send" and "od_recv" which are called within the loop. The ''od_recv'' function is called when the server loop detects that input needs to be processed, the result will always be a domain name which is then passed through the ''d'' function to get its data content, and this value is sent back to the client as an HTTP response message by ''od_send''.
 +
<source lang="perl">
 +
use IO::Socket;c("send","recv");sub w{wait}$SIG{CHLD}=\&w;
 +
$s=IO::Socket::INET->new(LocalPort=>2012,Type=>SOCK_STREAM,Reuse=>1,Listen=>10)or die"$@\n";
 +
while($c=$s->accept){next if$p=fork;die"fork:$!\n"unless defined$p;od_send(&od_recv);exit fork}
 +
continue{close$c;kill CHLD=>-$$}
 +
</source>
  
−
'''od_html''' is a standard DOCTYPE definition and opening HTML element for our applications page.
+
==== od_send ====
−
{{code|<xml><!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+
This composes the passed content into an HTTP message and sends it to the client.
−
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en" dir="ltr"></xml>}}
+
<source lang="perl">
 +
select$c;$|=1;$l=length$_;
 +
print$c"HTTP/1.0 200 OK\r\nContent-Type: text/html; charset=UTF-8\nContent-Length: $l\n\n$_\r\n\r\n";
 +
close$c
 +
</source>
  
 +
==== od_recv ====
 +
Reads in the GET request text and extracts just the valid domain part of it if any. If no domain is requested, then an HTML page is constructed from "od_html" (an HTML DOCTYPE definition and opening element), "od_head" (meta tags, CSS and useful JS frameworks) and five JavaScript code fragments; ''od_data'', ''od_code'', ''od_json'', ''od_loop'' and ''od_node''.
  
−
'''od_head''' is the HTML ''head'' element for our page including [[jQuery]] and jQueryUI which are practically prerequisites for any kind of web application these days.
+
The ''body'' of this default HTML document contains just a single &lt;script> element composed of the of the five domain's content which are all JavaScript code fragments. Each code fragment declares a global function; ''$d'', ''$c'', ''$j'', ''$l'' and ''$n'' respectively.
−
{{code|<xml><head>
+
<source lang="perl">
 +
$i="";while(<$c>){last if/^\r\n$/;$i.=$_}
 +
$_=$i=~/^GET \/([a-z0-9_]+)/?v($1):d("html").d("head")."<body>
 +
<script type=\"text/javascript\">".d("data").c("code").c("json").c("loop").c("node")."</script>
 +
</body></html>"
 +
</source>
 +
 
 +
=== od_html ===
 +
This defines the DOCTYPE definition, and opening HTML element used for all content within the interface.
 +
<source lang="xml">
 +
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
 +
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en" dir="ltr">
 +
</source>
 +
 
 +
==== od_head ====
 +
This is the HTML ''head'' element for our page including [[jQuery]] and jQueryUI which are practically prerequisites for any kind of web application these days.
 +
<source lang="xml">
 +
<head>
 
   <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
 
   <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
 
   <script type="text/javascript" src="http://code.jquery.com/jquery-latest.min.js"></script>
 
   <script type="text/javascript" src="http://code.jquery.com/jquery-latest.min.js"></script>
 
   <script type="text/javascript" src="http://code.jquery.com/ui/jquery-ui-git.js"></script>
 
   <script type="text/javascript" src="http://code.jquery.com/ui/jquery-ui-git.js"></script>
 
   <link rel="stylesheet" href="http://code.jquery.com/ui/jquery-ui-git.css" media="screen" />
 
   <link rel="stylesheet" href="http://code.jquery.com/ui/jquery-ui-git.css" media="screen" />
−
</head></xml>}}
+
</head>
 +
</source>
 +
 
 +
==== od_data ====
 +
This defines a JavaScript function called ''$d'' which performs the same role as the ''d'' function did in ''od_perl'', i.e. to read in the data content for the passed namecoin domain name. However, this time we're in JavaScript context so the name needs to be requested from the ''od_http'' server using Ajax and passing a function that should deal with the result when it returns, i.e. [[w:Continuation-passing style|Continuation-passing]].
 +
<source lang="js">
 +
function $d(n,f){$.ajax({type:"GET",url:"/od_"+n,success:function(d){f(d)}})}
 +
</source>
 +
 
 +
==== od_code ====
 +
Again this is analogous to ''od_perl'' in that it defines a ''$c'' function that passes the request to ''d'' and then declares the result as a JavaScript function. The resulting function is put inside ''$c'' so that there's no possibility of conflict with other function names in the JavaScript global scope, and we so don't need the "od_" prefix on any of them. The ''$c'' function name is assumed to be unique across all scopes, allowing the declared functions to be accessed globally.
 +
<source lang="js">
 +
function $c(n,f){$d(n,function(d){eval("$c."+n+"=function(){"+d+"}");f()})}
 +
</source>
 +
 
 +
==== od_json ====
 +
This function declares another global function called "$j" (for JSON) through which domain items that hold relational content will be accessed. Like the previous two functions, it accepts a domain name and a callback function to deal with the resulting value (obtained via ''$d''), but the result is treated as JSON data and stored as an object inside ''$j''. The ''$j'' function looks in a property called "map" for the properties it will store for the node, the reason being that the JSON is then compatible with the namecoin network's method of encoding relationships between domains. If the JSON was not valid, or if there was no "map" property, an empty object will be stored. The resulting object is then passed to the processing function to be handled.
 +
 
 +
This level of functionality now gives us the ability to define an [[ontology]], i.e. a [[web3|triple-space]] of nodes with a '''reasonable''' number of relationships to other nodes (remember the JSON description must currently stay under 520 bytes). The node is available to the JavaScript environment as a persistent [[closure]], i.e. a context which is stored when not in use, and when in use, is an executable scope containing variables and functions. When the functions execute, the ''this'' value will be set to the ''closure''.
 +
<source lang="js">
 +
function $j(n,f){v=$v(n,function(v){$j[n]={};
 +
try{j=$.parseJSON(v)}catch(e){j={}}
 +
x=$j[n]={};if('map'in j){for(i in j.map)x[i]=j.map[i]}})}
 +
 
 +
// check whether it has data or code nodes and if so, convert then to actual data/code with $d and $c
  
 +
// set its name so its accessible to its context
 +
x.name=n;
  
−
'''od_json''' is the initial script that gives us the ability to declare new functionality from other namecoin domain values, it defines ''$v'' and ''$d'' functions which perform the same role as the initial Perl stage. In this case the ''$v'' function is passed a domain name and a function references, it retrieves the value of the name from the Perl server via Ajax, and then executes the passed function when the value is returned.
+
// we add this method so that references to $j items resolve to their proper name when used as array keys etc
 +
x.toString=function(){return this.name}
  
−
Since all the names from here on are preceded by "od_" and we're declaring the functions inside ''$d'' to avoid naming conflicts, that common part can be ommitted from here on be added by ''$v'' when requesting the name from the Perl server. So for example, in the following code, "loop" is requested which gets retrieved from the ''od_loop'' domain, and can then be executed as ''$d.loop()'' after its retrieved and declared.
+
return x;
 +
</source>
  
−
The ''$d'' is also passed a name and a function reference, the name is passed to ''$v'' along with a function that declares a new function of the same name as the domain with its content being the value of the name. The new function is declared inside the ''$d'' function-object so it can be accessed from any scope.
+
Notes to add
 +
*$j items are just objects (not closures)
 +
*named items with empty value refer to a node class that should be instantiated
 +
*named item of name init should be declared as a function to be called on instantiationg
 +
*named item with a value is a property
 +
*named items repeated create lists - e.g. multiple instances of one kind of class
  
−
The ''$j'' (JSON) function is more high-level, like the previous two, its accepts a domain name and a callback function to deal with the resulting value (obtained via ''$v''), but the result is treated as JSON data and stored as an object inside ''$j''. The ''$j'' function looks in a property called "map" for the properties it will store for the node, the reason being that the JSON is then compatible with the namecoin network's method of encoding relationships between domains. If the JSON was not valid or there was no "map" property, an empty object will be stored. The resulting object is then passed to the processing function to handle.
+
==== od_loop ====
 +
This makes sure that the number of current JSON objects inside ''$j'' is kept to a maximum by disposing of the ones that are stale (most seldomly accessed). This is done by adding some properties to the objects in ''$j'' forming a closed linked-list, or [[loop]], that has one position considered to be the start (and the previous, to be the end).
  
−
This level of functionality now gives us the ability to define an [[Ontology]], i.e. a triple-space of nodes with a reasonable number of relationships to other nodes (remember the JSON description must currently stay under 520 bytes).
+
All newly created nodes (which are created simply by requesting items that aren't yet loaded or have been previously deleted - i.e. "lazy loaded") are inserted into the start point, becoming the new start, and if the total number becomes more than the maximum number allowed, then the one in the last position is deleted.
  
−
It then uses ''$d'' to declare a new function from the "od_loop" domain which defines some fundamental functionality for a small group of domains that will be treated as nodes.
+
All the items within ''$j'' should now only be accessed via a ''$l'' global function which ''od_loop'' defines. The actual items are still stored within the ''$j'' function object, but they're accessed via ''$l''.
−
{{code|<js>function $v(n,f){$.ajax({type:"GET",url:"/od_"+n,success:function(v){f(v)}})}
 
−
function $d(n,f){$v(n,function(v){eval("$d."+n+"=function(){"+v+"}");f()})}
 
−
function $j(n,f){v=$v(n,function(v){$j[n]={};try{j=$.parseJSON(v)}catch(e){j={}}if('map'in j){for(i in j.map)$j[n][i]=j.map[i]};f($j[n])})}
 
−
$d('loop',function(){$d.loop()})</js>}}
 
  
 +
The ''$l'' accessor function moves the ''$j'' object being accessed to the head of the list to ensure that only the objects with the least activity are ever deleted. Note that there is not really a list of items making up ''$l'', i.e. there is no array like there is within ''$j'', the loop accessed via ''$l'' exists entirely in the form of ''next'' and ''prev'' (''n'', ''p'') relationships between the ''$j'' items.
  
−
'''od_loop''' makes sure that the number of current JSON objects inside ''$j'' is kept to a maximum by disposing of the ones that haven't been accessed for longest. This is done by declaring another function called ''$l'' by which the objects are accessed rather than directly accessing them via ''$j''. This allows the objects in ''$j'' to be organised and "lazy loaded" on demand if necessray.
+
A unique string is defined in the ''$w'' global to represent a unique return state meaning that execution of this context must wait until data is retrieved and that any function receiving this special value must return immediately, and its parent will do the same. It is assumed that the original source of the "wait state" (probably ''$l'') will have ensured that this closure will be re-established when the required information is available. This isn't a very robust way of defining a unique wait state, but will do for now.
 +
<source lang="js">
 +
$w="_\x07";
 +
function $l(x){
  
−
The easiest way to do this is by adding some properties to the objects in ''$j'' forming a closed linked-list, or [[loop]], that has one position considered to be the start (and the previous, to be the end). All newly created nodes (which are created simply by requesting items that aren't yet loaded or have been previously deleted) are inserted into the start point, becoming the new start, and if the total number becomes more than the maximum number allowed, then the one in the last position is deleted.
+
  // if item exists...
 +
  if(x in $j){
 +
    x=$j[x]; f=$l[0]; l=f.p;
  
−
The ''$l'' accessor function moves the ''$j'' object being accessed to the head of the list to ensure that only the objects with the least activity are ever deleted. Note that there is not really a list of items making up ''$l'', i.e. there is no array like there is within ''$j'', the loop accessed via ''$l'' exists entirely in the form of ''next'' and ''prev'' relationships between the ''$j'' items.
+
    // unlink the item from its location in the loop
 +
    x.n.p=x.p; x.p.n=x.n; x.n=f; x.p=l;
  
 +
    // return the item to the caller
 +
    return x;
 +
  }
  
−
;od_node
+
  // retrieve it via $j and send a function to handle it when it arrives
−
After ''od_node'' is executed, all further program execution will be controlled by the relationships between nodes. It does this by treating a node as a reducable recursive queue, itself being the default starting point for this process if there is no node specified in the hash-fragment of the current URL. It declares another function from the ''od_hash'' domain, which allows this focal point to be connected to another node at any time by changing the hash-fragment, either directly, or by clicking on a link in the page that has a hash address.
+
  $j(x,function(x){
 +
    if(0 in $l){
  
−
If some executing code requests an object using ''$n'' and the object is already loaded, the reference is returned and the function execution continues on. However, if the object is not loaded, then the function returns ''null'' and execution should stop and any calling function will return ''null'' upon receivng a null - these nulls propagate all the way back to ensure a clean halt to the entire executing reduction structure. The same thread will continue again when the missing function has loaded because the the ''$n'' will ensure that the context from which it exceuted will be reestablished when the requested node has been loaded.
+
      f=$l[0]; l=f.p;
  
−
So in addition to defining ''$n'', ''od_node's'' job is also to define the program-flow mechanism using relationships between nodes. Each node that gets focus may have an associated function to execute which will then be called with ''this'' set to that node entry within the ''$j'' structure. Functions must not rely on local scope across calls to ''$n'' because execution can be broken at that point by the requested object needing to be loaded.
+
      // insert the new item (a closure) into start of the loop removing last if > max
 +
      x.n=f; f.p=x;
 +
      if(1000 in $l){ x.p=l.p; l.p.n=x; delete $j[x]; }else{ x.p=l; l.n=x; }
 +
    }
 +
 
 +
    // this is the first $l items - make it a one-item loop
 +
    else {$l[0]=x;x.n=x.p=x}
 +
 
 +
    // return execution to the closure that originally called $l
 +
    $n.call(x)
 +
 
 +
  }
 +
 
 +
  // return the "waiting" flag
 +
  return $w;
 +
}
 +
</source>
 +
*see also [http://perfectionkills.com/understanding-delete/ understanding delete in javascript]
 +
 
 +
==== od_node ====
 +
*redoing this section soon - $n instantiates the $l objects as [[closure]]s in the current closure (or root if none specified) and executes the init function. Any null valued associations in the context are instantiated as closures and their inits called, and so on recurslively. This creates a prototype-equivalent of a multiple-inheritence OO system.
 +
 
 +
 
 +
This is the last of the five domains that declare global functions (''$d'', ''$c'', ''$j'', ''$l'' and ''$n'') and that are read in as a JavaScript code fragment. Recall that these fragments are inserted directly into the body of the HTML document, and that the insertion is initially done by the ''od_recv'' function in the Perl server. All functionality from here on will be retrieved using ''$l'' so that it becomes part of the relational node structure.
 +
 
 +
The ''$n'' global function defined in this ''od_node'' fragment, is used to pass executional focus to a node in the ''$l'' loop and to allow that focus to flow in accord with the nodal relationships. The ''$n'' function allows this execution to be formed into hierarchical [[w:Continuation|continuation]]s that can execute smoothly regardless whether some operations must wait before execution can continue, such as when a node is requested that needs to be retrieved from the network, or when a user's response is needed.
 +
 
 +
After ''od_node'' is executed, all further program execution will be controlled by the relationships between nodes. It does this by treating a node as a reducible recursive queue.
 +
 
 +
If the ''$n'' function is called without any node name being specified, then by default, the ''od_root'' node, which begins initialising the nodal aspect of the application, is used.
 +
 
 +
If some executing code requests an object using ''$n'' and the object is already loaded, the reference is returned and the function execution continues on. However, if the object is not loaded, then the function returns ''null'' and execution should stop and any calling function will return ''null'' upon receiving a null - these nulls propagate all the way back to ensure a clean halt to the entire executing reduction structure. The same thread will continue again when the missing function has loaded because the the ''$n'' will ensure that the context from which it executed will be re established when the requested node has been loaded.
 +
 
 +
So in addition to defining ''$n'', ''od_node's'' job is also to define the [[w:Continuation|program continuation]] mechanism using relationships between nodes. Each node that gets focus may have an associated function to execute which will then be called with ''this'' set to that node entry within the ''$j'' structure. Functions must not rely on local scope across calls to ''$n'' because execution can be broken at that point by the requested object needing to be loaded.
  
 
The main purpose of ''od_node'' is basically to create a jQuery event-based version of the [[nodal reduction]] recursive toroidal execution algorithm developed in 2005. Here's the main reduction function written in C that ''od_node'' will be replicating.
 
The main purpose of ''od_node'' is basically to create a jQuery event-based version of the [[nodal reduction]] recursive toroidal execution algorithm developed in 2005. Here's the main reduction function written in C that ''od_node'' will be replicating.
 
{{code|{{:NR.c}}}}
 
{{code|{{:NR.c}}}}
  
 +
After ''od_node'' has defined the ''$n'' function, it then calls it with no parameter so that the default node is retrieved and executed.
 +
<source lang="js">
 +
$n=function(n,f){
 +
 +
  // todo
 +
 +
}
 +
$n()
 +
</source>
 +
 +
=== od_root ===
 +
This is the default node that's retrieved and executed by the ''$n'' global function if no parameter was supplied to it. The job of ''od_root'' is to initialise the nodal aspect of the Single Page Application environment.
 +
 +
=== od_view ===
 +
This represents which is a view from within the context of the node specified in the hash fragment of the current URL (or to put it in more common terms: it renders the website at the current location). The ''od_view'' node also hooks an event handler into the hash-changed event to cause the page to re-render in the new context.
 +
 +
''od_view'' must also take the next alphanumeric portion of the hash-fragment and call the corresponding method in the node's closure, which in turn interprets any remaining part of the hash-fragment after the method name.
 +
 +
<source lang="js">
 +
...todo...
 +
 +
// NOTE - view is a node so it needs to be in JSON format specifying its relations and pointing to another $j item for its code content
 +
 +
code:
 +
 +
h=window.location.hash.substr(1))
 +
 +
// process node part of fragment
 +
 +
// process method part of fragment
 +
 +
$(window).bind("hashchange",$n('view');
 +
</source>
 +
 +
== Nodal code conventions ==
 +
The idea of this system is to try and set up a simple and familiar environment for code-execution by making the context in which a node's code executes a standard JavaScript closure, but some conventions are unavoidable since the code needs to work in a way that supports the persistence and continuation aspects of the nodal environment.
  
−
;od_hash
+
The first important issue is that continuation occurs by re-executing the same closure's ''code'' function which will execute from the start, not at the point it left off last time. This means that all the effects that the function has on its closure should be based on assessing the state of the properties in that closure, not on the local function scope which will not persist through a continuation. In simple terms, this just means before doing anything, check the environment (the closure referred to by ''this'') to see what needs to be done.
−
When the URL hash-fragment changes, it should behave similarly to a page reload except without needing to reload anything. The result is that the focus (of nodal reduction) moves to the new node specified in the fragment, effectely putting the prior context "on ice" until it regains focus at a later time. Remember that the ''od_node'' function treats any passed node name as a reducable queue to execute if one is passed, and only does its function declaration followed by default-node reduction if no node-name was passed.
 
−
{{code|<js>$(window).bind("hashchange",$d.od_node(window.location.hash.substr(1)));</js>}}
 
  
 +
The second important issue is that referring to other nodes is always done through the ''$l'' global function, and must be written in such a way as to return immediately should the unique ''$w'' (wait state) be returned instead of an object (a reference to the requested node's closure). Here's how this might be done for a retrieving a single node, which will result either in ''x'' being the new node reference, or bailing with a wait state for later continuation.
 +
<source lang="js">
 +
if( ( x = $l(n) ) == $w ) { return $w }  // we can't just do "x = $l(n)", we have to test for a $w result
 +
</source>
  
 
== See also ==
 
== See also ==
 
*[https://github.com/namecoin/namecoin/issues/3 Bug - cannot update a value if current value is greater than 520 bytes]
 
*[https://github.com/namecoin/namecoin/issues/3 Bug - cannot update a value if current value is greater than 520 bytes]
 
*[[Common vision]] ''- interesting note on using vision as root node''
 
*[[Common vision]] ''- interesting note on using vision as root node''
 +
*[[Namecoin]]
 +
*[[Single Page Application]]
 +
*[[Closure]]
 +
*[[jQuery]]
 +
*[[P2P]]
 +
*[[w:Continuation|Continuation]]
 +
*[[w:Continuation-passing style|Continuation-passing style]]
 +
*[http://www.javascriptkit.com/javatutors/loadjavascriptcss.shtml Dynamically load js and css files] ''- allows nodes to add their own css and js''
 +
*[[Nodal interface using corMVC]]

Latest revision as of 18:11, 22 May 2015

Legacy.svg Legacy: This article describes a concept that has been superseded in the course of ongoing development on the Organic Design wiki. Please do not develop this any further or base work on this concept, now this page is for historic record only.

I'm making a Single Page Application which exists entirely inside the Namecoin network's name:value storage system. Values can only be a maximum size of one kilobyte, so the boot-strapping has to be very modular. In fact it turns out that the current version has a bug that prevents values from being updated if the current value is greater than 520 bytes! There's discussion about raising the limit at some point to around 9KB, but incurring steeper transaction costs for updates greater than 1KB.

What it actually does

This is the current version of implementing the viewer onto the unified ontology. It is a web-browser based Single Page Application which resides within the namecoin peer-to-peer network. This application is essentially providing the network layer for the Nodal interface using corMVC project, and then continuing work on the development of the foundation ontology in this nodal environment.

First is jQuery which gives us an excellent framework for asynchronous (event-based) communications with the network-layer and powerful DOM-handling, CSS and GUI components making high-level aspects of in-browser application building much simpler. The Single Page Application pattern is also really well suited to jQuery and the nodal system.

Second is Namecoin which gives us a huge amount of required functionality in one package:

  • Currency - it doesn't matter that Bitcoin is the more popular one, the main thing is that there's reliable means by which value can be stored and transferred between application users which can easily be exchanged for other currencies.
  • Names - it gives us an internal DNS system which is completely independent of external services or authorities.
  • P2P - these names can be used to store our data and already has a large critical mass of users who are really dedicated to keeping the content alive and well!
  • Privacy - the Namecoin system already has robust methods by which we can lock content down to accessibility by only those having the private keys.

Notes

  • The code uses very short variable names, minimal white-space and no comments because each node is limited to 520 bytes
  • Those that are spread out and commented are in-progress
  • Initially I'll run it using files instead of nodes to avoid unnecessary traffic and cost of namecoin updates and to assess "hosting" cost

The HTTP Server

The system uses it's own tiny HTTP server which is only about kilobyte of Perl code spread across a few nodes. There's no point using a real web-server package, because a normal web server is designed to serve files and map URLs to file names and directory paths, but this application has no need for that. Also even the lightest web servers like Lighttpd or NanoWeb are designed to server many concurrent users, whereas in this applications we're serving only the local user.

Our server has one simple job, which is to take the node name (first sequence of alphanumeric characters) in a GET request and return the content of the namecoin domain of the same name. And if no name was supplied, it returns a default HTML document composed of other namecoin domain's content.

Another purpose of the web server in the usual web-application environment is to provide the dynamic back-end behaviour of the site that ties all the requests together into an application and populates the page templates with dynamic information from the server. But in our case the entire application is written in JavaScript and runs in a single initially served HTML document, so the server is required only for retrieving the node content from the P2P network, not for generating any dynamic content or performing any other program logic.

Single Page Application

The URL format (http://localhost/#node/action) is really key to its operations. The only parts that ever change are the name of the node being requested (which is a namecoin domain name but with a prefix of "od_" added internally to avoid naming conflicts within the namecoin network) and the action part. Both of these parts come after the hash (#) which means the server is never requested for any URL change apart from internally by Ajax, yet all the global content in the network still has its own specific address. See Single Page Application for more details about this aspect.

Remember that the "hash fragment" in a URL is used to refer to information within the current document (or application) and hence doesn't involve a server request. But in this case, on the event of the hash fragment changing, an Ajax request is made for the content of the new node so that the application can rebuild (i.e. re-render, not re-load) its view within the context of the newly requested node.

Node

The general idea of a node is to have something similar to an object except that it's persistent and supports continuation (it can carry on from where it left off if something's required that takes time to occur, or can determine another node from where execution should carry on).

Nodal structure

Here I've described the nodes as they currently stand, the content is in the namecoin domains of these names, and is also in our subversion repository where they're a bit easier to work on and test. The first node is od_perl which is the initial boot-strapping node that defines and launches all further functionality including an HTTP server listening on port 2012 and the HTML document containing the application to be served to the local browser.

od_perl

This is the first node which declares two functions, the first called "d" (for data) takes a namecoin domain name and returns the content of its data value field, or "404" if no such name was found. All the names that make up this system start with the prefix "od_", so this is added automatically by the d function.

The second function called "c" (for code) extends the first function by taking one or more namecoin domain names and treating their values as the content of functions that it declares. After d and c are declared, it then calls c with "od_http" as the parameter to declare the contents of that domain as a function, and then calls that function. 

sub d{$_=shift;$_=`namecoind name_show d/od_$_`=~/"value"\s*:\s*"(.*?[^\\])"/s?$1:"404";s/\\"/"/g;s/\\n/\n/g;$_}
sub c{eval"sub ".($_=shift)."{".d($_)."}"while$#_>=0}
c("http");&od_http;

od_http

This is a small server loop which listens for HTTP requests on port 2012, it declares functions from two other domains, "od_send" and "od_recv" which are called within the loop. The od_recv function is called when the server loop detects that input needs to be processed, the result will always be a domain name which is then passed through the d function to get its data content, and this value is sent back to the client as an HTTP response message by od_send. 

use IO::Socket;c("send","recv");sub w{wait}$SIG{CHLD}=\&w;
$s=IO::Socket::INET->new(LocalPort=>2012,Type=>SOCK_STREAM,Reuse=>1,Listen=>10)or die"$@\n";
while($c=$s->accept){next if$p=fork;die"fork:$!\n"unless defined$p;od_send(&od_recv);exit fork}
continue{close$c;kill CHLD=>-$$}

od_send

This composes the passed content into an HTTP message and sends it to the client. 

select$c;$|=1;$l=length$_;
print$c"HTTP/1.0 200 OK\r\nContent-Type: text/html; charset=UTF-8\nContent-Length: $l\n\n$_\r\n\r\n";
close$c

od_recv

Reads in the GET request text and extracts just the valid domain part of it if any. If no domain is requested, then an HTML page is constructed from "od_html" (an HTML DOCTYPE definition and opening element), "od_head" (meta tags, CSS and useful JS frameworks) and five JavaScript code fragments; od_data, od_code, od_json, od_loop and od_node.

The body of this default HTML document contains just a single <script> element composed of the of the five domain's content which are all JavaScript code fragments. Each code fragment declares a global function; $d, $c, $j, $l and $n respectively. 

$i="";while(<$c>){last if/^\r\n$/;$i.=$_}
$_=$i=~/^GET \/([a-z0-9_]+)/?v($1):d("html").d("head")."<body>
<script type=\"text/javascript\">".d("data").c("code").c("json").c("loop").c("node")."</script>
</body></html>"

od_html

This defines the DOCTYPE definition, and opening HTML element used for all content within the interface. 

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en" dir="ltr">

od_head

This is the HTML head element for our page including jQuery and jQueryUI which are practically prerequisites for any kind of web application these days. 

<head>
   <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
   <script type="text/javascript" src="http://code.jquery.com/jquery-latest.min.js"></script>
   <script type="text/javascript" src="http://code.jquery.com/ui/jquery-ui-git.js"></script>
   <link rel="stylesheet" href="http://code.jquery.com/ui/jquery-ui-git.css" media="screen" />
</head>

od_data

This defines a JavaScript function called $d which performs the same role as the d function did in od_perl, i.e. to read in the data content for the passed namecoin domain name. However, this time we're in JavaScript context so the name needs to be requested from the od_http server using Ajax and passing a function that should deal with the result when it returns, i.e. Continuation-passing. 

function $d(n,f){$.ajax({type:"GET",url:"/od_"+n,success:function(d){f(d)}})}

od_code

Again this is analogous to od_perl in that it defines a $c function that passes the request to d and then declares the result as a JavaScript function. The resulting function is put inside $c so that there's no possibility of conflict with other function names in the JavaScript global scope, and we so don't need the "od_" prefix on any of them. The $c function name is assumed to be unique across all scopes, allowing the declared functions to be accessed globally. 

function $c(n,f){$d(n,function(d){eval("$c."+n+"=function(){"+d+"}");f()})}

od_json

This function declares another global function called "$j" (for JSON) through which domain items that hold relational content will be accessed. Like the previous two functions, it accepts a domain name and a callback function to deal with the resulting value (obtained via $d), but the result is treated as JSON data and stored as an object inside $j. The $j function looks in a property called "map" for the properties it will store for the node, the reason being that the JSON is then compatible with the namecoin network's method of encoding relationships between domains. If the JSON was not valid, or if there was no "map" property, an empty object will be stored. The resulting object is then passed to the processing function to be handled.

This level of functionality now gives us the ability to define an ontology, i.e. a triple-space of nodes with a reasonable number of relationships to other nodes (remember the JSON description must currently stay under 520 bytes). The node is available to the JavaScript environment as a persistent closure, i.e. a context which is stored when not in use, and when in use, is an executable scope containing variables and functions. When the functions execute, the this value will be set to the closure. 

function $j(n,f){v=$v(n,function(v){$j[n]={};
try{j=$.parseJSON(v)}catch(e){j={}}
x=$j[n]={};if('map'in j){for(i in j.map)x[i]=j.map[i]}})}

// check whether it has data or code nodes and if so, convert then to actual data/code with $d and $c

// set its name so its accessible to its context
x.name=n;

// we add this method so that references to $j items resolve to their proper name when used as array keys etc
x.toString=function(){return this.name}

return x;

Notes to add

  • $j items are just objects (not closures)
  • named items with empty value refer to a node class that should be instantiated
  • named item of name init should be declared as a function to be called on instantiationg
  • named item with a value is a property
  • named items repeated create lists - e.g. multiple instances of one kind of class

od_loop

This makes sure that the number of current JSON objects inside $j is kept to a maximum by disposing of the ones that are stale (most seldomly accessed). This is done by adding some properties to the objects in $j forming a closed linked-list, or loop, that has one position considered to be the start (and the previous, to be the end).

All newly created nodes (which are created simply by requesting items that aren't yet loaded or have been previously deleted - i.e. "lazy loaded") are inserted into the start point, becoming the new start, and if the total number becomes more than the maximum number allowed, then the one in the last position is deleted.

All the items within $j should now only be accessed via a $l global function which od_loop defines. The actual items are still stored within the $j function object, but they're accessed via $l.

The $l accessor function moves the $j object being accessed to the head of the list to ensure that only the objects with the least activity are ever deleted. Note that there is not really a list of items making up $l, i.e. there is no array like there is within $j, the loop accessed via $l exists entirely in the form of next and prev (n, p) relationships between the $j items.

A unique string is defined in the $w global to represent a unique return state meaning that execution of this context must wait until data is retrieved and that any function receiving this special value must return immediately, and its parent will do the same. It is assumed that the original source of the "wait state" (probably $l) will have ensured that this closure will be re-established when the required information is available. This isn't a very robust way of defining a unique wait state, but will do for now. 

$w="_\x07";
function $l(x){

  // if item exists...
  if(x in $j){
    x=$j[x]; f=$l[0]; l=f.p;

    // unlink the item from its location in the loop
    x.n.p=x.p; x.p.n=x.n; x.n=f; x.p=l;

    // return the item to the caller
    return x;
  }

  // retrieve it via $j and send a function to handle it when it arrives
  $j(x,function(x){
    if(0 in $l){

      f=$l[0]; l=f.p;

      // insert the new item (a closure) into start of the loop removing last if > max
      x.n=f; f.p=x; 
      if(1000 in $l){ x.p=l.p; l.p.n=x; delete $j[x]; }else{ x.p=l; l.n=x; }
    }

    // this is the first $l items - make it a one-item loop
    else {$l[0]=x;x.n=x.p=x}

    // return execution to the closure that originally called $l
    $n.call(x)

  }
  
  // return the "waiting" flag
  return $w;
}

od_node

  • redoing this section soon - $n instantiates the $l objects as closures in the current closure (or root if none specified) and executes the init function. Any null valued associations in the context are instantiated as closures and their inits called, and so on recurslively. This creates a prototype-equivalent of a multiple-inheritence OO system.


This is the last of the five domains that declare global functions ($d, $c, $j, $l and $n) and that are read in as a JavaScript code fragment. Recall that these fragments are inserted directly into the body of the HTML document, and that the insertion is initially done by the od_recv function in the Perl server. All functionality from here on will be retrieved using $l so that it becomes part of the relational node structure.

The $n global function defined in this od_node fragment, is used to pass executional focus to a node in the $l loop and to allow that focus to flow in accord with the nodal relationships. The $n function allows this execution to be formed into hierarchical continuations that can execute smoothly regardless whether some operations must wait before execution can continue, such as when a node is requested that needs to be retrieved from the network, or when a user's response is needed.

After od_node is executed, all further program execution will be controlled by the relationships between nodes. It does this by treating a node as a reducible recursive queue.

If the $n function is called without any node name being specified, then by default, the od_root node, which begins initialising the nodal aspect of the application, is used.

If some executing code requests an object using $n and the object is already loaded, the reference is returned and the function execution continues on. However, if the object is not loaded, then the function returns null and execution should stop and any calling function will return null upon receiving a null - these nulls propagate all the way back to ensure a clean halt to the entire executing reduction structure. The same thread will continue again when the missing function has loaded because the the $n will ensure that the context from which it executed will be re established when the requested node has been loaded.

So in addition to defining $n, od_node's job is also to define the program continuation mechanism using relationships between nodes. Each node that gets focus may have an associated function to execute which will then be called with this set to that node entry within the $j structure. Functions must not rely on local scope across calls to $n because execution can be broken at that point by the requested object needing to be loaded.

The main purpose of od_node is basically to create a jQuery event-based version of the nodal reduction recursive toroidal execution algorithm developed in 2005. Here's the main reduction function written in C that od_node will be replicating. 

// Moves "this" to node's focus if exists then rotates loop and executes/reduces the focus item
void nodeReduce() {
    if(this = nodeLoopRotate(parent = this)) {        // Move "this" to the focus in the node's loop and rotate
        nodeSetValue(this, nodePARENT, parent);       // Update the parent association
        if(code = *nodeState(this, nodeCODE)) code(); // nodeCODE value is a pointer-index, execute as function-reference if non-zero
    }
}

After od_node has defined the $n function, it then calls it with no parameter so that the default node is retrieved and executed. 

$n=function(n,f){

  // todo

}
$n()

od_root

This is the default node that's retrieved and executed by the $n global function if no parameter was supplied to it. The job of od_root is to initialise the nodal aspect of the Single Page Application environment.

od_view

This represents which is a view from within the context of the node specified in the hash fragment of the current URL (or to put it in more common terms: it renders the website at the current location). The od_view node also hooks an event handler into the hash-changed event to cause the page to re-render in the new context.

od_view must also take the next alphanumeric portion of the hash-fragment and call the corresponding method in the node's closure, which in turn interprets any remaining part of the hash-fragment after the method name. 

...todo...

// NOTE - view is a node so it needs to be in JSON format specifying its relations and pointing to another $j item for its code content

code:

h=window.location.hash.substr(1))

// process node part of fragment

// process method part of fragment

$(window).bind("hashchange",$n('view');

Nodal code conventions

The idea of this system is to try and set up a simple and familiar environment for code-execution by making the context in which a node's code executes a standard JavaScript closure, but some conventions are unavoidable since the code needs to work in a way that supports the persistence and continuation aspects of the nodal environment.

The first important issue is that continuation occurs by re-executing the same closure's code function which will execute from the start, not at the point it left off last time. This means that all the effects that the function has on its closure should be based on assessing the state of the properties in that closure, not on the local function scope which will not persist through a continuation. In simple terms, this just means before doing anything, check the environment (the closure referred to by this) to see what needs to be done.

The second important issue is that referring to other nodes is always done through the $l global function, and must be written in such a way as to return immediately should the unique $w (wait state) be returned instead of an object (a reference to the requested node's closure). Here's how this might be done for a retrieving a single node, which will result either in x being the new node reference, or bailing with a wait state for later continuation. 

if( ( x = $l(n) ) == $w ) { return $w }  // we can't just do "x = $l(n)", we have to test for a $w result

See also

Retrieved from "https://organicdesign.nz/wiki/index.php?title=Namecoin_SPA&oldid=119899"