Talk:OpenCobalt
I wanted to gather these notes somewhere for posterity. I'm following up on some of the very good ideas gleaned from these threads, while cutting down on the reading friction. --Infomaniac 15:30, 24 July 2011 (PDT)
P2P CMS/PM system with SeaSide and TeaTime
Hi, I'm part of a development team who are helping an organisation to architect a free CMS based project-management system that they want to work in a P2P network rather than using a centralised web-server.
We've researched existing CMS's such as Plone to see if they could be modified to operate on top of a DHT but found that they rely too heavily on querying methodologies that are incompatible with the P2P paradigm.
I talked to the OpenCobalt developers and they were very positive about the feasibility of the idea and gave me a lot of good advice and links to check out including Seaside which seems like it could the ideal technology to build our system in.
We have a specific application in mind that we'd like to develop which is a project-management/workflow environment running in a CMS with some other standard tools such as wiki/blog, but rather than a web-server we'd be using a local P2P app as the backend. I'm wondering what you guys, the Seaside developers think of the idea of extracting the TeaTime P2P aspect of OpenCobalt and running Seaside on it so we could build P2P browser-based applications?
We have a good budget available for this and will be developing it as a completely free open source system, so we'd also like to hear from developers who may be interested in working on the project too.
Thanks a lot,
I'm not sure who you talked to but there's really no meaningful definition of the term "P2P" that applies to Teatime as implemented in Hedgehog (which is what Cobalt is built upon). Teatime is a synchronization mechanism for distributed objects, but it can be built with or without using P2P communication. Early Teatime versions used P2P (rather: multi-cast) but the version implemented in Hedgehog (and Cobalt) does not use P2P communication, for various reasons.
From what I've seen in the Cobalt discussions [1,2], I would agree with the person who mentioned the replication features of CouchDB or similar.
It looks like this would be a better and more straightforward fit with what you've been describing in that discussion.
- http://groups.google.com/group/opencobalt/browse_thread/thread/ae071311a099be9a
- http://groups.google.com/group/opencobalt/browse_thread/thread/9829a8971570ba5c
maybe P2P is too general a term, and I may have misunderstood the specifics of what components do what in the Cobalt system. Here's more information on what we're wanting to develop:
The users will be running a p2p application which forms a DHT from all the currently online instances. The CMS/Project management application is used via the browser like a normal web-application, but the address of it resolves to localhost and uses the local p2p application is it's backend "web-server" and the DHT as the persistent storage layer with local caching as necessary etc.
The users then go to the URL of the project management system
Croquet/Cobalt will not be helping you with your DHT work.
Unfortunately, the Croquet site is down right now (we're working on getting it back up) otherwise I'd add a link to our Hedgehog presentation which might clarify what Croquet is and what it isn't.
 | The users will be running a p2p application which forms a DHT from all the currently online instances. The CMS/Project management application is used via the browser like a normal web-application, but the address of it resolves to localhost and uses the local p2p application is it's backend "web-server" and the DHT as the persistent storage layer with local caching as necessary etc. That part sounds doable with Seaside. Given you can make the resolv.conf hacks working. |
| — Aran |
Our Very Good Reason (tm) for building a p2p serverless system is because our organisation (Organic Design) and some other associated projects we've been working with are very passionate about the "bottom up" philosophy.
We're particularly interested in building a network that includes content management and project management tools that work in a mesh-like way where it all continues to operate properly within small regions when their connectivity to the larger network breaks down.
We believe its important for communities to have access to such independent systems, and it's becoming even more important as more communities are pushed below the poverty line and entire nation's economies are collapsing.
We've connected with other like-minded organisations that want the same things and together we now have some funds to develop it.
Currently we're in the process of specifying the application clearly for developers, and asking around on lists such as this one to find what technology and frameworks would best suit the idea.
Using OpenCobalt as a P2P browser-based application framework (1)
I'm part of a development team who are helping an organisation to architect a CMS based project that they want to work in a P2P network rather than using a centralised web-server.
We've researched existing CMS's such as Plone to see if they could be modified to operate on top of a DHT but found that they rely too heavily on querying methodologies that are incompatible with the P2P paradigm.
I realise that Cobalt is really intended as a virtual world system, but it seems that it has a lot of the P2P applicational functionality in place that could be developed to serve content to a local standard browser.
We have a specific application in mind that we like to develop which is a project-management/workflow environment running in a CMS with some other standard tools such as wiki/blog, but rather than a web-server we'd be using a local P2P app as the backend. I'm wondering what you guys, the OpenCobalt developers, think of the practicalities of this idea?
We have a good budget available for this and will be developing it as a completely free open source component, so we'd also like to hear from developers who may be interested in working on the project too.
The P2P part of Open Cobalt and Croquet is the TeaTime system.
The idea of TeaTime is that the same code is executed in the same order on all clients, giving the perception that everything is happening at the same time. This is useful for syncing clients in a 3D simulation. There has been discussion about separating TeaTime out as an independent feature. I'm not sure how much progress has been done on that.
I'm not quite sure what features you intend to implement as P2P
P2P can mean that clients store a repository of resources that all other clients can access, like BitTorrent.
It can also mean that clients see and manipulate what's on each other's screen simultaneously, like a whiteboard.
P2P also add the added concern as to how quickly data is accessed from one client by another.
So issues of how large a cache is available on the hard drives and how fast/slow are the connections between clients. One also must consider how often new clients come into the network and when older data stores go "off grid". Would one's client's screen/web page need data from a hundred different other clients? Also, are clients behind corporate firewalls which they do not control? If so, tunneling is a complex issue that Open Cobalt does not address well and is only tackeled by corporations with large resources like Skype.
So, it's hard to make a recommendation based what you've said. NoSQL backends (like RIAK or MongoDB) as synced storage for normal web apps or native OS apps can provide the syncing of user created data needed if clients don't need to see the same thing at exactly the same time. Think of Ubuntu One. RIAK is more for syncing servers with large bandwidth in-between and rare interruptions, but can still be used for peer clients.
TeaTime is more for the seeing or calculating "the same thing at the same time" applications. And even then, only dozens of clients seeing the same thing, not thousands of clients.
That said, you might still consider using the Smalltalk web server Seaside, with the Pier CMS, and data definition Magritte on the Squeak or Pharo dialects of Smalltalk. The Seaside webserver is so easy to install on a client's computer once setup, it's almost like just adding an app. Then, add the Smalltalk MongoDB drivers and MongoDB, for larger amounts of data or something like memcache or HTTP (P2P) or WebSockets for smaller amounts.
Best wishes for a successful project. :)
I'm an open cobalt wannabe user that is exploring ways to use it. your e-mail caught my attention and i want to share the OpenACS[1] with it's learning (http://dotlrn.org) and business[2]) packages that may be worth looking into for what you are doing.
I should also mention, playing to Open Cobalt's strengths, if your clients can really use the memory aids of 3D immersion and visually placing complex ideas near each other for comparison, Open Cobalt can leverage all the aforementioned technologies as well.
I remember there is a demo in Croquet that shows a dynamic spreadsheet, who can help me in finding the classes that builds that?
TSpreadSheetTxtr and the other classes in it's category. We don't use it in Cobalt at present, check some Open Croquet versions for usage info. It doesn't look very complex though.
Using OpenCobalt as a P2P browser-based application framework (2)
Thanks for your interest. What you suggest seems very interesting and appropriate to Open Cobalt and the entire Squeak community. Open Cobalt can provide the P2P and virtual world framework. Seaside or Aida has the dynamic web framework. Pier is a CMS system. Magma is an object database. And there is more. I believe the Squeak community has the expertise and the necessary software components to help you.
How can we discuss more of the specifics that you wish to have or accomplish?
All the best, Aik-Siong Koh
Thanks so much for the rapid responses and useful links :-)
Here's some more detail of our project:
What we're developing is best described as a web site since it will run in a normal browser and appear to be a normal kind of wiki/blog site.
It doesn't need to handle enormous traffic on any one article like wikipedia would because it's more group-centric to help small organisations to collaborate on content, make decisions together and do basic project management. In fact we'd probably be enforcing some kind of splitting of large traffic nodes into smaller ones, for example dividing large organisations into smaller departments to ensure that each portal of activity always has a practical number of active members.
One of the parts of this system is an interface allowing organisation's members to collaborate on their own applicational requirements by supplying an interface for creating and maintaining collections of workflows and forms and conditions for them to move between roles.
The reason we won't be extending an existing system like Drupal or Plone is that we have a strict requirement to do this in a serverless way, and it looks like this requirement needs to be designed for right from the ground up since real-time queries on a global data-set would choke the network.
To deal with this we're ensuring that the architecture avoids network-wide queries/searches. Any such queries, for example a list of the 5 newest organisations in the network, would be handled as a directory (in the p2p space) that's updated when an organisation is created, rather than a query being broadcast around the network when a peer needs the list. With this general "directory not search" approach to the architecture we'd be minimising the expectations we have for the p2p network's performance.
We also intend that the system be used by many organisations that have very low quality or limited connectivity, so we'd be designing in long-term eventual consistency and edit-conflict workflow mechanisms.
I'm working on the architecture and have some idea about how to achieve this, but we're looking for a lead developer with good P2P experience to guide us in the core technology decisions. For example maybe we don't need all the capabilities of TeaTime and a simple DHT might suffice, or maybe a standard CMS can be modified more easily than we think.
We still have a lot of research to do, and all your comments and advice are very much appreciated :-) Thanks a lot, Aran
Sounds like CouchDB's replication feature could provide all the P2P horsepower your project needs. See the section on "Distributed Updates and Replication" in http://couchdb.apache.org/docs/overview.html .
Adding Seaside for web UI and Pier CMS and/or Gjallar collaboration to a CouchDB back-end could be a comprehensive solution. To get an idea of what's possible with workflows, forms and roles, check out Gjallar's features here: http://www.gjallar.se/gjallar/7. Thanks, Patrick
