Scuttlebutt
Secure Scuttlebutt (SSB) is a peer-to-peer append-only messaging system on top of which a variety networking applications can be built such as the Patchwork social network. Scuttlebutt is able to operate offline and then sync when in the presence of another peer making it perfect for meshing environments, and even supports synchronisation via removable media to work as a "sneakernet". The word "scuttlebutt" is the equivalent of "watercooler" in an office, but applies to boats and ships.
 | Scuttlebutt, aims to provide a platform that is wholly modular, increasingly secure, and brilliantly inspired in order to tackle some of the roots of these problems and make the internet truly ungovernable. |
| — Emmi Bevensee |
Contents
Pubs
Pubs are permenantly connected peers in the Scuttlebutt network with the job of routing messages and storing them while recipients are offline. The name "pub" was used since they play a similar role to pubs in villages. Once you have scuttlebot running (see next section) you can configure it to be a pub simply by creating invites to it. see how to set up a pub for more detail.
A list of some of the pubs can be found here, and I set up ssb.organicdesign.pub to learn more about the process :-)
Updating the pub Docker image
First pull the latest images:
docker pull ahdinosaur/healer
docker pull ahdinosaur/ssb-pubThen if an upgrade happened, first get the list of currently running containers and their images with docker ps, stop the current sbot container with docker stop <comtainer-id>, delete the container with docker rm <container-id> then delete the old image that container was using with docker rmi <image-id>. Then you can start the new image using the docker run command shown below, or using your create-sbot script which contains that command.
Scuttlebot
The easiest way to use Scuttlebutt is by running an end-user application like the Patchwork desktop client, or the Manyverse Android client, which will come pre-packaged with the necessary dependencies. But to run it on a server you need to install the scuttlebot (sbot) service.
If you want safely to run it on an exiting server and avoid version conflicts or other dependency issues, you can use the Docker image as described here. The default port that the pub will run on is 8008. To run it natively you can install the latest version of nodejs or use an existing version if you have one.
cd ~
sudo apt-get install autotools-dev automake
git clone https://github.com/ssbc/scuttlebot.git
cd scuttlebot
npm i
sudo ln -s ~/scuttlebot/bin.js /usr/bin/sbot
sbot server &
sbot whoamisbot commands
If you're running the Docker image, you can't call the 'sbot daemon directly, you call it as follows instead (you may want to also add a cpu limit as the container can get quite hungry, the following adds an option limiting it to 25%):
To start it:
docker run -d --cpu-quota 25000 --name sbot -v /home/USER/ssb-pub-data/:/home/node/.ssb/ -e ssb_host="YOUR-DOMAIN" -p 8008:8008 --restart unless-stopped --memory "800m" ahdinosaur/ssb-pubNote: it's a good idea to put this command into a create-sbot script so you can run it easily after upgrades etc.
To be able to call the sbot command, you can create a script to call the Docker image with the passed arguments, e.g.
#!/bin/bash
docker exec -it sbot sbot $*
Find the pubs Scuttlebutt ID:
sbot whoami
Create an invitation to your pub (the number is how many times it can be used):
sbot invite.create 1Note: To allow a web page to get invitations when sbot is in a Docker container, the page must NOT have the -it (interactive terminal) part in the command.
Give your pub a name:
sbot publish --type about --about @2mIg...5519 --name foobar.comSee more on updating profiles such as adding an image here, note that you can also add a --description field.
Posts:
sbot publish --type post --text "This is a test!"
To block someone from the pub, first unfollow them then block:
sbot publish --type contact --contact @...5519 --no-following
sbot publish --type contact --contact @...5519 --blockingPlugins
Scuttlebot is extendable via a plugins model. Plugins are managed using the plugins commands in a running scuttlebot, e.g. to add the capability to send private encrypted messages, use the following:
sbot plugins.install ssb-privateJavaScript
To access the full functionality of sbot, commands need to be called from JavaScript. This is simply a matter of requiring the ssb-client, for example to send a private message (which can't be done from the command-line) you can use the following basic script which resides in the scuttlebot directory:
#!/usr/bin/env node
var ssbClient = require('ssb-client')
// Creates a client at localhost:8080, using key found at ~/.ssb/secret
ssbClient(function (err, sbot) {
if (err) throw err
// publish a private message to me
sbot.private.publish({ type: 'post', text: 'My First Private Post!' }, ['@WeZBYERzjvfOVlgU7vMz7RSheqe4WaGEd87ewbMvnvQ=.ed25519'], function (err, msg) {
console.log(msg);
})
// Stop the client
sbot.close();
});sbot troubleshooting
Node user: The Docker container seems to require some hard-wired paths involving the "node" UNIX account. I installed it in my own home directory as myself, and things kept failing due to missing files in /home/node, so I just made a symlink from /home/node to my own user dir and then things worked.
Calling sbot from web pages: Again this only affects Docker users, the recommended command to call sbot shown in the installation guide uses the -it (interactive terminal and TTY) options, but these will only work when you are running it from a shell and need to be removed if calling it from a web page (or a background task or something).
Dealing with problem plugins: You might install a plugin (with sbot plugins.install <PLUGIN>), but it turns out to have errors on startup which means the container can't start so you can't call sbot again to remove the plugin! You can edit the plugins manually since they are persistent data outside of the container which is transient. Plugins are listed in the config file in the root of the ssb data directory. You can nuke the container, remove the plugin from the list or set it to false, then run a new container.
Pruning the blobs directory: The main space consumed is the blobs, which are just a non-essential cache of files, these can be pruned by size or age without causing any problems. old-files-purge.pl is a script that allows the recursive nuking of files in a directory structure older than a specified age.
Differences between Scuttlebutt and...
Retroshare has the features all inclusive in the application, requires explicit key sharing between people in order to exchange data, does not use rumor message passing, and does not use pubs as an on-ramp (or slip road) to the network. This was a few years ago, but the process of creating and sharing your key with another person in order to share things with them was far more difficult than using SSB. Also according to the outdated wiki, Retroshare does not support IPv6.
Projects based on ssb protocol
- LocalNative.app - a note/bookmark taking tool to save your notes in a local sqlite database, synced with other devices via ssb
- Bluetooth sync for Manyverse
Backup and restore
The most important file to back up in order to preserve your identity is Scuttlebutt is the secret file which is the private key corresponding to your public ID. But with this alone it's difficult to rejoin the network from scratch, for example if you try rejoining via a pub you'll find that after a few hours of synchronisation, all of your own historical feed and follows are missing. A more reliable way is to back up both the secret and the conn.json file, this should retrieve all the information. You can use a conn.json from the past or even one from one of your contacts and that should be sufficient to get you connected back into your social graph where you can retrieve all your information as people come and go offline.
Protocol
The documentation for the SSB protocol is very clear and easy to understand. There is also the handbook for more information if needed.
Reference formats:
- @ - Signifies a public key meaning the string is a user or pub.
- % - Signifies a message - it is a string which contains the hash of the message.
- & - Signifies a blob - it is a string which contains the hash of the blob's content.
Messages in a feed form an append only log, each entry refers to the previous (with the first pointing to null) by it's id. The id of a message is just a hash of a message including the signature.
Since the messages and blobs are referenced by ID they are immutable.
Message ID's
Computing message ID's can be difficult to start with so here are some examples:
You can compute a message id in JavaScript like this:
const sha256 = require("crypto-js/sha256");
const Base64 = require("crypto-js/enc-base64");
const hashDigest = sha256(
`{
"previous": "%XphMUkWQtomKjXQvFGfsGYpt69sgEY7Y4Vou9cEuJho=.sha256",
"author": "@FCX/tsDLpubCPKKfIrw4gc+SQkHcaD17s7GI6i/ziWY=.ed25519",
"sequence": 2,
"timestamp": 1514517078157,
"hash": "sha256",
"content": {
"type": "post",
"text": "Second post!"
},
"signature": "z7W1ERg9UYZjNfE72ZwEuJF79khG+eOHWFp6iF+KLuSrw8Lqa6IousK4cCn9T5qFa8E14GVek4cAMmMbjqDnAg==.sig.ed25519"
}`
);
const hmacDigest = Base64.stringify(hashDigest);
console.log(hmacDigest);You can compute a message id in python like this:
from hashlib import sha256
from base64 import b64encode
b64encode(sha256("STRINGIFIED JSON".encode('utf-8')).digest()).decode('ascii')Extending
The easiest way to make an application on scuttlebutt is to use ssb-client. If you want to extend the functionality you can publish messages of types that aren't used - e.g. the chess game uses the type "chess_invite". All of these types are stored in your feed. if you want to write to another feed you need to make another key pair.
