From Organic Design

Matrix is an open source project that publishes the Matrix open standard for secure, decentralised, real-time communication, and its Apache licensed reference implementations. Maintained by the non-profit Foundation who aim to create an open platform which is as independent, vibrant and evolving as the Web itself... but for communication. As of June 2019, Matrix is out of beta, and the protocol is fully suitable for production usage.

Docker installation

First you'll need to configure your web-server as a reverse proxy from SSL ports 443 and 8448 to the internal non-SSL port 8008. This is the default Matrix port for unsecured HTTP traffic, so that a reverse proxy needs to be set up from your web-server to handle the HTTPS side of things on exposing the default Matrix HTTPS port of 8448 to the public that connects to the the internal HTTP port on 8008. Also there needs to be a connection from port 443, see the official reverse proxy notes for details about the reverse proxy setup.

We'll be using PostgreSQL instead of the default SQLite database, which means that we'll need to use docker-compose. So first create a directory for the configuration and data and then put a docker-compose.yml file in it with the following content which will create persistent volumes to put the synapse data in data/system and the PostgreSQL data in data/postgres.

version: '3'

    restart: unless-stopped
    image: postgres:9.6-alpine
      - POSTGRES_USER=synapse
      - POSTGRES_DB=synapse
     - internal_network
      - ./data/postgres:/var/lib/postgresql/data

    image: matrixdotorg/synapse:latest
    restart: unless-stopped
      - external_network
      - internal_network
      - ""
      - postgres
      - ./data/system:/data

    internal: true

Next, generate a default configuration file for your domain as follows. This will create a new volume with your persistent configuration file in it called homeserver.yaml as well as some keys for your domain.

docker run -it --rm -v "/FULL/PATH/TO/DIR/data/system:/data" -e -e SYNAPSE_REPORT_STATS=no matrixdotorg/synapse:latest generate

Then start the container in the background.

docker-compose up -d

Now we need to create a database with the correct encoding (we may need to drop an initially created one first). So first log in to the PostgreSQL database.

docker exec -it synapse-docker_postgres_1 psql -U synapse

Connect to the postgres database so you can drop synapse, and then create a new synapse database with the correct encoding.

\connect postgres
 OWNER synapse;

Then edit the data/system/homeserver.yaml configuration and add the following to the database section. Note that the database host is postgres not localhost, because it needs to be accessed via the hostname given to the database service defined in the docker-compose.yml file. The database name and database user must also match the environment given to the database service in the docker-compose.yml file.

  name: psycopg2
    user: synapse
    database: synapse
    host: postgres
    cp_min: 5
    cp_max: 10

Then exit out of PostgreSQL, restart the container and set up a user (check the logs to ensure its running):

docker-compose down
docker-compose up -d
docker exec -it synapse-docker_synapse_1 register_new_matrix_user -c /data/homeserver.yaml http://localhost:8008


Apart from backing up the data directory, it's a good idea to back up the database with a proper dump as well:

docker exec synapse-docker_postgres_1 pg_dump -U synapse synapse > dump.pgsql


Simply upgrade the container and start a new instance:

docker-compose pull
docker-compose up -d

Enabling email

Synapse can use email for user password resetting and notification of missed messages. I was unable to figure out how to connect synapse's in-built SMTP sending facility to the local server, it might be a TLS version conflict judging from the messages in the Exim4 log. So for now a workaround is to add the following SMTP service into the docker-compose.yml configuration file:

    image: juanluisbaptiste/postfix:latest
    restart: unless-stopped
      - SMTP_USERNAME=*****
      - SMTP_PASSWORD=*****
      - DEBUG=yes
      - internal_network
      - external_network
      - "/etc/localtime:/etc/localtime:ro"

Note: The SERVER_HOSTNAME must be different from the domain used by the email clients otherwise the relay server will try to perform local delivery on them instead of relaying them to Exim4.

Use the following settings in the data/system/homeserver.yaml configuration's smtp section, and comment out any other authentication settings to use their defaults.

smtp_host: smtp
  notif_from: "Your Friendly %(app)s homeserver <>"
  app_name: Organic Design Matrix
  enable_notifs: true
  notif_template_html: notif_mail.html
  notif_template_text: notif_mail.txt

Finally, you need to add an email address in your settings in the client. In Riot, you need to enter the email address and click "add", then a confirmation email address gets sent. You click the link in the confirmation email, and then after that you click "continue" in the Riot client. If you click "continue" before you confirmed the email address, it will not be added. You can clearly see when it has been correctly added, because it will ask for your account password to confirm the addition and then is shown whenever you enter the settings window and has a big red "remove" button to the right of it. And don't forget to enable email notifications in the notifications tab of Riot's settings!

Change a user password

First you need to create the password hash using the hash_password script. This needs to be downloaded and can run on the server or done locally. you may need to install some missing packages such as bcrypt which can be done with pip.

./hash_password -p "MyNewPassword"

Next you need to log into the postgreSQL docker container.

docker exec -it synapse-docker_postgres_1 psql -U synapse synapse

Then list the users to get the exact user names, then set the password for the relevant user to the hash obtained above.

SELECT * FROM users;
UPDATE users SET password_hash='$2b$12...XlZyHO' WHERE name='';



File structure

The installation files such as res/templates are inside the container in the directory /usr/local/lib/python3.7/site-packages/synapse/, e.g. list them as follows:

docker exec -it synapse-docker_synapse_1 ls /usr/local/lib/python3.7/site-packages/synapse

CORS issues

Check https://YOURDOMAIN/_matrix/client/versions in a browser, it should respond with something like the following:

  "versions": ["r0.0.1", "r0.1.0", "r0.2.0", "r0.3.0", "r0.4.0", "r0.5.0"],
  "unstable_features": {
    "m.id_access_token": true,
    "m.require_identity_server": false,
    "m.separate_add_and_bind": true,
    "org.matrix.label_based_filtering": true,
    "org.matrix.e2e_cross_signing": true,
    "org.matrix.msc2432": true

And it should have the following CORS headers:

access-control-allow-headers  Origin, X-Requested-With, Content-Type, Accept, Authorization
access-control-allow-methods  GET, POST, PUT, DELETE, OPTIONS
access-control-allow-origin   *

Local Synapse Installation

A local installation of synapse can be helpful to test configurations without worrying about doing a proper setup and federation.
Most of these instructions were obtained from the official installation guide.

Docker Installation

Note: this will setup the server name as - you can change this but for a local install it does not really matter.
The configuration needed for the client using this setup is as follows:

	"userId": "",
	"password": "PASSWORD",
	"baseUrl": "http://localhost:8008"


mkdir /data # This is where synapse will mount to.

Install synapse:

docker run -it --rm \
    --mount type=volume,src=synapse-data,dst=/data \
    -e \
    matrixdotorg/synapse:latest generate

Run synapse:

docker run -d --name synapse \
    --mount type=volume,src=synapse-data,dst=/data \
    -p 8008:8008 \

Start synapse (In the event of it stopping, e.g. restart)

docker start /synapse

Remove synapse

docker rm /synapse

Check that synapse is running:

docker logs synapse # check logs
http://localhost:8008/_matrix/static/ # check the page is running


The homeserver.yaml can typically be found at:


Register User

Since user registration is disabled by default, you will need to create one:

docker exec -it synapse register_new_matrix_user http://localhost:8008 -u USERNAME -p PASSWORD -c /data/homeserver.yaml

Obtain user's access token:

curl -XPOST -d '{"type":"m.login.password", "user":"USERNAME", "password":"PASSWORD"}' "http://localhost:8008/_matrix/client/r0/login"

Performing Queries

See the user admin API for more information.


curl --header "Authorization: Bearer <access_token>" <the_rest_of_your_API_request>

Query user example:

curl --header "Authorization: Bearer <access_token>" localhost:8008/_synapse/admin/v2/users/<username>

Create room example:

curl --header "Authorization: Bearer <access_token>" -XPOST -d '{"name":"test"}' "http://localhost:8008/_matrix/client/r0/createRoom"

Create message example:

curl -X PUT \
	-d '{"msgtype": "m.text", "format": "org.matrix.custom.html", "body": "Hello world!"}' \
	--header "Authorization: Bearer <access_token>" \

The Element client

The main messaging client for use with the Matrix protocol is Element (was Riot until mid 2020). This has a very good and mature user interface now, but one important thing to note about it is that the end-to-end encryption is session-based which means that if you log out, you will not have access to any of the messages previously encrypted with the keys of the previous session.

When logging in, you get the option to verify yourself using open sessions you have running on other devices which will transfer all the necessary keys to the current session. But if you don't have any other active sessions, you'll need to use the recovery key (the recovery passphrase method offered by default is insufficient). To do this first click "or use recovery passphrase or key", then in the next window click "use your recovery key" and enter the recovery key that you were asked to back up when you signed in for the first time.


Custom configuration

A custom config.json file can be placed in the root of the data directory (e.g. ~/.config/Element on Linux) and it will be processed after a sign out and full app restart. The configuration options are explained in detail here, and the default configuration can be found here.

Voice/video calls

The element client allows voice or video calls to be made from within any room, it uses a direct encrypted p2p connection or a TURN server for conversations between two people, and a Jitsi server for more people. By default the Jitsi server used will be, but this can be changed by creating a custom config.json file and adding a different domain in the preferredDomain key of the jitsi option. For more detail see this document.


Matrix has been running tests to get Matrix working in P2P system by moving the homeserver onto the client's machine. The standard homeserver (Synapse) was too heavy for this to be feasible so a second generation homeserver of the name Dendrite was created. Dendrite has been compiled into WASM for running in the browser - you can try this for yourself:

P2P Matrix is getting designed in a way that allows communication with the standard federated system and requires no client side changes - you can use your favorite matrix client without any modifications. Bridges and Bots will remain unchanged too. The one thing that I can see changing for the P2P users is application services - since they are made for running on the server they simply can't run if there is no server, however I believe they may have some creative uses client side.


Matrix has two sets of API, one for Client-Server communication and one for Server-Server (federation) communication. Communication is done using JSON over REST making it quite simple and easy to start interacting with Matrix. Clients can implement whatever features they want but servers must implement the minimum required features (however it should implement much more than this), for this reason it is best to put it through some testing like systest or compliment.


Matrix supports the OpenID standard for authentication of users.
Note: this token authentication method only implies existence of the user - the user may not exist if the server behind the domain is rouge so the server should check the existence of that user.
Request token from the client:

curl -X POST \
	--header 'Content-Type: application/json' \
	--header 'Accept: application/json' \
	--header "Authorization: Bearer <YOUR_TOKEN>" \
	-d '{}' \

Sample output:


Once the user has their token it should be passed to the server where the server will make a request like:

curl -X GET \
	--header 'Accept: application/json' \

Which should in turn respond with:


Or if it failed:

  "errcode": "M_UNKNOWN_TOKEN",
  "error": "Access token unknown or expired"

Server Validation

Using this method I believe it is still possible (but complicated) for a non-matix user in control of a domain to validate an account on that domain.
matrix-express-authorization is a basic application implementing this approach.

  1. The user must pass their Matrix ID and OpenID access token to the server.
  2. The server should check that the user exists using the profile api.
  3. Then the server must retrieve the domain from that Matrix ID and send the token to that domain using the OpenID api.
  4. The server must then check the response is the same as the Matrix ID provided.

If this last step successful the user is who they say they are.

Custom User Data

Matrix allows users to have custom user data to be set for the user or the user and room combination.
By default the data is not encrypted on transport.
Other users cannot access or set your user data.
NOTE: custom user data is stored unencrypted in the home-server's database. See this for a solution.

Custom Events

Matrix allows clients to send custom events, by default the existing clients will ignore (since it is not setup to listen for it).
Custom events can be useful for custom client + application service configurations.

A normal event of type message can be sent like so:

curl -X PUT \
	-d '{"msgtype": "m.text", "format": "org.matrix.custom.html", "body": "Hello world!", "formatted_body": "Hello World!"}' \
	--header "Authorization: Bearer <access_token>" \

A custom event can be emitted like so:

curl -X PUT \
	-d '{"msgtype": "m.text", "format": "org.matrix.custom.html", "body": "Hello world!", "formatted_body": "Hello World!"}' \
	--header "Authorization: Bearer <access_token>" \

See Also


Although not required, working with matrix from JS is easiest from the matrix-js-sdk or using matrix-appservice-node.


Events can be listened to like so:

client.on("event type", callback);


To check for messages you need to listen to these three events:

  • Room.timeline - I don't know what triggers this at this stage.
  • room.message - A message event (May be encrypted).
  • Event.decrypted - A event that has been decrypted.

Both Room.timeline and room.message need to be checked if it is indeed a message and that it is not encrypted:

if (event.isEncrypted() || event.getType() !== "")

// Handle event

The room.message event needs to check that it has successfully decrypted it and it is a message.

if (event.isDecryptionFailure() || event.getType() !== "")

// Handle event

Duplicate Events

Sometimes a duplicate event may occur - I think it may be to do with this.

Event History

Event history is a little hard to figure out in the matrix-js-sdk when you don't know what you are looking for, so here is a quick example:

matrix.client.scrollback(room, 30).then(() => {

Enabling Encryption

The matrix client's method setRoomEncryption only sets the room encryption on the client side, an event needs to also be pushed to enable it.

const roomId = "!";
const encryptionOptions = { algorithm: "m.megolm.v1.aes-sha2" };

await client.sendStateEvent(roomId, "", encryptionOptions));
await client.setRoomEncryption(roomId, encryptionOptions));

Verifying Devices

To verify a device it needs to both be marked as known and verified.

await client.setDeviceKnown(userId, deviceId, true)
await client.setDeviceVerified(userId, deviceId, true);

You can verify everyone in a room by getting the members and all their devices.

const roomId = "!";
const room = client.getRoom(roomId);
const members = await room.getEncryptionTargetMembers();

for (const member of members) {
	const devices = client.getStoredDevicesForUser(member.userId);

	for (const device of devices) {
		if (device.isUnverified())
			// Verify the device

Checking Login Status

It's slightly more convoluted than it initially appears to see if the user is successfully logged in but here is the process I have arrived at.

// userId will return anything if the user has attempted a login.
const userId = client.getUserId();

if (!userId)
	return; // No login attempt made.

// user will only return something if the account has permission to access this user's details
const user = client.getUser(userId);

if (!user)
	return; // Not your account.

// User must be logged in if they made a login attempt and can access that user's data.

Bot Examples

See Also

Application Services

Application services are a way to modify the matrix functionality by having an application running alongside the homeserver.

Register Application Service

To register the application service you need to edit the homeserver.yaml file and add the following config (it is commented out so search for app_service_config_files:)

  - "/path/to/appservice/registration.yaml"

If you are on a local client through docker you will want to copy your registration.yaml to /var/lib/docker/volumes/synapse-data/_data/ and add the following config:

  - "/data/registration.yaml"

Then restart the homeserver to apply changes and check the logs to ensure it hasn't thrown an error.


Test that your application service gives a response when running:

curl -X PUT http://localhost:8010/transactions/1?access_token=yourtoken

localhost IPv6 Socket

I had trouble getting an application service running initially the synapse logs were showing me:

INFO - as-recoverer-test-appservice-3 - Error sending request to PUT http://localhost:8010/transactions/1?access_token=<redacted>: ConnectError Cannot assign requested address

The issue was that I had set up the application service to use localhost and synapse was trying to create an IPv6 socket - which my application service was not setup for.
To resolve this I changed localhost to an IPv4 address.

You may also want to check your /etc/hosts file to verify that localhost does not resolve to IPv6 (it wasn't in my case).

Matrix Appservice Node


Note the setId method is not shown in the docs but is required. Also note url is the url of the app service not the homeserver.
Also note that if localhost is used synapse resolve this to an ipv6 socket which may not work.

registration.js (for creating registation.yaml file)

const { AppServiceRegistration } = require("matrix-appservice");

// creating registration files
const reg = new AppServiceRegistration();
reg.setAppServiceUrl("http://<YOUR IP ADDR>:8010");
reg.addRegexPattern("users", "@.*", true);
reg.setProtocols(["exampleservice"]); // For 3PID lookups

Example output:

id: test-appservice
hs_token: 35d98ec4115123751d2dbab8e441e277d7d4e364130f474b59cdbfdb164f94a1
as_token: e83a2ef0add9be4e77ab3b04aedb3df3d1dafdb9bb1b4c4374a169c5524d270d
url: 'http://<YOUR IP ADDR>:8010'
sender_localpart: example-appservice
  - exampleservice
    - exclusive: true
      regex: '@.*'


See also