Install a new Pi

Install a new Pi Organic Design procedure

We now distributing Raspberry Pis around the world on participating user's LANs for our backup and file hosting solution rather then relying on corporate cloud solutions. The Pi's currently have two main purposes, first to receive backups of our various sites from our servers, and secondly some of them run Nextclouds so that they can host files to be shared with users, and these files can be synced with other Pi's running Nextclouds and with user's local devices. This article covers the installation procedure we're using to set up our Pi's.

What you need

• You need a Raspberry Pi of whatever model you like
• an external USB hard drive of good size
• a micro SDcard of at least 16GB
• an official Raspberry Pi power supply (the official power supply is important, because it operates at 5.1v to avoid under-voltage kernel errors which cause a lot of trouble if you use a USB-powered external drive with only 5v no matter how much current you have)
• another Linux computer on the same LAN as the Pi and which can also accept the SDcard so you can use it to write the Raspbian OS onto and to SSH into the Pi from.
• the login and password to access and configure your network's router.
• a LAN cable to connect your Pi to the router (you can use wifi, but cable is best)

SD card setup

First head to raspberrypi.org and download the latest version of Raspbian Lite (the Debian distro built for ARM and without any desktop), and write it onto your SDcard with your favourite disk writing utility. After the OS is written onto it and its mounted you should see a partition called "boot", add an empty file called "ssh" into the root of this partition so your Pi will automatically open SSH access on first boot.

If you want your Pi to automatically connect to the local wifi, also create a new file in the boot partition with name of "wpa_supplicant.conf" with the following content:

ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
network={
	ssid="YOUR_WIFI_SSID"
	psk="YOUR_WIFI_PASSWORD"
}

Accessing the Pi

Connect the Pi to the router and power it up. Then log in to your router and head to the DHCP settings. You should have a list of all the connected clients in there somewhere and you can then set your Pi to a static IP address. It's best to use an address which is outside the main address pool. For example you might set your address pool to 192.168.0.1 to 192.168.0.100 and set your Pi's static address to 192.168.0.254.

If you router doesn't show you a connected client list, or it doesn't give you enough information for you to know which client is the Pi, then you can do the following command on your computer to scan your local subnet to get the necessary information. Note that you need to change this to the subnet/netmask of your own LAN, and you may need to install the nmap package first.

sudo nmap -sP 192.168.1.0/24

You should then get a list of all the hosts found on the LAN along with their hostnames, MAC addresses and local LAN IP addresses, one of them should be recognisable as your Pi, looking something like the following and gives you the info you need to configure the router to give the Pi its static LAN IP address if you couldn't do it yet.

Host is up (0.21s latency).
MAC Address: A4:50:46:45:D0:C9 (Unknown)
Nmap scan report for raspberrypi (192.168.1.6)

Now you can ssh into the Pi using "pi" as the username and "raspberry" as the password. Note that the IP may not have changed yet, so you may need to log in with the old one, and then run dhclient -r to release the old address and obtain the new one.

Configuration

Bring the system up to date and install a few other packages we need:

apt update
apt upgrade
apt install git p7zip-full unzip

Set up all the user accounts you need and set them to use RSA keys for their logins by adding their public key to the ~/.ssh/authorized_keys file (remember that this file must be owned by the user and should be of mode 644). Give the necessary users root access by adding them to the sudo group or adding them in visudo. You may want to disable password logins and remove the original "pi" account as well, or at least give it a decent password. The main settings in /etc/ssh/sshd_config for this are:

PermitRootLogin no
RSAAuthentication yes
PubkeyAuthentication yes
PasswordAuthentication no

Note: After you made the changes to sshd_config and restarted ssh, ensure that you can log in as a new user and gain root access before exiting out of your current root session or you may loose access to the Pi!

Any accounts that are set up to receive backups should have their entry in authorized_keys limited so that they can only perform that single action in their designated directory in the backup drive, for example:

command="scp -t /backup/fred",no-port-forwarding,no-pty,no-agent-forwarding,no-X11-forwarding ssh-rsa AAAAB3NzaC1yc2E....

Port forwarding

Head to the port-forwarding configuration in your router and open the required ports in the router with rules to direct that incoming net traffic to its local LAN IP address. You definitely want port 22 for ssh, and maybe 80 and 443 if you'll be running a web site. Sometimes, the router doesn't have any configuration for port-forwarding, or the ISP does not allowed self-hosting of services and blocks incoming requests, if that's the case you can get around this by using ssh to do reverse-port-forwarding instead. If your port-forwards have worked fine, skip to the next section.

To set up reverse-port-forwards for your ports you need to use some unused ports on another server you have ssh access to. The ports can be forwarded and maintained persistently by calling reverse-ssh-persistent.pl script every minute which checks the process list to see if the port forward is active, and if not calls ssh to activate it. Here's an example crontab entry that maintains a reverse-port-forward of the Pi's port 22 onto example.com's port 2222:

*  *    * * *   root      /var/www/tools/reverse-ssh-persistent.pl 22 ssh_user@example.com 2222

Note that there are a couple of configuration changes required in /etc/ssh/sshd_config on the remote server, to open up our forwarded ports for external access, and to prevent the ssh tunnels from closing after an idle time:

GatewayPorts yes
ClientAliveInterval 120

Now you can ssh into your Pi any time using the new open port on the remote server:

ssh -p 2222 ssh_user@example.com

Dynamic DNS

The external IP address of the Pi could change at any time, so we need to set up a domain name for the Pi that stays up to date with the Pi's current external IP. But note that if you are in the situation where you needed to set up reverse-port-forwarding with ssh, then there's no need for you to set up dynamic DNS because you're only ever connecting with the forwarded ports on your designated remote host.

We use namecheap's dynamic-DNS service which is free for any domains registered with them. Absorto has written a python script ddns.py which you can clone as follows:

cd /var/www
git clone https://gitlab.com/absorto/ddns.git

It can then be called from the crontab each minute with parameters of sub-domain, domain-name and Namecheap DDNS key respectively. For example to set your dynamic domain name to backup.example.com add a crontab entry as follows:

*  *    * * *   root      python3 /var/www/ddns/ddns.py backup example.com <NAMECHEAP-DDNS-KEY>

Note that this script uses a URL at organicdesign.nz to determine the current external IP address of the Pi, and we need the returned address to be IPv4 because the Namecheap dynamic DNS system does not currently support IPv6. Absorto's script does not currently force the request to IPv4, so we need to add an entry for it into /etc/hosts with IPv4 value:

213.5.71.227    organicdesign.nz

External drive and partitioning

Once your external drive's connected you can see what it's details are using lsblk --fs and then add a mount entry for it in /etc/fstab, for example:

NAME        FSTYPE LABEL    UUID                                 MOUNTPOINT
sda                                                              
└─sda1      ext4   PiBackie cda2f440-2267-4d39-9bfb-f5b31fc48a74
mmcblk0                                                          
├─mmcblk0p1 vfat   boot     9304-D9FD                            /boot
└─mmcblk0p2 ext4   rootfs   29075e46-f0d4-44e2-a9e7-55ac02d6e6cc /

Here we can see our external drive on sda1 with it's UUID, and we can see it's already formatted, so we can add an entry to /etc/fstab such as the following where we're mounting it to /backup (the directory you mount it to must exist). If the drive needs to be formatted, then a good tutorial for doing this from the command line is here, in that case you can create two extra paritions for swap and var as well.

UUID=cda2f440-2267-4d39-9bfb-f5b31fc48a74  /backup  ext4  defaults  0  0

It's best to have the SDcard used as little as possible, because if there is a spontaneous power disconnect the likelihood of the card becoming corrupted is high, and if this happens the system can easily become unbootable requiring the card filesystem to be repaired on another machine. To reduce the chances of this happening we make partitions for swap and /var/log on the external drive, and then mount these in /etc/fstab. Some people even put the entire root partition on the external drive and boot the SDcard in read-only. If it's too late to add new partitions to the drive (as is the case with our Pi), then we can move the swap and the log directories post install as follows.

Raspberry Pi's use the dphys-swapfile utility for swap which uses a file instead of a partition. Change the location to your external drive path (we used /backup/var/swap, and the size is best at two times the RAM. These are changed in the /etc/dphys-swapfile. Then restart dphys-swapfile and check it's the right size with free -h.

# service dphys-swapfile restart
# free -h
              total        used        free      shared  buff/cache   available
Mem:           927M        112M        650M         34M        164M        730M
Swap:          2.0G          0B        2.0G

To move directories you need onto the external drive, we can use Bind Mounts. For example to move /var/log, add a bind mount entry into /etc/fstab, make a copy of the current log directory into the new location and reboot the machine:

The new /etc/fstab line:

/backup/var/log	      /var/log        none    bind		0	0

cp -pR /var/log /backup/var/
reboot

After the machine has rebooted, you can check your new mount:

# mount | grep var
/dev/sda1 on /var/log type ext4 (rw,relatime,data=ordered)

Nextcloud

The Nextcloud has the following dependencies:

apt install mariadb-server-10.1 nginx php7.0-fpm php7.0-cli php7.0-mysqlnd php7.0-gd php7.0-mcrypt php7.0-intl php7.0-curl php7.0-simplexml php7.0-mbstring php7.0-apc php7.0-bcmath php-imagick php7.0-zip

Note: It's best to set the MariaDB data location to the external drive, for example to /backup/mysql in /etc/mysql/mariadb.conf.d/50-server.cnf.

Then you can create a database and database user for your Nextcloud.

Download the PHP files onto the server, unpack them into /var/www/nextcloud and set ownership of the structure to www-data. To configure the web-server, you can just set the etc/nginx/sites-available/default config to your Nextcloud configuration. An example of a single-file web-server configuration for a Nextcloud on a Pi configured like this is here. You'll need to have SSL certificates sorted out which can be done with LetsEncrypt. Then you can go through the web-installer. Set the data directory to a location on the external drive such as /backup/nextcloud.

Hotspot

Your Pi can serve a wifi signal so people can use its LAN connection over Wifi. This is very useful if there is no open wifi present at the location. Simply run through the Set up a wifi hotspot procedure including the section to automatically start it on boot.