Install a new Pi

We now distributing Raspberry Pis around the world on participating user's LANs for our backup and file hosting solution rather then relying on corporate cloud solutions. The Pi's currently have two main purposes, first to receive backups of our various sites from our servers, and secondly some of them run Nextclouds so that they can host files to be shared with users, and these files can be synced with other Pi's running Nextclouds and with user's local devices. This article covers the installation procedure we're using to set up our Pi's.

What you need

• You need a Raspberry Pi of whatever model you like
• an external USB hard drive of good size
• a micro SDcard of at least 16GB
• an official Raspberry Pi power supply (the official power supply is important, because it operates at 5.1v to avoid under-voltage kernel errors which cause a lot of trouble if you use a USB-powered external drive with only 5v no matter how much current you have)
• another Linux computer on the same LAN as the Pi and which can also accept the SDcard so you can use it to write the Raspbian OS onto and to SSH into the Pi from.
• the login and password to access and configure your network's router.
• a LAN cable to connect your Pi to the router (you can use wifi, but cable is best)

SD card setup

First head to raspberrypi.org and download the latest version of Raspbian Lite (the Debian distro built for ARM and without any desktop), and write it onto your SDcard with your favourite disk writing utility. After the OS is written onto it and its mounted you should see a partition called "boot", add an empty file called "ssh" into the root of this partition so your Pi will automatically open SSH access on first boot.

If you want your Pi to automatically connect to the local wifi, also create a new file in the boot partition with name of "wpa_supplicant.conf" with the following content:

ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
network={
	ssid="YOUR_WIFI_SSID"
	psk="YOUR_WIFI_PASSWORD"
}

Power up your Pi!

Connect the Pi to the router and power it up. Then log in to your router and head to the DHCP settings. You should have a list of all the connected clients in there somewhere and you can then set your Pi to a static IP address. It's best to use an address which is outside the main address pool. For example you might set your address pool to 192.168.0.1 to 192.168.0.100 and set your Pi's static address to 192.168.0.254.

If you router doesn't show you a connected client list, or it doesn't give you enough information for you to know which client is the Pi, then you can do the following command on your computer to scan your local subnet to get the necessary information. Note that you need to change this to the subnet/netmask of your own LAN, and you may need to install the nmap package first.

sudo nmap -sP 192.168.1.0/24

You should then get a list of all the hosts found on the LAN along with their hostnames, MAC addresses and local LAN IP addresses, one of them should be recognisable as your Pi, looking something like this:

Host is up (0.21s latency).
MAC Address: A4:50:46:45:D0:C9 (Unknown)
Nmap scan report for raspberrypi (192.168.1.6)

You should now be able to configure the router to give the Pi its static LAN IP address, and ssh into the Pi using "pi" as the username and "raspberry" as the password.

Connectivity

The external IP address of the Pi could change at any time, so we need to set up a domain name for the Pi that stays up to date with the Pi's current external IP. We use namecheap's dynamic-DNS service which is free for any domains registered with them. Absorto has written a python script ddns.py which can be called from the crontab each minute with parameters of sub-domain, domain-name and Namecheap DDNS key respectively. Note that it has to be IPv4, so if the domain resolves to an IPv6 address you should add an entry into /etc/hosts to it's IPv4 value. For example to set your dynamic domain name to backup.example.com add a crontab entry as follows:

*  *    * * *   root      python3 /var/www/ddns/ddns.py backup example.com <NAMECHEAP-DDNS-KEY>

Head to the port-forwarding configuration in your router and open the required ports in the router with rules to direct that incoming net traffic to its local LAN IP address. You definitely want port 22 for ssh, and maybe 80 and 443 if you'll be running a web site.

Note: Sometimes, the router doesn't have any configuration for port-forwarding, or the ISP does not allowed self-hosting of services and blocks incoming requests. In this case you can use ssh to set up reverse-port-forwards for your ports to unused ports on another server you have ssh access to. The ports are forwarded and maintained persistently by calling reverse-ssh-persistent.pl every minute which checks the process list to see if the port forward is active, and if not calls ssh to activate it. Here's an example crontab entry that maintains a reverse-port-forward of the Pi's port 22 onto example.com's port 2222:

*  *    * * *   root      /var/www/tools/reverse-ssh-persistent.pl 22 ssh_user@example.com 2222

External drive and partitioning

Once your external drive's connected you can see what it's details are using lsblk --fs and then add a mount entry for it in /etc/fstab, for example:

NAME        FSTYPE LABEL    UUID                                 MOUNTPOINT
sda                                                              
└─sda1      ext4   PiBackie cda2f440-2267-4d39-9bfb-f5b31fc48a74
mmcblk0                                                          
├─mmcblk0p1 vfat   boot     9304-D9FD                            /boot
└─mmcblk0p2 ext4   rootfs   29075e46-f0d4-44e2-a9e7-55ac02d6e6cc /

Here we can see our external drive on sda1 with it's UUID, and we can see it's already formatted, so we can add an entry to /etc/fstab such as the following. If the drive needs to be formatted, then a good tutorial for doing this from the command line is here, in that case you can create two extra paritions for swap and var as well.

UUID=cda2f440-2267-4d39-9bfb-f5b31fc48a74  /backup  ext4  defaults  0  0

It's best to have the SDcard used as little as possible, because if there is a spontaneous power disconnect the likelihood of the card becoming corrupted is high, and if this happens the system can easily become unbootable requiring the card filesystem to be repaired on another machine. To reduce the chances of this happening we make partitions for swap and /var/log on the external drive, and then mount these in /etc/fstab. Some people even put the entire root partition on the external drive and boot the SDcard in read-only. If it's too late to add new partitions to the drive (as is the case with our Pi), then we can move the swap and the log directories post install as follows.

Raspberry Pi's use the dphys-swapfile utility for swap which uses a file instead of a partition. Change the location to your external drive path (we used /backup/var/swap, and the size is best at two times the RAM. These are changed in the /etc/dphys-swapfile. Then restart dphys-swapfile and check it's the right size with free -h.

# service dphys-swapfile restart
# free -h
              total        used        free      shared  buff/cache   available
Mem:           927M        112M        650M         34M        164M        730M
Swap:          2.0G          0B        2.0G

To move directories you need onto the external drive, we can use Bind Mounts. For example to move /var/log, add a bind mount entry into /etc/fstab, make a copy of the current log directory into the new location and reboot the machine:

The new /etc/fstab line:

/backup/var/log	      /var/log        none    bind		0	0

cp -pR /var/log /backup/var/
reboot

After the machine has rebooted, you can check your new mount:

# mount | grep var
/dev/sda1 on /var/log type ext4 (rw,relatime,data=ordered)

Configuration

Set up all the user accounts you need and set them to use RSA keys for their logins. Any accounts that are set up to receive backups should have their entry in authorized_keys limited so that they can only perform that single action in their designated directory in the backup drive, for example:

command="scp -t /backup/fred",no-port-forwarding,no-pty,no-agent-forwarding,no-X11-forwarding ssh-rsa AAAAB3NzaC1yc2E....

These basic packages were not included by default:

apt install git locate p7zip-full

Nextcloud

The Nextcloud is available via the persistent port forward referred to above at the URL https://dharma.casa:4443.

The Nextcloud has the following dependencies:

apt install mariadb-server-10.1 nginx php7.0-fpm php7.0-cli php7.0-mysqlnd php7.0-gd php7.0-mcrypt php7.0-intl php7.0-curl php7.0-simplexml php7.0-mbstring php7.0-apc php7.0-bcmath php-imagick php7.0-zip

Set the MariaDB data location to the external drive, for example to /backup/mysql in /etc/mysql/mariadb.conf.d/50-server.cnf.

The web-server configuration is single file in nginx.gelek.conf. It refers to a number of SSL certificate files which are just copied directly from the Ligmincha's Nginx SSL configuration since it's running from the same domain just on a different port. These are all *.pem files which are stored in /var/www/ssl on the Pi.

Hotspot

Your Pi can serve a wifi signal so people can use its LAN connection over Wifi. This is very useful if there is no open wifi present at the location.