Difference between revisions of "Install a new server"

From Organic Design wiki
(Domain names: note re localhost)
(Timezone: set to UTC)
(234 intermediate revisions by 4 users not shown)
Line 1: Line 1:
{{procedure
+
<noinclude>{{procedure}}</noinclude>
|description = The [[Install a new server]] procedure defines the steps required to prepare a new computer or [[w:VPS|VPS]] for use as a server for the organisation.
+
== Choose a hosting provider ==
|role = Sysop
+
First an appropriate hosting provider needs to be found, or if running a server in-house, see the [[Configure LAN]] procedure. Some possible points to check out when looking for a server [[hosting]] service apart from just the cost are:
|status = in use
+
*Ease of hardware upgrading - can you upgrade disk/memory/cpu without reinstalling the system?
}}
+
*Contention rate (how many concurrent clients share the hardware if its a VPS)
== Download and install Debian ==
+
*Control panel usefulness (the most important features are rebooting and virtual console access)
If the server has no OS then download and install Debian first. Depending on the kind of access you have to the server, the following links may be of interest here:
+
*OS choices available (up to date Debian or Ubuntu are most important for us)
*[[Debian Conversion]]
+
*Historical downtime statistics
 +
*What jurisdiction are they hosting in and what laws apply? for example can your run hidden services, i2p/tor routers or torrent daemons?
 +
*Do they accept [[Bitcoin]] or [[Crypto-currencies|other crypto-currencies]] for payment?
 +
*What kind of data backup options do they provide?
  
== Setting up the software environment ==
+
== Get reverse DNS set up ==
Ensure that you're using the same package repository for your Debian installation as we are:
+
Any site that sends emails should have reverse DNS correctly configured. Having a reverse DNS correctly set up will help to prevent the site's mails being trashed as spam. Many mail-servers will do a reverse lookup on the sending IP address and ensure it matches the senders specified domain.
{{code|<pre>
 
deb http://ftp.us.debian.org/debian stable main contrib non-free
 
deb http://security.debian.org stable/updates main contrib non-free
 
</pre>}}
 
  
 +
This is not done by the domain registrar, it's done by the company hosting the server (the IP address owner), sometimes they include the ability to set it in the server management interface. If not, raise a support ticket asking them if they can set up a PTR record for the server's IP pointing to your naked domain.
  
Bring the system up to date and install the [[organicdesign-server]] package:
+
You can check the reverse DNS for a domain at [http://www.dnsstuff.com DNSstuff] in their IP section, and you can find out more about what reverse DNS is and why it's important [http://aplawrence.com/Blog/B961.html here].
{{code|<pre>
 
echo "deb http://packages.organicdesign.co.nz main/" >> /etc/apt/sources.list
 
apt-get update
 
apt-get upgrade
 
apt-get install organicdesign-server
 
</pre>}}
 
  
 +
== Set up a caching DNS server ==
 +
{{:Configure DNS}} See [[Configure DNS]] for installation details.
  
If you would like math markup support, also install the following, and see [[Enabling math markup]] for more details.
+
== Download and install Debian or Ubuntu ==
{{code|<pre>
+
If the server has no OS then download and install Debian/Ubuntu first. Depending on the kind of access you have to the server and the kind of media it can accept, the following links may be of interest.
apt-get install dvipng tetex-extra cjk-latex ocaml
+
*[[Debian Conversion]] ''- Change an existing Linux distro into Debian using only SSH access''
</pre>}}
+
*[[Debian installation from memory stick]] ''- This is actually one of the most convenient means of installation even when DVD/CD are available''
  
You will have a functioning server and LAMP environment.
+
== Setting up the OS environment ==
 +
First, bring the system up to date.
 +
<source>
 +
apt update
 +
apt upgrade
 +
</source>
  
== Post install checklist ==
+
Give the server's ''root'' account (and any other accounts that send mail) a friendly name so it looks better in the inbox when it sends mail. Do this by replacing the name "root" in the full-name field in ''/etc/passwd'' as follows:
*/etc/hostname, hostname -F /etc/hostname, /etc/hosts
+
<source>
*tzselect, tzconfig
+
root:x:0:0:{!Organic Design server!}:/root:/bin/bash
*DB info for wikia, webmail, crm
+
</source>
*/etc/ssh/sshd_config
 
*/etc/crontab
 
*/var/www/backup.pl
 
*/var/www and /home structures (should be automatically maintained by adding new server as a peer)
 
*Exim4 (this will need to be configured even for sending mail, see [[Configure mail server]])
 
*Import spamassassin bayesian rules
 
  
== Setting up the Wikia file structure ==
+
'''Note:''' If you want mail from the ''root'' account sent to something other that ''root@yourdomain'' then, set the address in ''/root/.forward''.
Either copy the ''/var/www'' directory structure from an existing wikia installation (excluding the specific content from the ''domains'' and ''wikis'' directories). Alternatively, unpack a recent ''www-yyyy-mm-dd.tgz'' backup into ''/var/www'' and remove specific wiki content.
 
{{code|<pre>
 
7za x www-yyyy-mm-dd-tgz
 
tar -xf www.tar /var
 
</pre>}}
 
  
You will need to adjust the following items in the structure:
+
=== Timezone ===
*/var/www/extensions/wikia.php
+
It's best for a server to run on UTC timezone,
*/var/www/activity.log
+
<source lang="bash">
*/var/www/domains
+
sudo timedatectl set-timezone UTC
*/var/www/wikis
+
</source>
  
== LAMP configuration ==
 
The easiest way to configure this is to copy the ''vhosts'' file which originated from the backup file over the default Apache configuration file, and then make the ''vhosts'' file a symlink pointing at the default configuration as usual. Then adjust the file to the wikia's specific needs.
 
{{code|<pre>
 
mysqladmin -u root -p password *******
 
mv /var/www/vhosts /etc/apache2/sites-available/default
 
ln /etc/apache2/sites-available/default vhosts
 
a2enmod ssl
 
a2enmod rewrite
 
</pre>}}
 
  
Activate the slow query log for mysqld. Make sure this directive is uncommented.
+
Getting the following warning plus a bunch of others whenever Perl scripts run?
{{code|<pre>
+
<source>
log_slow_queries = /var/log/mysql/mysql-slow.log
+
perl: warning: Falling back to the standard locale ("C").
</pre>}}
+
</source>
  
The differences to the default ''php.ini'' file in our servers are as follows:
 
{{code|<pre>
 
max_execution_time = 300
 
memory_limit = 64M
 
log_errors = On
 
error_log = syslog
 
post_max_size = 100M
 
upload_max_filesize = 100M
 
extension=domxml
 
extension=fileinfo.so
 
</pre>}}
 
  
 +
Configure the locales and tick the time zones you'd like to have available on the system, make sure that ''en_US.8859-1'' and ''en_US.UTF-8'' are selected because it seems that some programs expect them to exist. Also make sure that all locales ate included that were mentioned anywhere within the warning messages.
 +
<source lang="bash">
 +
dpkg-reconfigure locales
 +
</source>
 +
 +
 +
If you're still getting the problem, you can explicitly set the associated variables to one of the locales you made available by appending entries to your ''~/.bashrc'' as in the following example, you may need to log out and back in for these changes to take effect.
 +
<source lang="bash">
 +
echo "export LC_ALL=en_US.UTF-8" >> /home/foo/.bashrc
 +
</source>
 +
 +
=== Security ===
 +
By default the server login is the ''root'' user with a password, so the first thing I did was to set up another user for myself, add the user to ''/etc/sudoers'' with full privileges and no password requirement. Note that you need to use the '''sudo''' or '''visudo''' utility to modify, not the usual ''vi'' or ''nano'' utilities.
 +
 +
The '''www-data'' line allows git repositories to be automatically updated in response to [https://developer.github.com/webhooks/ WebHooks] events sent by Guthub in response to push events.
 +
<source>
 +
fred    ALL=(ALL) NOPASSWD : ALL
 +
www-data ALL=(ALL) NOPASSWD : /usr/bin/git pull --no-edit
 +
</source>
 +
 +
 +
Then we want to disable passwords for [[SSH]] access and use RSA keys as typing passwords is insecure. Check or add these settings in ''/etc/ssh/sshd_config'':
 +
<source>
 +
AllowUsers fred bob sam
 +
PermitRootLogin no
 +
PasswordAuthentication no
 +
RSAAuthentication yes
 +
PubkeyAuthentication yes
 +
</source>
 +
 +
 +
And don't forget to add your public RSA key to '''~/.ssh/authorized_keys''. Note that you'll probably need to create the directory since the account has just been created, and the owner and mode is important.
 +
<source lang="bash">
 +
mkdir /home/USER/.ssh
 +
echo "RSA_KEY" > /home/USER/.ssh/authorized_keys
 +
chown USER:USER -R /home/USER/.ssh
 +
chmod 644 /home/USER/.ssh/authorized_keys
 +
</source>
 +
 +
Restart the SSH server and test that you can login from another terminal window before exiting the current session. You now login as your own user, not the ''root'' user, and then use '''sudo bash''' to gain a ''root'' shell.
 +
<source lang="bash">
 +
service ssh restart
 +
</source>
 +
 +
See also [https://www.booleanworld.com/protecting-ssh-fail2ban/ protecting SSH with fail2ban] if you'd like more control over blocking IPs after failed login attempts.
 +
 +
==== Fail2Ban ====
 +
Fail2ban allows the blocking of repeated login attempts to the server. Install it with '''apt install fail2ban''' and then add the following basic configuration to ''/etc/fail2ban/jail.local'':
 +
<source>
 +
[DEFAULT]
 +
destemail = your@email.here
 +
sendername = Fail2Ban
 +
 +
[sshd]
 +
enabled = true
 +
port = 22
 +
 +
[sshd-ddos]
 +
enabled = true
 +
port = 22
 +
</source>
 +
 +
==== Rootkit Hunter ====
 +
Install [http://rkhunter.sourceforge.net/ The Rootkit Hunter] with '''apt-get install rkhunter''' and uncomment the following lines as these files are normal on Debian systems and should not be considered as attacks. Also have a look at the Debian README file with '''zcat /usr/share/doc/rkhunter/README.Debian.gz'''.
 +
<source>
 +
ALLOWHIDDENDIR=/dev/.udev
 +
ALLOWHIDDENDIR=/dev/.static
 +
ALLOWHIDDENDIR=/dev/.initramfs
 +
ALLOWHIDDENDIR=/dev/.mdadm
 +
...
 +
RTKT_FILE_WHITELIST="/etc/init.d/hdparm /etc/init.d/.depend.boot"
 +
...
 +
USER_FILEPROP_FILES_DIRS="/etc/init.d/.depend.boot"
 +
</source>
  
The Apache configuration will already have been configured from within the decompressed FS backup, but if you're not installing from a backup, then here's our typical vhost configuration which maps domains to filesystem structure. We usually just replace the default virtual hosts file with this one which handles all domains and sites.
 
{{code|<pre>
 
NameVirtualHost *:80
 
<VirtualHost *:80>
 
        DocumentRoot /var/www/domains
 
  
        RewriteEngine On
+
Then run a properties update on it since we've added some custom files to the whitelist and need notification if they change, and then run a local test to see if there are any warnings.
 +
<source>
 +
rkhunter --propupd
 +
rkhunter -c
 +
</source>
  
        RewriteCond %{REQUEST_URI} ^/$
+
==== Monitoring filesystem events ====
        RewriteCond %{HTTP_HOST} ^(www\.|wiki\.)?(.+)$
+
A good idea, especially if you suspect any kind of remote code execution (RCE) bugs in webapps etc is to monitor for changes in the filesystem. Here's a simple method using [https://github.com/emcrisostomo/fswatch FSWatch]. first download the latest release and unpack it, then install it:
        RewriteRule ^ /%2/wiki/index.php/Main_Page [L]
+
<source lang="bash">
 +
./configure
 +
make
 +
sudo make install
 +
sudo ldconfig
 +
</source>
  
        RewriteCond %{REQUEST_URI} ^/files/thumb/./../.+?/[0-9]+px-
+
I used the following command format to log what I'm interested in. I'm, using the inefficient ''poll-monitor'' here because I for some reason the default ''inotify-monitor'' wouldn't return events for files in sub-directories, I added the ''l 30'' option to specify a latency of 30 seconds to save CPU resource. I'm piping the output through ''egrep -v'' so I can specify a pattern to filter out of the log.
        RewriteCond %{HTTP_HOST} ^(www\.|wiki\.)?(.+)$
+
<source lang="bash">
        RewriteRule ^.+/(.+?)/([0-9]+)px- /%2/wiki/thumb.php?w=$2&f=$1 [L]
+
fswatch --monitor=poll_monitor -trxl 30 /var/www | egrep -v "(gitlist-cache|\.log$)" >> /var/log/fswatch.log &
 +
</source>
 +
I added the command to the ''@reboot'' event in the ''crontab'' along with a second item to mail me the last 20 items in the log each day.
 +
<source>
 +
@reboot    root  fswatch --monitor=poll_monitor -trxl 30 /var/www | egrep -v "(gitlist-cache|\.log$)" >> /var/log/fswatch.log
 +
0 0 * * *  root  tail -n 20 /var/log/fswatch.log | mail -s "Daily FS changes" "foo@bar.baz"
 +
</source>
  
        RewriteCond %{REQUEST_URI} ^/(wiki/|html|files/|[fF]avicon.ico|[rR]obots.txt)
+
=== Installing packages ===
        RewriteCond %{HTTP_HOST} ^(www\.|wiki\.)?(.+)$
+
Then begin installing the necessary packages,
        RewriteRule (.*) /%2$1 [L]
+
<source lang="bash">
 +
apt install sudo fail2ban host net-tools screen cpulimit build-essential python-dev ntp p7zip-full bzip2 unzip git poppler-utils encfs curl htmldoc librsvg2-bin imagemagick redis-server
 +
</source>
  
        RewriteCond %{HTTP_HOST} ^(www\.|wiki\.)?(.+)$
 
        RewriteRule (.*) /%2/wiki/index.php$1 [L]
 
  
</VirtualHost>
+
The following if you're going to be using email on the server:
</pre>}}
+
<source lang="bash">
 +
apt install exim4-daemon-heavy dovecot-common dovecot-imapd spamassassin spamc maildirsync
 +
</source>
  
== Domain names ==
 
Adjust the names of the symlinks in the ''/var/www/domains'' directory to local domain names and ensure that those names are added to the ''/etc/hosts'' file.
 
*'''Note:''' If you're installing your wikia structure on a local machine, then you must ensure that your domains such as ''foo.localhost'' are set in ''/etc/hosts'' as aliases for ''127.0.0.1''
 
  
== Extracting Databases from a Backup ==
+
The following Perl packages and utilites:
Extract the most recent database backup (this may overwrite existing databases of the same names)
+
<source lang="bash">
{{code|<pre>
+
apt install libwww-perl libio-socket-ssl-perl libtimedate-perl
7za x all-yyyy-mm-dd.sql.7z
+
cpan JSON HTML::Entities Archive::Zip XML::Simple Net::DNS Term::ReadPassword Perl::Version Email::MIME Email::Sender::Simple Math::Random::Secure
mysql -u root -p < all.sql
+
</source>
mysqladmin -u root -p flush-privileges
+
'''Note:''' If XML::Simple fails to install, try the ''apt'' package ''libxml-libxml-simple-perl'' first.
</pre>}}
 
  
== Setting up FTP access ==
 
Some clients may require standard FTP access which although not very secure, can have some restrictions put on it to make it a little safer such as restricting users to their home directories and using a non standard port. We use the GPL [http://www.proftpd.org proFTPD] server in '''standalone''' mode.
 
{{code|<pre>
 
apt-get install proftpd
 
</pre>}}
 
Edit the ''/etc/proftpd/proftpd.conf'' file and change the port to something other than 22 and add the following directive to restrict users to their home directories.
 
{{code|<pre>
 
DefaultRoot ~
 
</pre>}}
 
  
== Setting up SSL for Apache==
+
'''Math markup:''' We've always used Latex for this, but the installation can be complicated and it requires over a GB (yes a Gigabyte!) of packages to be installed that we don't use for anything else. But now client side rendering is possible, or server-side via the new Mathoid node.js service both of which are vastly preferable. See [[MW:Extension:Math]] for more detail about installation options. For a fully client-side rendering solution with no server-side installation at all and no configuration, the simplest method is just to install the [[MW:Extension:SimpleMathJax|SimpleMathJax extension]].
*[[Generate a self signed certificate]]
 
*[[Generate a certificate request for a commercial Certificate Authority]]
 
  
Create a file in the same dir called organicdesign.co.nz with this content.
+
=== NodeJS ===
{{code|<pre>
+
Many services depend on [https://nodejs.org NodeJS] these days, so it's a good idea to have the latest long term service release installed. For Debian installation instructions check [https://github.com/nodesource/distributions#debinstall here].
<VirtualHost organicdesign.co.nz:443>
 
        DocumentRoot /var/www/domains
 
ServerAlias private.organicdesign.co.nz
 
ServerAlias www.organicdesign.co.nz
 
  
        LogFormat "%h %l %u %t \"%r\" %>s %b" common
+
=== MariaDB ===
        CustomLog /var/log/apache2/access.log common
+
Either install the '''mysql-server''' package with ''apt-get'', or go through [https://kb.askmonty.org/en/installing-mariadb-deb-files/ this procedure] for installing [[MariaDB]] instead which is a truly open source drop-in replacement for MySQL forked from the original by the creators.
  
        RewriteEngine On
+
Here's some settings for '''/etc/mysql/mariadb.conf.d/50-server.cnf''' that we've found to be a good idea:
 +
<source lang="nginx">
 +
bind-address=127.0.0.1    # Ensure the database is not open to external connections (already default in Debian 10)
 +
max_statement_time=0.5    # Don't allow long executing SQL statements that clog the machine up
 +
</source>
  
        SSLEngine on
+
One issue that can occur after moving server for both MySQL and MariaDB is the following error produced every day:
        SSLCertificateFile /var/www/ssl/organicdesign.co.nz.crt
+
<source>
        SSLCertificateKeyFile /var/www/ssl/organicdesign.co.nz.key
+
/etc/cron.daily/logrotate:
 +
error: error running shared postrotate script for '/var/log/mysql.log /var/log/mysql/mysql.log /var/log/mysql/mysql-slow.log '
 +
run-parts: /etc/cron.daily/logrotate exited with return code 1
 +
</source>
 +
This is due to the ''debian-sys-maint'' user not having permission to access ''mysqladmin'' to rotate the logs either due to the MySQL user missing, or having the wrong password (thanks to [http://www.lornajane.net/about Lornajane] for her solution in [http://www.lornajane.net/posts/2008/logrotate-error-on-ubuntu#comment-13031 this post]). Get the password from the ''/etc/mysql/debian.cnf'' configuration file and then either update the password if the user exists, or create the user with the correct password if not.
 +
<source lang="mysql">
 +
USE mysql
 +
UPDATE user SET Password = PASSWORD('**************') WHERE User = 'debian-sys-maint' && Host = 'localhost';
 +
FLUSH PRIVILEGES
 +
</source>
 +
::or
 +
<source lang="mysql">
 +
GRANT RELOAD, SHUTDOWN, PROCESS, SHOW DATABASES, SUPER, LOCK TABLES ON *.* TO 'debian-sys-maint'@'localhost' IDENTIFIED BY PASSWORD '**************';
 +
FLUSH PRIVILEGES
 +
</source>
  
        RewriteCond %{REQUEST_URI} ^/$
+
You can check if the maintenance user has its access correctly configured with the following command:
        RewriteCond %{HTTP_HOST} ^(www\.|wiki\.)?(.+)$
+
<source lang="bash">
        RewriteRule ^ /%2/wiki/index.php/Main_Page [L]
+
mysqladmin --defaults-file=/etc/mysql/debian.cnf ping
 +
</source>
  
        RewriteCond %{REQUEST_URI} ^/files/thumb/./../.+?/[0-9]+px-
+
'''Open files limits:''' After installing a lot of Joomla's all using table prefixes in shared databases, each using about 140 tables, I started getting "too many open files" errors, so I've doubled the ''open-files-limit'' and ''innodb_open_files'' values.
        RewriteCond %{HTTP_HOST} ^(www\.|wiki\.)?(.+)$
 
        RewriteRule ^.+/(.+?)/([0-9]+)px- /%2/wiki/thumb.php?w=$2&f=$1 [L]
 
  
        RewriteCond %{REQUEST_URI} ^/(wiki/|html|files/|common/|blogs|[fF]avicon.ico|[rR]obots.txt)
+
== Web-server and PHP ==
        RewriteCond %{HTTP_HOST} ^(www\.|wiki\.)?(.+)$
+
First install the packages:
        RewriteRule (.*) /%2$1 [L]
+
<source lang="bash">
 +
apt install nginx php-fpm php-cli composer php-mysql php-gd php-intl php-curl php-mbstring php-bcmath php-imagick php-zip pbp-xml
 +
</source>
  
        RewriteCond %{HTTP_HOST} ^(www\.|wiki\.)?(.+)$
+
The differences to the default ''php.ini'' file in our servers are as follows:
        RewriteRule (.*) /%2/wiki/index.php$1 [L]
+
<source>
</VirtualHost>
+
post_max_size = 100M
</pre>}}
+
    ︙
 +
upload_max_filesize = 100M
 +
    ︙
 +
[opcache]
 +
opcache.enable=1
 +
opcache.enable_cli=1
 +
opcache.interned_strings_buffer=8
 +
opcache.max_accelerated_files=10000
 +
opcache.memory_consumption=128
 +
opcache.save_comments=1
 +
opcache.revalidate_freq=1
 +
</source>
  
 +
Un-comment the ''env'' entries towards the end of ''www.conf'' in the ''pool.d'' sub-directory as well:
 +
<source>
 +
;env[HOSTNAME] = $HOSTNAME
 +
;env[PATH] = /usr/local/bin:/usr/bin:/bin
 +
;env[TMP] = /tmp
 +
;env[TMPDIR] = /tmp
 +
;env[TEMP] = /tmp
 +
</source>
  
Run this line
+
=== Setting up SSL ===
{{code|<pre>
+
See [[SSL]]
echo 'Include /var/www/ssl/organicdesign.co.nz' >> /var/www/vhosts
 
</pre>}}
 
  
 +
== Domain names ==
 +
Adjust the names of the symlinks in the '''/var/www/domains''' directory to local domain names and ensure that those names are added to the ''/etc/hosts'' file.
 +
*'''Note:''' If you're installing your wikia structure on a local machine, then you must ensure that your domains such as ''foo.localhost'' are set in ''/etc/hosts'' as aliases for ''127.0.0.1''
 +
*'''DNS:''' if you need to set up a DNS server or Dymamic DNS system, see [[Configure DNS]]
  
Check the apache config before reloading
+
== Extracting Databases from a Backup ==
{{code|<pre>
+
Extract the most recent database backup (this may overwrite existing databases of the same names)
apachectl -t
+
<source>
</pre>}}
+
7za x all-yyyy-mm-dd.sql.7z
 +
mysql -u root -p < all.sql
 +
mysqladmin -u root -p flush-privileges
 +
</source>
  
 +
== Setting up SFTP access ==
 +
The [http://www.openssh.org/ OpenSSH server] comes with good [[w:SSH File Transfer Protocol|SFTP]] support built in and allows users to be set up that have only ''SFTP'' access and can be restricted to specified sub-directories. The configuration is done from '''/etc/ssh/sshd_config''' (it must be the OpenSSH server), first enable the SFTP subsystem by un-commenting or adding the following directive:
 +
<source>
 +
Subsystem sftp internal-sftp
 +
</source>
  
Check the cert with this command:
 
{{code|<pre>
 
cd /var/www/ssl
 
openssl s_server -cert organicdesign.co.nz.crt -key organicdesign.co.nz.key
 
</pre>}}
 
  
 +
Next add a section like the following example for each user requiring access,
 +
<source>
 +
Match User foo
 +
ChrootDirectory /var/www
 +
ForceCommand internal-sftp
 +
X11Forwarding no
 +
AllowTcpForwarding no
 +
</source>
  
The following output indicates the cert is working correctly
 
{{code|<pre>
 
Using default temp DH parameters
 
Using default temp ECDH parameters
 
ACCEPT
 
</pre>}}
 
  
 +
You can check for problems in the '''/var/log/auth.log''' file. The most common issue will be to do with permissions. {{h|The root folder that is given access to the SFTP subsystem must be owned by ''root'' and be in the ''root'' group. It must be writable only by ''root'', but readable by the SFTP user.}} The connecting clients use a path relative to the ''chroot'' directory given to them in their matching configuration section.
  
If everything is ok reload the server
+
=== SFTP windows clients ===
{{code|<pre>
+
Windows users can use the [https://filezilla-project.org/ FileZilla] FTP client to connect to the server over SFTP using key-based logins.
/etc/init.d/apache2 reload
 
</pre>}}
 
  
 +
First you need to import your private key by going into edit/settings and then SFTP in the treeview and click the Add Key button. This will then allow you to convert the key to the windows ppk format and save it in its list.
  
Check the error log for problems.
+
You can then set up a new site entry using protocol ''SFTP'', and authentication type ''Interactive''.
{{code|<pre>
 
tail -f /var/log/apache2/error.log
 
</pre>}}
 
  
 +
== Setting up FTP access ==
 +
Some clients may require standard FTP access which although not very secure, can have some restrictions put on it to make it a little safer such as restricting users to their home directories and using a non standard port. We use the GPL [http://www.proftpd.org proFTPD] server in '''standalone''' mode.
 +
<source>
 +
apt-get install proftpd
 +
</source>
  
If you see a message like this everything is ok.
 
{{code|<pre>
 
[Sat Mar 14 11:32:18 2009] [notice]
 
    Apache/2.2.9 (Debian) DAV/2 SVN/1.4.2 PHP/5.2.0-8+etch13 mod_ssl/2.2.9 OpenSSL/0.9.8g configured
 
    -- resuming normal operations
 
</pre>}}
 
  
 +
Edit the '''/etc/proftpd/proftpd.conf''' file and change the port to something other than 21 and add the following directive to restrict users to their home directories (or set it to a shared FTP directory).
 +
<source>
 +
DefaultRoot ~
 +
</source>
  
If you see something like this you have problems:
+
=== Following Symlinks ===
{{code|<pre>
+
Note that following symlinks is not supported if the ''DefaultRoot'' directive is used because the directive creates a "jail" preventing access to any directories outside of it. Some administrators have said that ''mount --bind'' can be used to achieve this but it hasn't worked for us as that seems to just create a normal symlink as well.
[Fri Mar 13 21:46:54 2009] [error] SSL Library Error:
 
    218529960 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag
 
[Fri Mar 13 21:46:54 2009] [error] SSL Library Error:
 
    218595386 error:0D07803A:asn1 encoding routines:ASN1_ITEM_EX_D2I:nested asn1 error
 
Check permissions and paths of certs and keys. The server will be down now.
 
    Comment the last ''Include'' from ''vhosts'' and ''/etc/init.d/apache2 start''.
 
    Server will be up now with no ssl.
 
</pre>}}
 
  
 
== Next steps ==
 
== Next steps ==
 
*[[Configure mail server]]
 
*[[Configure mail server]]
*[[Install a MediaWiki code-base]]
+
*[[NextCloud]]
*[[Install a new wiki]]
+
*[[Configure DNS]] ''- LAN/Server based DNS and dynamic DNS solutions''
  
 
== See also ==
 
== See also ==
*[[Debian Post Install]]
+
*[[Server migration checklist]]
 +
*[[Install a new server (CentOS)]] ''- legacy''
 +
*[[Backup]]
 +
*[[Hosting]]
 
*[[Add an IP address to a server]]
 
*[[Add an IP address to a server]]
 
[[Category:Installation]]
 
[[Category:Installation]]

Revision as of 23:34, 19 July 2019

Procedure.svg Install a new server
Organic Design procedure

Choose a hosting provider

First an appropriate hosting provider needs to be found, or if running a server in-house, see the Configure LAN procedure. Some possible points to check out when looking for a server hosting service apart from just the cost are:

  • Ease of hardware upgrading - can you upgrade disk/memory/cpu without reinstalling the system?
  • Contention rate (how many concurrent clients share the hardware if its a VPS)
  • Control panel usefulness (the most important features are rebooting and virtual console access)
  • OS choices available (up to date Debian or Ubuntu are most important for us)
  • Historical downtime statistics
  • What jurisdiction are they hosting in and what laws apply? for example can your run hidden services, i2p/tor routers or torrent daemons?
  • Do they accept Bitcoin or other crypto-currencies for payment?
  • What kind of data backup options do they provide?

Get reverse DNS set up

Any site that sends emails should have reverse DNS correctly configured. Having a reverse DNS correctly set up will help to prevent the site's mails being trashed as spam. Many mail-servers will do a reverse lookup on the sending IP address and ensure it matches the senders specified domain.

This is not done by the domain registrar, it's done by the company hosting the server (the IP address owner), sometimes they include the ability to set it in the server management interface. If not, raise a support ticket asking them if they can set up a PTR record for the server's IP pointing to your naked domain.

You can check the reverse DNS for a domain at DNSstuff in their IP section, and you can find out more about what reverse DNS is and why it's important here.

Set up a caching DNS server

Setting up a local caching non-forwarding DNS server is a good idea to ensure that DNS requests return quickly (especially useful if you have sites that make requests before returning the pages). It's also essential if you're running spam assassin because the domain black-lists (DNSBL) services operate over DNS and will often block requests made from large ISP's DNS servers. See Configure DNS for installation details.

Download and install Debian or Ubuntu

If the server has no OS then download and install Debian/Ubuntu first. Depending on the kind of access you have to the server and the kind of media it can accept, the following links may be of interest.

Setting up the OS environment

First, bring the system up to date.

apt update
apt upgrade

Give the server's root account (and any other accounts that send mail) a friendly name so it looks better in the inbox when it sends mail. Do this by replacing the name "root" in the full-name field in /etc/passwd as follows:

root:x:0:0:Organic Design server:/root:/bin/bash

Note: If you want mail from the root account sent to something other that root@yourdomain then, set the address in /root/.forward.

Timezone

It's best for a server to run on UTC timezone,

sudo timedatectl set-timezone UTC


Getting the following warning plus a bunch of others whenever Perl scripts run?

perl: warning: Falling back to the standard locale ("C").


Configure the locales and tick the time zones you'd like to have available on the system, make sure that en_US.8859-1 and en_US.UTF-8 are selected because it seems that some programs expect them to exist. Also make sure that all locales ate included that were mentioned anywhere within the warning messages.

dpkg-reconfigure locales


If you're still getting the problem, you can explicitly set the associated variables to one of the locales you made available by appending entries to your ~/.bashrc as in the following example, you may need to log out and back in for these changes to take effect.

echo "export LC_ALL=en_US.UTF-8" >> /home/foo/.bashrc

Security

By default the server login is the root user with a password, so the first thing I did was to set up another user for myself, add the user to /etc/sudoers with full privileges and no password requirement. Note that you need to use the sudo or visudo utility to modify, not the usual vi or nano utilities.

The 'www-data line allows git repositories to be automatically updated in response to WebHooks events sent by Guthub in response to push events.

fred     ALL=(ALL) NOPASSWD : ALL
www-data ALL=(ALL) NOPASSWD : /usr/bin/git pull --no-edit


Then we want to disable passwords for SSH access and use RSA keys as typing passwords is insecure. Check or add these settings in /etc/ssh/sshd_config:

AllowUsers fred bob sam
PermitRootLogin no
PasswordAuthentication no
RSAAuthentication yes
PubkeyAuthentication yes


And don't forget to add your public RSA key to '~/.ssh/authorized_keys. Note that you'll probably need to create the directory since the account has just been created, and the owner and mode is important.

mkdir /home/USER/.ssh
echo "RSA_KEY" > /home/USER/.ssh/authorized_keys
chown USER:USER -R /home/USER/.ssh
chmod 644 /home/USER/.ssh/authorized_keys

Restart the SSH server and test that you can login from another terminal window before exiting the current session. You now login as your own user, not the root user, and then use sudo bash to gain a root shell.

service ssh restart

See also protecting SSH with fail2ban if you'd like more control over blocking IPs after failed login attempts.

Fail2Ban

Fail2ban allows the blocking of repeated login attempts to the server. Install it with apt install fail2ban and then add the following basic configuration to /etc/fail2ban/jail.local:

[DEFAULT]
destemail = your@email.here
sendername = Fail2Ban

[sshd]
enabled = true
port = 22

[sshd-ddos]
enabled = true
port = 22

Rootkit Hunter

Install The Rootkit Hunter with apt-get install rkhunter and uncomment the following lines as these files are normal on Debian systems and should not be considered as attacks. Also have a look at the Debian README file with zcat /usr/share/doc/rkhunter/README.Debian.gz.

ALLOWHIDDENDIR=/dev/.udev
ALLOWHIDDENDIR=/dev/.static
ALLOWHIDDENDIR=/dev/.initramfs
ALLOWHIDDENDIR=/dev/.mdadm
...
RTKT_FILE_WHITELIST="/etc/init.d/hdparm /etc/init.d/.depend.boot"
...
USER_FILEPROP_FILES_DIRS="/etc/init.d/.depend.boot"


Then run a properties update on it since we've added some custom files to the whitelist and need notification if they change, and then run a local test to see if there are any warnings.

rkhunter --propupd
rkhunter -c

Monitoring filesystem events

A good idea, especially if you suspect any kind of remote code execution (RCE) bugs in webapps etc is to monitor for changes in the filesystem. Here's a simple method using FSWatch. first download the latest release and unpack it, then install it:

./configure
make
sudo make install
sudo ldconfig

I used the following command format to log what I'm interested in. I'm, using the inefficient poll-monitor here because I for some reason the default inotify-monitor wouldn't return events for files in sub-directories, I added the l 30 option to specify a latency of 30 seconds to save CPU resource. I'm piping the output through egrep -v so I can specify a pattern to filter out of the log.

fswatch --monitor=poll_monitor -trxl 30 /var/www | egrep -v "(gitlist-cache|\.log$)" >> /var/log/fswatch.log &

I added the command to the @reboot event in the crontab along with a second item to mail me the last 20 items in the log each day.

@reboot     root   fswatch --monitor=poll_monitor -trxl 30 /var/www | egrep -v "(gitlist-cache|\.log$)" >> /var/log/fswatch.log
0 0 * * *   root   tail -n 20 /var/log/fswatch.log | mail -s "Daily FS changes" "foo@bar.baz"

Installing packages

Then begin installing the necessary packages,

apt install sudo fail2ban host net-tools screen cpulimit build-essential python-dev ntp p7zip-full bzip2 unzip git poppler-utils encfs curl htmldoc librsvg2-bin imagemagick redis-server


The following if you're going to be using email on the server:

apt install exim4-daemon-heavy dovecot-common dovecot-imapd spamassassin spamc maildirsync


The following Perl packages and utilites:

apt install libwww-perl libio-socket-ssl-perl libtimedate-perl
cpan JSON HTML::Entities Archive::Zip XML::Simple Net::DNS Term::ReadPassword Perl::Version Email::MIME Email::Sender::Simple Math::Random::Secure

Note: If XML::Simple fails to install, try the apt package libxml-libxml-simple-perl first.


Math markup: We've always used Latex for this, but the installation can be complicated and it requires over a GB (yes a Gigabyte!) of packages to be installed that we don't use for anything else. But now client side rendering is possible, or server-side via the new Mathoid node.js service both of which are vastly preferable. See MW:Extension:Math for more detail about installation options. For a fully client-side rendering solution with no server-side installation at all and no configuration, the simplest method is just to install the SimpleMathJax extension.

NodeJS

Many services depend on NodeJS these days, so it's a good idea to have the latest long term service release installed. For Debian installation instructions check here.

MariaDB

Either install the mysql-server package with apt-get, or go through this procedure for installing MariaDB instead which is a truly open source drop-in replacement for MySQL forked from the original by the creators.

Here's some settings for /etc/mysql/mariadb.conf.d/50-server.cnf that we've found to be a good idea:

bind-address=127.0.0.1    # Ensure the database is not open to external connections (already default in Debian 10)
max_statement_time=0.5    # Don't allow long executing SQL statements that clog the machine up

One issue that can occur after moving server for both MySQL and MariaDB is the following error produced every day:

/etc/cron.daily/logrotate:
error: error running shared postrotate script for '/var/log/mysql.log /var/log/mysql/mysql.log /var/log/mysql/mysql-slow.log '
run-parts: /etc/cron.daily/logrotate exited with return code 1

This is due to the debian-sys-maint user not having permission to access mysqladmin to rotate the logs either due to the MySQL user missing, or having the wrong password (thanks to Lornajane for her solution in this post). Get the password from the /etc/mysql/debian.cnf configuration file and then either update the password if the user exists, or create the user with the correct password if not.

USE mysql
UPDATE user SET Password = PASSWORD('**************') WHERE User = 'debian-sys-maint' && Host = 'localhost';
FLUSH PRIVILEGES
or
GRANT RELOAD, SHUTDOWN, PROCESS, SHOW DATABASES, SUPER, LOCK TABLES ON *.* TO 'debian-sys-maint'@'localhost' IDENTIFIED BY PASSWORD '**************';
FLUSH PRIVILEGES

You can check if the maintenance user has its access correctly configured with the following command:

mysqladmin --defaults-file=/etc/mysql/debian.cnf ping

Open files limits: After installing a lot of Joomla's all using table prefixes in shared databases, each using about 140 tables, I started getting "too many open files" errors, so I've doubled the open-files-limit and innodb_open_files values.

Web-server and PHP

First install the packages:

apt install nginx php-fpm php-cli composer php-mysql php-gd php-intl php-curl php-mbstring php-bcmath php-imagick php-zip pbp-xml

The differences to the default php.ini file in our servers are as follows:

post_max_size = 100M
    ︙
upload_max_filesize = 100M
    ︙
[opcache]
opcache.enable=1
opcache.enable_cli=1
opcache.interned_strings_buffer=8
opcache.max_accelerated_files=10000
opcache.memory_consumption=128
opcache.save_comments=1
opcache.revalidate_freq=1

Un-comment the env entries towards the end of www.conf in the pool.d sub-directory as well:

;env[HOSTNAME] = $HOSTNAME
;env[PATH] = /usr/local/bin:/usr/bin:/bin
;env[TMP] = /tmp
;env[TMPDIR] = /tmp
;env[TEMP] = /tmp

Setting up SSL

See SSL

Domain names

Adjust the names of the symlinks in the /var/www/domains directory to local domain names and ensure that those names are added to the /etc/hosts file.

  • Note: If you're installing your wikia structure on a local machine, then you must ensure that your domains such as foo.localhost are set in /etc/hosts as aliases for 127.0.0.1
  • DNS: if you need to set up a DNS server or Dymamic DNS system, see Configure DNS

Extracting Databases from a Backup

Extract the most recent database backup (this may overwrite existing databases of the same names)

7za x all-yyyy-mm-dd.sql.7z
mysql -u root -p < all.sql
mysqladmin -u root -p flush-privileges

Setting up SFTP access

The OpenSSH server comes with good SFTP support built in and allows users to be set up that have only SFTP access and can be restricted to specified sub-directories. The configuration is done from /etc/ssh/sshd_config (it must be the OpenSSH server), first enable the SFTP subsystem by un-commenting or adding the following directive:

Subsystem sftp internal-sftp


Next add a section like the following example for each user requiring access,

Match User foo
ChrootDirectory /var/www
ForceCommand internal-sftp
X11Forwarding no
AllowTcpForwarding no


You can check for problems in the /var/log/auth.log file. The most common issue will be to do with permissions. The root folder that is given access to the SFTP subsystem must be owned by root and be in the root group. It must be writable only by root, but readable by the SFTP user. The connecting clients use a path relative to the chroot directory given to them in their matching configuration section.

SFTP windows clients

Windows users can use the FileZilla FTP client to connect to the server over SFTP using key-based logins.

First you need to import your private key by going into edit/settings and then SFTP in the treeview and click the Add Key button. This will then allow you to convert the key to the windows ppk format and save it in its list.

You can then set up a new site entry using protocol SFTP, and authentication type Interactive.

Setting up FTP access

Some clients may require standard FTP access which although not very secure, can have some restrictions put on it to make it a little safer such as restricting users to their home directories and using a non standard port. We use the GPL proFTPD server in standalone mode.

apt-get install proftpd


Edit the /etc/proftpd/proftpd.conf file and change the port to something other than 21 and add the following directive to restrict users to their home directories (or set it to a shared FTP directory).

DefaultRoot ~

Following Symlinks

Note that following symlinks is not supported if the DefaultRoot directive is used because the directive creates a "jail" preventing access to any directories outside of it. Some administrators have said that mount --bind can be used to achieve this but it hasn't worked for us as that seems to just create a normal symlink as well.

Next steps

See also