User:Alvonruff/ISFDB Move

Items While System Only Has an IP Address

The new ISFDB location has the static IP address: 67.225.129.86 (Link: http://67.225.129.86/cgi-bin/index.cgi)

Prerequisites

The system is a minimum configuration of Fedora Core. As such, many packages need to be installed with dnf.

• dnf install gcc
• dnf install make
• dnf install tar
• dnf install zip.x86_64
• dnf install bzip2.x86_64
• dnf install wget
• dnf install mod_dav_svn subversion
• dnf install telnet
• dnf install ImageMagick

Apache

• dnf install httpd
• firewall-cmd --add-service=http --add-service=https --permanent
• Add to etc/httpd/httpd.conf: LoadModule dir_module modules/mod_dir.so
• Change etc/httpd/httpd.conf: DirectoryIndex /cgi-bin/index.cgi index.html index.html.var
• systemctl enable httpd
• systemctl start httpd

MySQL

• dnf update
• dnf module enable mysql:8.0
• dnf install @mysql
• systemctl enable mysqld
• systemctl start mysqld
• Issue: mysql
• While in mysql, issue the command: create database isfdb;
• While in mysql, issue the command: use isfdb;
• While in mysql, issue the command: alter database isfdb character set latin1 collate latin1_swedish_ci;
• While in mysql, issue the command: source <<backupfile>>;'
• create user isfdb1@localhost identified by 'PASSWORD';
• GRANT ALL PRIVILEGES ON isfdb.* TO 'isfdb1'@'localhost';
• SET GLOBAL sql_mode = 'NO_ENGINE_SUBSTITUTION';"

Python 2.7.18

• dnf install python2
• dnf install python2-devel
• dnf install mysql-devel
• python2 -m ensurepip --no-default-pip
• pip install --upgrade pip
• pip2 install mysqlclient
• alternatives --install /usr/bin/python python /usr/bin/python3.9 2
• alternatives --install /usr/bin/python python /usr/bin/python2.7 1
• alternatives --config python
• python --version

ISFDB

• In home directory: svn checkout https://svn.code.sf.net/p/isfdb/code-svn/trunk isfdb-code-svn
• Update common/localdefs.py:
  • COOKIEHOST = "67.225.129.86"
  • HTFAKE = "/67.225.129.86/cgi-bin"
  • HTMLHOST = "67.225.129.86"
  • HTMLLOC = "67.225.129.86"
  • PASSWORD = "PASSWORD"
  • USERNAME = "isfdb1"
  • WIKILOC = "67.225.129.86/wiki"
• make

PHP

• dnf module reset php
• dnf module enable php:7.4
• dnf install php php-common php-opcache php-cli php-gd php-curl php-mysqlnd
• systemctl start php-fpm
• systemctl enable php-fpm
• php -v
• systemctl stop httpd
• systemctl start httpd

Add Users

• Add the necessary users to the new system
• Give them sudo access by adding them to the wheel group

MediaWiki

• cd to /var/www/html
• Fetch the current stable long-term support release of MediaWiki:
  • wget https://releases.wikimedia.org/mediawiki/1.35/mediawiki-1.35.6.zip
• unzip mediawiki-1.35.6.zip
• mv mediawiki-1.35.6 wiki
• cd wiki && place the old isfdb version of LocalSettings.php here
• Edit LocalSettings.php and make the follow changes:
  • Change wgServer
  • Change wgDBuser
  • Change wgDBpassword
  • Comment out the ConfirmEdit extension
  • Comment out the SyntaxHighlight extension
  • Comment out the SVGtag extension
• mkdir wiki/images
• chown apache images
• chgrp apache images
• Copy all image subdirectory content from isfdb.org:/var/www/html/wiki/images to the new server.
• php maintenance/update.php
  • Wait a very long time for the update to finish (about 3.5 hours). This will perform hundreds of thousands of revision updates, but it issues a constant stream of progress lines. There is a web-based alternative, but I recommend against that, as the long processing time will generate a 504 error from Apache.
  • It helps to ssh into the system with the "-o ServerAliveInterval=600" option, which will generate a null keep alive packet every 10 minutes, keeping the session alive.
• cp isfdb.gif to wiki/skins/common/images
• Add the following lines to the end of LocalSettings.php and remove the old commented out versions:
  • wfLoadSkin( 'Vector' );
  • wfLoadExtension( 'ConfirmEdit' );
  • wfLoadExtension( 'SyntaxHighlight_GeSHi' );
  • Uncomment and change to: $wgImageMagickConvertCommand = "/usr/bin/convert";
  • Uncomment and change to: $wgLogo = "$wgStylePath/common/images/isfdb.gif";

New Password Support

• pip2 install backports.pbkdf2
• Copy new submitlogin.py into biblio
• set legacy_mode = 0 in submitlogin.py
• reinstall (make) isfdb source

Move Servers (Commence 9/6)

• At old server:
  • DONE: Turn off editing at isfdb.org
  • DONE: Create final backups of database and images.
• At new server:
  • SKIP: Integrate the new password code in submitlogin.py. legacy_mode should be set to 0. The file has been manually placed, but not integrated.
  • DONE: Ahasuerus should do a source install to ensure he has the ability to install source updates.
  • DONE: Drop the isfdb tables from mysql using the commands listed here. (Dropping the entire database may cause goofyness with the isfdb1 user, and the privileges tables)
  • DONE: Remove the wiki images from the filesystem (Everything under /var/www/html/wiki/images)
  • DONE: Re-execute the operations to get the full database backup installed.
  • DONE: Install the latest wiki image backups
  • DONE: Put the original LocalSettings.php in place.
  • DONE: Run: php maintenance/update.php. (Note: This took about 24 hours to execute on the new server last time. If the ssh connection drops during the update, it does keep track of where it is in the process)
  • Disable scripts/wikitrim.py until it has been updated to the new version of MediaWiki

Move Domain Name

• DONE: Update the domain name servers to point to the new server
• DONE: Wait for domain name to propagate (up to 48 hours).
• DONE: Update the following fields in localdefs.py:
  • COOKIEHOST
  • HTFAKE
  • HTMLHOST
  • HTMLLOC
  • WIKILOC
• DONE: Update the following fields in LocalSettings.php
  • wgServer

Mail

• DONE: dnf install postfix
• DONE: Edit /etc/postfix/main.cf:
  • Set myhostname = isfdb.org
  • Set mydomain = isfdb.org
  • Set mail_spool_directory = /var/mail
  • Set mynetworks_style = host
  • Set mynetworks = 127.0.0.0/8 67.225.129.86
  • Uncomment relay_domains = $mydestination
  • Uncomment inet_interfaces = $myhostname, localhost
• DONE: systemctl start postfix
• DONE: systemctl enable postfix
• DONE: firewall-cmd --add-service=smtp --permanent

Certs

Apache

• DONE: dnf install mod_ssl
• DONE: systemctl restart httpd
• DONE: cd /etc/httpd/conf.d
• DONE: Create file isfdb.org.conf
• DONE: Add the following contents:

<VirtualHost *:80>
    ServerName server.isfdb.org
    DocumentRoot /var/www/html
    ServerAlias isfdb.org
    ErrorLog /var/www/error.log
    CustomLog /var/www/requests.log combined
    Redirect permanent / https://isfdb.org/
</VirtualHost>

• systemctl restart httpd

Certificate Support Software

• SKIP: dnf install epel-release (not a valid package any longer)
• DONE: dnf install snapd
• DONE: dnf install certbot
• DONE: dnf install python3-certbot-apache

Certificates

• DONE: certbot --apache
  • Select isfdb.org
• DONE: systemctl restart httpd

Settings

• DONE: Change PROTOCOL in localdefs.py to "https"
• DONE: Change $wgServer in /var/www/html/wiki/LocalSettings.php to use https
• DONE: Go to the wiki and type MediaWiki:Sidebar in the search window. Edit that page and change http to https

-->> WE ARE HERE <<--

While everything at isfdb.org seems to be working, and external links at Wikipedia follow okay, the domain www.isfdb.org is still busted, and points to a GoDaddy parking page. Will debug that tomorrow. Alvonruff (talk) 22:14, 6 September 2022 (EDT)

Just in case you aren't aware, as at 12:05 BST Add Publication (and likely other pages/scripts) is currently broken - browser dev tools implies it's failing to load a bunch of static images, .js files etc from 67.225.129.86 on a http:// URL, but as this is coming from an https:// page, my browser (Vivaldi) is blocking this. I guess whatever setting is used to generate the URLs will have to be switched to use https and a proper domain name ErsatzCulture (talk) 07:05, 7 September 2022 (EDT)

Final Steps

• Uncomment the daily backup job in root's crontab
• Uncomment the nightly job in root's crontab
• Uncomment the weekly job in root's crontab
• Keep the monthly job in root's crontab commented out
• Enable editing at new server
• Update scripts/create_user.py to generate a pbkdf2 hash for the password
• Folks that have logins and data in their home directories on the current isfdb.org will need to move it to the new server, as it will be decommissioned. IP address of the current (old) isfdb.org is: 72.44.93.41

Wiki Content

• The TablePager class was deprecated, so instead of using class="TablePager" use class="wikitable" in all Wiki tables
• Update ISFDB_Downloads#Installation_and_Setup to reflect new Python/MySQL/etc versions and new installation steps
• Remove ISFDB:FAQ#Why_do_browsers_say_that_the_connection_to_ISFDB_is_not_secure.3F
• Update ISFDB:How To Create a Public Backup File with the MediaWiki 1.35+-compliant instructions currently displayed in a separate section at the bottom of the page
• In MediaWiki:Sidebar, change "http://www.isfdb.org/cgi-bin/index.cgi" to "https://www.isfdb.org/cgi-bin/index.cgi"

Versions

• Linux: 5.14.18-100.fc33.x86_64
• Apache: Apache/2.4.37 (AlmaLinux)
• MySQL: 8.0.27
• Python: 2.7.18
• PHP: 7.4.26
• MediaWiki: 1.35.6