User:Alvonruff

From ISFDB
Jump to navigation Jump to search

Founder of the ISFDB.


Notes for Installing ISFDB on Ubuntu 20.04

SSH

  • Issue: sudo apt-get install openssh-server

Apache 2.4.41

  • Issue: uname -a, and note the name of your PC, which we will refer to henceforth as <<PCNAME>>.
  • Issue: sudo apt-get install apache2
  • Issue: sudo /etc/init.d/apache2 start
  • Point your browser to <<PCNAME>>.local - you should see the default apache page.
  • Issue: sudo systemctl enable apache2

Mysql 8.0.28-0ubuntu0.20.04.3

  • Download a backup file
  • Issue: sudo apt install mysql-server
  • Issue: sudo mysql
  • While in mysql, issue the command: create database isfdb;
  • While in mysql, issue the command: use isfdb;
  • While in mysql, issue the command: alter database isfdb character set latin1 collate latin1_swedish_ci;
  • While in mysql, issue the command: source <<backupfile>>;

Python 2.7.18

  • Issue: sudo apt install python2
  • Issue: sudo apt install curl
  • Issue: curl https://bootstrap.pypa.io/pip/2.7/get-pip.py --output get-pip.py
  • Issue: sudo python2 get-pip.py
  • Issue: pip2 --version
  • Issue: sudo apt-get install python2-dev
  • Issue: sudo apt-get install default-libmysqlclient-dev
  • Issue: sudo pip2 install mysqlclient

ISFDB Install

  • Previously, python2 was installed by default as python on Linux, with python3 optionally installed as python3. Now python2 is installed as python2 and python3 is installed as python3'. As such the current ISFDB makefiles are broken, as they invoke /usr/bin/python, which is not present. This needs to be fixed, but for now change python install.py to python2 install.py across the local.mk files.
  • Update common/localdefs.py, by using <<PCNAME>>.local for COOKIEHOST, HTMLHOST, and HTMLLOC, and update HTFAKE and WIKILOC.
  • Update INSTALLDIRS to use /usr/lib/cgi-bin, which is the new preferred cgi-bin location
  • cd to isfdb directory and issue: make
  • cd to /etc/apache2/mods-enabled and issue: sudo ln -s ../mods-available/cgi.load - this will enable the execution of cgi-bin by Apache.
  • if you reload the <<PCNAME>>.local in your browser, you should now see the ISFDB frontpage, with a down for maintenance message.

Link ISFDB to MySQL

  • Issue: sudo mysql
  • While in mysql, issue the command: create user 'isfdb1'@'localhost' identified by 'YourPassword';
  • In the ISFDB, update common/localdefs.py to reflect the new USERNAME and PASSWORD.
  • cd isfdb and issue make
  • Issue: sudo mysql -u root -e "SET GLOBAL sql_mode = 'NO_ENGINE_SUBSTITUTION';" - this will allow MySQL to use strange dates, like 2010-05-00, which indicates sometime in May.
  • Reload the ISFDB page in your browser. You should now see a mostly good web page (minus international character issues).

Issues to Resolve

  • Need a better solution for the python2 makefile issues. Affected files:
    • rest/local.mk
    • mod/local.mk
    • css/Makefile
    • nightly/local.mk
    • edit/local.mk
    • biblio/local.mk
  • Some international characters do not display properly. Seems like names from latin countries are the problem area, as others (Chinese, Japanese, Russian) are fine. Using an online encoder/decoder, it's clear the output is UTF-8 instead of ISO-8859-1.
  • Need to decouple the ISFDB from the Wiki. We currently use the account/login capabilities from the Wiki, and share them with the ISFDB. This makes upgrading either side independently more difficult.
  • There is an issue in common.py in PrintSubmissionLinks, where SQLloadNextSubmission generates an exception. I suspect this is an untested boundary condition related to there being only one moderator in the system, a situation which would not occur at isfdb.org. The issue is in a rather complex SQL statement, which needs further debugging. An exception handler in PrintSubmissionLinks seems to rectify the issue.

ASCII vs UTF-8 vs Latin1

Mysql

Clearly the most expressive character set is utf8mb4. That current ISFDB database is using latin1 (ISO-8859-1).

  • My database (a fresh install defaults to UTF8) has been set to the same character set as the online ISFDB with: alter database isfdb character set latin1 collate latin1_swedish_ci;
  • An attempt was made to update the character set to something that works with the new MySQLdb with: alter database isfdb character set utf8mb4 collate utf8mb4_unicode_ci;. This had no affect on the output.

MySQLdb

It turns out the MySQLdb has moved along as well. There is now a MySQLdb1, which is the legacy version intended for python2, and there is MySQLdb2, which is the new version for python3. But MySQLdb1 is outdated and unsupported, and MySQLdb2 is not ready for production use.

  • MySQLdb - the latest version of MySQLdb has two relevant optional arguments when connecting: use_unicode and charset:
    • use_unicode. The proper format would be use_unicode=True. This is supposed to cause MySQLdb to return unicode characters instead of bytes. This doesn't seem to change any visible behaviors, and the code defaults to False on python2 and True on python3.
    • charset. The proper format would be charset='target_charset', where valid charsets are:
      • utf8mb4 - generates an ascii encoding exception
      • utf8mb3 - generates an ascii encoding exception
      • latin1 - generates an ascii encoding exception
      • koi8r - generates ? in place of the target character.
      • koi8u - generates ? in place of the target character.

I installed mysqldb using sudo pip2 install mysqlclient but that installs MySQLdb2, which is intended for python3. Alternatively, the old MySQLdb (aka MySQLdb1) can be installed through this convoluted method:

MySQL Connector

MySQL has released their own module called MySQL Connector. According to their compatibility chart, Connector release 8.0 can work with MySQL 8.0 and python2.7, but only before connector release 8.0.24. Initial attempts at installing 8.0.23 were unsuccessful, failing the dependency checks.

The latest release can be installed with: sudo pip3 install mysql-connector-python

python2.7

The default encoding in python2.7 is ASCII.

ISFDB

Web Browser

Reading List

2021

2020

2019

2018

2017

2016

2015

2014

2012

2011

2010

2009

2008

2007

2006

Retrieved from "https://isfdb.org/wiki/index.php?title=User:Alvonruff&oldid=628218"