Difference between revisions of "User/Alvonruff/Python3 Conversion"

From ISFDB
Jump to navigation Jump to search
(Created page with "== Steps == [https://isfdb.org/wiki/index.php/User:Alvonruff#Python3_Notes Underway]. This is a larger project than the LAMP update, and will take a few months. Steps ** Do a...")
 
Line 1: Line 1:
−
== Steps ==
+
The primary difficulty with a python3 conversion project is trying to avoid a massive rewrite of the website, and then checkin all those changes with a single big bang integration. That said, the primary function of the scripts is to read data from MySQL, and then organize and print that information to the browser. But the two things that change with python3 is the MySQL connector (which requires a rewrite of all code interfacing with MySQL), and the way print statements work (which requires a rewrite of all code outputting information). So the first goal is to find a way for the ISFDB to exist simultaneously in Python2 and Python3 format. General outline of steps to move to Python3:
−
[https://isfdb.org/wiki/index.php/User:Alvonruff#Python3_Notes Underway]. This is a larger project than the LAMP update, and will take a few months. Steps
 
−
** Do a blunt-force port to python3 on isfdb2.org. This will discover the porting patterns we need to worry about.
 
−
** Integrate tab/spaces changes in small bundles. This should be invisible to isfdb.org.
 
−
** Integrate print changes with futurize imports in small bundles. This should be invisible to isfdb.org.
 
−
** Integrate string changes in small bundles. This should be invisible to isfdb.org.
 
−
** Integrate mysql connector class in small bundles. This should be invisible to isfdb.org.
 
−
** I reserve writing down the next steps until I get further along and can diff the python2 and python3 versions.
 
−
* 2024: Update charset to UTF-8.
 
  
−
== Python3 Notes ==
+
* Perform a blunt-force port to Python3 at isfdb2.org.
 +
* Introduce a python selection mechanism in the build system.
 +
* Integrate the formal tests in the new tests directory.
 +
* Fix the mixed tab/space issues.
 +
* Make modifications to use python3 print syntax via futurize. Some Futurize imports may need to utilize the python selection mechanism.
 +
* Change usage of string objects to str.
 +
* Introduce the connector class which hides the details of which MySQL connector is in use.
 +
* Modify all code to use the new connector class.
 +
* Make the use of encode/decode selectable via the python selection mechanism.
 +
* Make a has_key convenience function that can select the supported routines based on the python selection mechanism.
  
−
The primary difficulty with a python3 conversion project is trying to avoid a massive rewrite of the website, and then checkin all those changes with a single big bang integration. That said, the primary function of the scripts is to read data from MySQL, and then organize and print that information to the browser. But the two things that change with python3 is the MySQL connector (which requires a rewrite of all code interfacing with MySQL), and the way print statements work (which requires a rewrite of all code outputting information). So the first goal is to find a way for the ISFDB to exist simultaneously in Python2 and Python3 format. General outline of steps to move to Python3:
+
==The "Quick" Port==
 +
 
 +
The first step is to just do whatever it takes to get the scripts to run. This includes hand-editing files, as well as running through futurize. Futurize does (occasionally) make mistakes. The two most egregious so far are:
 +
* Making spurious insertions of '''str'''() casts to things that should remain '''int'''.
 +
* Treating the string 'next' as evidence of iterator use, and converting most (but not all!) of the appearances of 'next' to '__next__'.
 +
As such, we can't simply check this port in when done and call it a day. The integration will need to proceed in a more disciplined manner. The order of the quick port will follow these directories:
 +
 
 +
* '''common''' - DONE
 +
* '''biblio''' - DONE
 +
* Formal test cases for SQLparsing and *Class files - In progress
 +
* '''edit''' - In progress.
 +
* '''mod'''
 +
* '''rest'''
 +
* '''nightly'''
 +
* '''scripts'''
 +
 
 +
Since there are a few hundred files, this phase will likely complete sometime in July.
  
−
# Python3 does not tolerate mixed tabs and spaces. There needs to be a project to convert the tabs in all files to 8 spaces. This change works on either Python2 or Python3
+
==Python Selection Mechanism==
−
# Introduce a MySQL connector class, which hides the differences between MySQLdb (which only works on Python2) and mysql.connector (which only works on Python3).
 
−
# Convert all MySQL code to use the new connector class. This is a large project in itself, with details listed [[User:Alvonruff/MySQLdb_Conversion|here]].
 
−
# [[User:Alvonruff/Python2.7_Futurize|futurize]] all print statements. This can be done by running futurize, and then keeping only the print() syntax changes. These will run fine under python2.
 
−
# Perform the [[User/Alvonruff/2to3|Python3 Conversion]], and follow up on the numerous porting issues.
 
−
# Update all character sets. Final procedure still TBD.
 
−
# Change the default charset in MySQL
 
−
# Repair strings which have URL encodings in MySQL
 
  
−
=== Status Trackers ===
+
xx
−
* [[User:Alvonruff/Debugging_Remarks|Debugging Remarks]]
 
−
* Per-File Status: [[User:Alvonruff/Python3_Files|Python3 File Status]]
 
−
* ISFDB2 Status: [[User:Alvonruff/Python3_ISFDB2|ISFDB2 Per-Script Status]]
 
−
* Formal Tests: [[User:Alvonruff/ISFDB_Tests|Formal ISFDB Test Status]]
 

Revision as of 07:08, 21 May 2023

The primary difficulty with a python3 conversion project is trying to avoid a massive rewrite of the website, and then checkin all those changes with a single big bang integration. That said, the primary function of the scripts is to read data from MySQL, and then organize and print that information to the browser. But the two things that change with python3 is the MySQL connector (which requires a rewrite of all code interfacing with MySQL), and the way print statements work (which requires a rewrite of all code outputting information). So the first goal is to find a way for the ISFDB to exist simultaneously in Python2 and Python3 format. General outline of steps to move to Python3:

  • Perform a blunt-force port to Python3 at isfdb2.org.
  • Introduce a python selection mechanism in the build system.
  • Integrate the formal tests in the new tests directory.
  • Fix the mixed tab/space issues.
  • Make modifications to use python3 print syntax via futurize. Some Futurize imports may need to utilize the python selection mechanism.
  • Change usage of string objects to str.
  • Introduce the connector class which hides the details of which MySQL connector is in use.
  • Modify all code to use the new connector class.
  • Make the use of encode/decode selectable via the python selection mechanism.
  • Make a has_key convenience function that can select the supported routines based on the python selection mechanism.

The "Quick" Port

The first step is to just do whatever it takes to get the scripts to run. This includes hand-editing files, as well as running through futurize. Futurize does (occasionally) make mistakes. The two most egregious so far are:

  • Making spurious insertions of str() casts to things that should remain int.
  • Treating the string 'next' as evidence of iterator use, and converting most (but not all!) of the appearances of 'next' to '__next__'.

As such, we can't simply check this port in when done and call it a day. The integration will need to proceed in a more disciplined manner. The order of the quick port will follow these directories:

  • common - DONE
  • biblio - DONE
  • Formal test cases for SQLparsing and *Class files - In progress
  • edit - In progress.
  • mod
  • rest
  • nightly
  • scripts

Since there are a few hundred files, this phase will likely complete sometime in July.

Python Selection Mechanism

xx

Retrieved from "https://isfdb.org/wiki/index.php?title=User/Alvonruff/Python3_Conversion&oldid=662768"