User talk:Ahasuerus/Archive/2011

Unknown December 1939
Ahasuerus, the OED is looking for a copy of the Dec 1939 Unknown -- I'm only missing three Unknowns, but that's one of my gaps. Do you have access to your copy? Would you be willing to look something up in it? They are looking for a use of "timeslip" -- currently the earliest usage they've found is 1949, in a reprint of "Lest Darkness Fall", but they want to check the magazine version.
If you're willing to look, the phrase is "Only two hypotheses remained: delirium and time-slip." It appears on p.10 of the book version they were looking at, so it would be quite early in the magazine version. If you do find it, we would like to get the exact form of the quote, including any typos and exact punctuation, and the location of the quote -- which page, which column. If you could also give the name of the publisher as it is on the masthead or indicia, and the magazine title, again from the indicia, that would be perfect. If you can't help, no worries -- I will eventually get a copy for myself! Thanks -- Mike Christie (talk) 12:53, 5 February 2011 (UTC)
Yes, I do have my Unknowns right here. I also have a paperback reprint and an SFBC edition in my collection.
Cross-checking the first 10 pages, I see that the "arrival" scene, which was only a few paragraphs long in the magazine version, was greatly expanded for book publication. The "time-slip" reference (p.8 of the SFBC edition) is in the last paragraph of a two page section which was not present in the magazine version. The latter does mention "slipping back in time", but no "time-slip" that I can find.
I should also point out that a quick check of a few key sections finds that the book version added sentences like "History is a four-dimensional web", so there were other terminological changes.
One key question that I can't answer is whether these changes were first made in the 1941 Holt edition or in the 1949 Prime Press edition. Perhaps someone who owns the Henry Holt version may be able to help. Unfortunately, it is currently not verified, so we don't know if any ISFDB contributors own a copy. Ahasuerus 22:22, 5 February 2011 (UTC)
Thanks; I'll let the OED know. They have a 1949 cite, so it must be in the Prime Press edition; that just leaves the 1941 Henry Holt. I appreciate the help! Mike Christie (talk) 23:00, 5 February 2011 (UTC)
Glad to be able to help! Ahasuerus 01:00, 6 February 2011 (UTC)
FWIW, Google Books appears to have the 1949 Prime Press edition and it has "time-slip" on pages 7 and 10. I don't know if 3 pages earlier counts as a prior use, though. BLongley 01:14, 6 February 2011 (UTC)
No, the date is what counts -- but the extra citation might be interesting anyway; it could be a slightly different usage, for example. So it's often worth spotting those things. Mike Christie (talk) 03:29, 6 February 2011 (UTC)
Development
I thought I'd better check whether I can still remember how to develop, so made a small FR change. Please do let me know if I've got any of the process (or indeed the coding) wrong. BLongley 19:31, 15 February 2011 (UTC)
Thanks for the heads up! SourceForge has a notification system which is supposed to send e-mail updates whenever a script is checked in, but it looks like it was broken when CVS was restored after the hacker attack. I will review and test the code later today. Ahasuerus 19:52, 16 February 2011 (UTC)
There's not a lot to review - there's more changes made to this wiki than there is to the code! I thought I'd better re-start small - I do need to get back in practice but I'm not confident of creating entire new pages or suchlike yet. There's still a few minor annoyances within my abilities though. BLongley 21:26, 16 February 2011 (UTC)
Everything was fine; the code is now live. Thanks! Ahasuerus 04:05, 18 February 2011 (UTC)
Checked in another minor FR and got this:
Generating notification message... Traceback (most recent call last): File "/cvsroot/sitedocs/CVSROOT/cvstools/syncmail", line 433, in ? main() File "/cvsroot/sitedocs/CVSROOT/cvstools/syncmail", line 426, in main contextlines, fromhost, replyto) File "/cvsroot/sitedocs/CVSROOT/cvstools/syncmail", line 223, in blast_mail conn.connect(MAILHOST, MAILPORT) File "/usr/lib64/python2.4/smtplib.py", line 306, in connect raise socket.error, msg socket.error: (111, 'Connection refused')
Is that a problem my end? BLongley 14:32, 19 February 2011 (UTC)
It's the same error that I got when I committed a change last week. According to the Sourceforge blog, "outbound email is not functional, so any script utilizing this will fail. If you need to send outbound email, a workaround would be to use curl and and external site or Project Web to send the email." This is unfortunate since it means that other developers won't be notified when a change is committed. For now, I am checking the Development page and the CVS repository, but we'll need to do something long term. Ahasuerus 20:00, 19 February 2011 (UTC)
Looks like mail is working now, as I just checked in the remains of the "Dup Pub Tag Generation" fix and got the email. I didn't create a new bug, but can't reopen the old one - can you oblige? BLongley 17:03, 28 February 2011 (UTC)
Done! Ahasuerus 17:23, 28 February 2011 (UTC)
By the way, while testing this bug I got different results recreating the problem on the live server and my local copy. Locally I get a python error when the Pub Tag gets over 32 characters, on the live server it truncates painlessly and creates the duplicate. Is this a MySQL setting or version difference? BLongley 17:14, 28 February 2011 (UTC)
Sounds like a version difference, but I'll have to check... Ahasuerus 17:23, 28 February 2011 (UTC)
(unindent)I too noticed and realized check-in errors were some SF issue with mail and now that you mention that I do recall some comment about outbound mail not working. It is good to know it is working again.
http://code.activestate.com/recipes/310322-generating-a-rss-feed-from-cvs-commits/
Instead of using mail perhaps an RSS feed is a better idea? Uzume 13:08, 1 March 2011 (UTC)
The good news is that I have received an e-mail notification from a change committed by Bill earlier today. The bad news is that SourceForge may be discontinuing their CVS service in the foreseeable future, in which case we will need to migrate to SVN or something similar and the issue will be moot. I am going to check what SourceForge's stated intentions are and check with Al to see what he thinks. Ahasuerus 23:49, 2 March 2011 (UTC)
Continued Fixer discussion
When you get a chance could you take a look at the further comments I added to this discussion on Fixer's page? I'm not sure if you may have missed it in the intervening days. Thanks. Mhhutchins 19:08, 28 February 2011 (UTC)
Bug 1820414: Page numbers are lost
Ive just checked in a fix for this - slightly meatier than recent changes as it'll need testing for various title types to make sure I haven't broken Container-types while fixing Shortfiction. I thought I'd let you know directly as you don't seem to have got the email last time, although your sourceforge e-address is also on the email I got. However, Uzume doesn't appear to be getting notified so there may still be something wrong. BLongley 19:25, 2 March 2011 (UTC)
I just got a notification, so it looks like they are once again available -- hopefully. Ahasuerus 00:38, 3 March 2011 (UTC)
Hope you get well soon
BLongley 22:16, 12 March 2011 (UTC)
Thanks! I am mostly alive, but it's hard to do anything CPU-intensive (like testing) when you are running on 2 cylinders. Except mixing metaphors, apparently ;-) Ahasuerus 00:09, 13 March 2011 (UTC)
Latest FR
Not sure where I'm going wrong, but I tried to add some files to kick-start the "FR 3136359 Data Cleanup Scripts". I worked on mod\isfdblib.py 1.13, added mod\cleanup.py 1.1 and mod\es_search.py 1.1 and mod\ve_search.py 1.1 but couldn't adjust mod\TARGETS for some reason. Is that a generated file rather than something we should modify directly? BLongley 20:07, 18 March 2011 (UTC)
"TARGETS" files are regular files as far as CVS is concerned, so they should be updatable. I'll poke around in the morning and see if I can get it to work. Ahasuerus 04:03, 19 March 2011 (UTC)
I tried re-editing it today and got it to work this time. I think I had a stray swap file around causing CVS not to spot any changes. BLongley 14:54, 19 March 2011 (UTC)
Oh, and the two scripts I added are rather crude and could certainly be improved with followup "100-199" etc pages and parameter searches, but I thought I'd just make a start. It may turn out that once we've fixed the worst offenders we won't need follow-up pages anyway, but feel free to improve on the stubs. BLongley 20:12, 18 March 2011 (UTC)
Looks like a good start! Ahasuerus 04:03, 19 March 2011 (UTC)
Amidst all the April-Foolery, there was a serious question - how should people indicate their desire for new features/fixes to be prioritised for testing? Mike seems to want "My primary verifications" although that's currently last on the list. And no doubt some of the Data-Cleanup search scripts are of more interest than others, although I appreciate I lumped a load of them together. BLongley 16:05, 2 April 2011 (UTC)
I haven't been able to spend much time on testing lately, but I will be concentrating on it this weekend. Let's see where we are by Sunday night and then re-prioritize what's left. Ahasuerus 17:12, 2 April 2011 (UTC)
That really doesn't answer my (well, Mike's) question. How should people indicate what they want moved up in priority? You are by default the arbiter of all potential software improvements, so how should we notify you of of any particular change's desirability? On Sourceforge, Development page here, or bug you directly? BLongley 19:41, 2 April 2011 (UTC)
Apologies, I misunderstood the question. I'll try to answer later today -- in a hurry right now... Ahasuerus 17:26, 3 April 2011 (UTC)
No worries - I understand we all have other pressures. I've just finished one of mine (Pool Team captain for the Monday Night League - we missed the play-offs by one point) so might be even more active on ISFDB now until the next season starts. If you want to keep me busy on other stuff, re-enable the Award Editing tools! ;-) (I presume you know how, but if not I can help with that.) BLongley
Yes, I can activate award editing easily. I am thinking that I should simply enable it for you and me for now so that you could play with it on the live server. As you said, it seems reasonably stable and you could whip up the UI into shape by adding drop down lists and such reasonably quickly.
As far as prioritization goes, I have a list of things that I planned to do in 2011 (including various dependencies), but I haven't had the time to work on it. I am afraid that at the moment I am (or rather my lack of availability is) our biggest bottleneck. I am typically sporadically available for policy discussions and occasional edits during the day, but most days I can't access the development server until around midnight, at which point I am too tired to do anything serious. Hence the recent emphasis on Fixer, which I can run in my sleep (often literally.) I have a big deliverable on Tuesday/Wednesday and then I should have more free time. The smart thing to do would be to retire and concentrate on ISFDB, but there are always seemingly good reasons why I need to stay on for another X months...
Anyway, to get back to the issue at hand, there are some dependencies. For example, you probably don't want to change the way the Summary page behaves until we get rid of the code that handles "lexical match for serials". It's been inactive for well over a year now, but we still need to remove all of its vestiges before we can proceed with anything else. I also have a Unicode fix in the works (listed on the Development page) that I wanted to install before improving foreign language support, but there are still issues with it. Other than that, I can't think of any significant dependencies.
Back to prioritization. We have a "Priority" field over on Sourceforge, but it hasn't been used much. The only high priority request that I recall from the last 2 years is Swfritter's Support for multiple web links at title level. It's a fairly simple request except for two things: (1) the "alias" capability is probably more work than we want to undertake at the moment and (2) it will require changes to the JavaScript generator, which is a royal pain and probably needs to be rewritten as regular js files first. Do you know enough JS to give it a shot? Ahasuerus 05:16, 4 April 2011 (UTC)
I gave it a stab - you're right, the Aliasing can wait, and I avoided the Javascript issue. (I presume that relates to adding an "Add Webpage" button? I just allowed for one extra webpage to be added in any title edit.) I'm afraid I did mess up the checkin slightly and all the comments have a typoed FR number on, but functionally it should be OK. I'll see if I can persuade swfritter to test it out himself, as it his request. BLongley 17:19, 6 April 2011 (UTC)
BTW, I know that the new feature has a similar bug to Bug 3267292, but we've lived with that for ages so I didn't worry about it too much. BLongley 17:33, 6 April 2011 (UTC)
And I really do NOT mind if you reject my changes as long as you explain why. I'm happy to learn more Python or MySQL as necessary - your changes to avoid Global Variables taught me something new and I appreciate that. BLongley 19:41, 2 April 2011 (UTC)
Award Editing changes
I created Bug 3272198 but I'm not sure of the best way to progress in fixing it. The "Remove Award Information from This Title" option seems to be intended to allow mass-removals: however, if you go via "Edit an Award", select an Award, then "Delete record" you can enter a deletion reason, which looks safer. Do you think we should fix the mass removal, or remove the option and force people to delete awards one by one? BLongley 16:32, 3 April 2011 (UTC)
Publisher Bug?
See here. Any ideas why this publisher is still around when there are no pubs left? I'll check myself after the next backup is posted, but only you can actually look at the recent submissions that caused it - 1576631 and 1576630. (I've since further corrected the publisher to "Love Spell" with a space, but I don't think that has anything to do with the problem.) BLongley 17:53, 5 April 2011 (UTC)
Ah, seems there is evidence in the current backup: this pub still exists. 8888-00-00 title with a 9999-00-00 pub date. I guess it's a display bug rather than a publisher bug. BLongley 21:13, 5 April 2011 (UTC)
Yes, we may need to add special cells to the HTML table for 8888-00-00, 0000-00-00 and 9999-00-00. Ahasuerus 06:15, 6 April 2011 (UTC)
OK, Bug 3277692 created. BLongley 17:41, 6 April 2011 (UTC)
And hopefully now fixed. BLongley 20:32, 8 April 2011 (UTC)
In two cases, the HTML was slightly off -- <a> instead of </a> -- so I cleaned it up, consolidated the new HTML code and installed version 1.10 on the live server. We are getting there, one script at a time :-)
Thanks - I see you also reworked it for exceptions. Probably a good idea, I'll remember that for future changes. BLongley 00:27, 11 April 2011 (UTC)
BTW, have you tried running Web pages by an online HTML validator? Web browsers are very forgiving these days and may not warn you about malformed HTML. Ahasuerus 00:12, 11 April 2011 (UTC)
I used to - my old home page I once bothered with even linked to one. I guess I should add that to my testing, as there's so many browsers around now I can't be bothered to test with them all. We really could do with some more ISFDB testers, couldn't we? BLongley 00:27, 11 April 2011 (UTC)
But is there actually any reason for 9999-00-00 dates to appear in the database? I know it's used in the display software to sort unknown dates last, but I don't recall any reason for it being permittable at any other time. BLongley 21:13, 5 April 2011 (UTC)
9999-00-00 stands for "forthcoming" while "0000-00-00" represents "unknown". In this case the Title is still supposedly forthcoming, at least as an e-book. On the other hand, the Pub date has been set to "8888-00-00" (unpublished) because if the book ever appears, it will be with a different ISBN.
I guess we may want to add a way to identify and review "forthcoming" Titles once in a while. Something like "Last Dangerous Visions" was at one point "forthcoming" but eventually became "unpublished". Ahasuerus 06:15, 6 April 2011 (UTC)
FR 3279968 Add options to search title and publication notes
Sorry to increase your testing workload again. :-/ I just thought it's worth a note that I found it really necessary to create another index on the pubs table, and to use hints to ensure it was used. I'm not entirely sure WHY, as direct SQL seemed to work fine but the ISFDB version didn't work as well despite generating the same SQL. Any hints (no pun intended) as to where I am going wrong would be appreciated. BLongley 23:42, 7 April 2011 (UTC)
I'll take a look over the weekend, but I am getting worried that what we have available over on SourceForge is getting to be increasingly different vis a vis what we have installed on the live server. As long as it was just a few scripts getting out of sync once in a while, it wasn't too bad, but now the difference is quite pronounced. It will be a pain to rebuild ISFDB from what we have at Sourceforge if we ever need to or if a new developer comes along.
I think we need to change the process so that new and modified scripts would be sent directly to me. Once tested and installed on the live server, they will be committed to Sourceforge. That way the two copies of the software will always be the same (knock on wood.) Ahasuerus 03:21, 8 April 2011 (UTC)
That's not really the way CVS is supposed to be used. You should be able to tag the whole set of live code after each implementation, so that a developer or tester can use that tag to pull out the current live set rather than the current development set, if desired? BLongley 15:57, 8 April 2011 (UTC)
Yes, it's possible to use tags to create internally consistent sets of script versions. Al and Roglo did it a couple of times with "archive_YYYY-MM-DD" tags in 2008 and 2009 -- see versions 1.1 ans 1.5 of this script for examples. The easiest way to do this is to reconcile all scripts at SourceForge with what we have installed on the live server and then to tag the latest versions of all scripts with an "archive" tag. If the two versions are out of sync as they are now, tagging the right version of each script becomes harder and more error-prone. If the live server dies tomorrow and the ISP can't find the backups, rebuilding what is currently installed will be a royal pain. Ahasuerus 23:40, 10 April 2011 (UTC)
What you're proposing looks as though it will enforce the bottlenecks even more, and I think sending you changes directly would increase the risk of the Windows/Linux problems occurring again. BLongley 15:57, 8 April 2011 (UTC)
If a script is sent as an attachment, then copying that attachment on top of an existing script will mostly likely cause an LF/CR problem. I made that mistake once and it took me hours to figure out where I had gone wrong. The good news, however, is that if you open the attachment in Notepad and then copy the text to the currently installed script, you won't have a problem. Ahasuerus 23:40, 10 April 2011 (UTC)
Sorry (again), but I've been coding some more and have stretched my knowledge of ISFDB code a bit further. Do you think we should call out for more testers? BLongley 00:05, 10 April 2011 (UTC)
Marty is good at code reviews and he has also done some testing in the past, but he is usually very busy with other things. Normally, I would be able to process this volume of changes, but I was very busy for a while and now that I am no longer particularly busy, my heart is apparently acting up. We just can't get a break... I am feeling somewhat better right now, so I will try to test and install a couple of changes tonight. I'll go harass my doctors tomorrow and we'll see where it takes us. Ahasuerus 23:40, 10 April 2011 (UTC)
Sorry, I missed this update till I added the next section - I hope you're actually better, rather than just feeling better. I had heart worries a year or two ago, but the ECG wasn't too bad and I seem to be getting by with seven pills a day for blood pressure and cholesterol problems. I may yet live long enough to exhaust my savings :-/ BLongley 02:04, 12 April 2011 (UTC)
The doctors think that my heart should be OK for a while, but the lungs seem more problematic. To quote an old joke, the autopsy will tell whether are were right :-)
It's easy to joke about health and longevity because we have little control over them, but outliving one's savings is a different story. Hopefully the work you are doing on ISFDB can be, as you mentioned, leveraged elsewhere. BTW, you may want to gather a collection of quotes showing that ISFDB is a well respected resource, e.g. Wikipedia points out that Rails For Java Developers (a free PDF) uses ISFDB's schema as an example. Ahasuerus 04:38, 12 April 2011 (UTC)
Thanks for the pointers. I really must look for paying work soonish: I've enjoyed my time on ISFDB despite the 12-hour days, 6 or 7 days a week. And no doubt I'll still keep my hand in even if I have to go back to mundane 'save the company a million pounds a year' or "prevent the executives going to jail" stuff. I got myself into a bit of a dead-end by being the best in the country at what I do, then losing that job, when there's only five employers in that field. I'm pretty sure one of the other four would take me on but I don't want to move house yet again - I haven't finished unpacking the books from the last move yet! BLongley 05:03, 12 April 2011 (UTC)
FR 2811812 Capture reason or summary of an edit
For now, I've submitted just the two changes that I think Fixer uses - AddPub and NewPub. Having done so, it suddenly occurred to me that Fixer probably doesn't use the submission forms so you could get away with just implementing the moderator module pv_new.py at first, and teaching Fixer to submit "<ModNote></ModNote>" sections as well as "<Note></Note>" ones. That gives the Moderators time to learn the changes before we find out how mere humans will abuse the new feature. BLongley 01:56, 12 April 2011 (UTC)
There is some more good news - it looks like FR 2800730 is a duplicate that can be closed off. So you can clear one FR without any testing! ;-) BLongley
Oh, and re: "BTW, if we are messing with the submission table anyway, we may want to add a new field for new pub ID" - I agree, but I haven't needed to mess with the submission table for this. Most of the "Recent Submissions" links come from the XML data in the submissions table, where the ID is already in the submission - so it might be more consistent to update that field? (I'm not really sure how you feel about more XML versus more DB fields.) BLongley 02:30, 12 April 2011 (UTC)
I would be hesitant to update the XML (i.e. sub_data) after the submission has been created. After all, it's supposed to capture what the submitter originally entered in the browser form. On the other hand, we added a new field (sub_holdid) to the submissions table a couple of years ago and it seems to be working fine, so adding an extra field shouldn't be too hard. Ahasuerus 04:44, 12 April 2011 (UTC)
OK, I'll look into that. BLongley 05:13, 12 April 2011 (UTC)
However, one quick win we could make is to link to dumpxml.cgi for ALL submissions on the recent integrations list, apart from the Author Update which already has code to deal with it. That would allow people to go back and see the transient notes if ever required. What do you think? BLongley 02:30, 12 April 2011 (UTC)
Raw XML will look something like this:
<?xml version="1.0" encoding="iso-8859-1" ?> <NewPub> <Submitter>Fixer</Submitter> <Parent>1105307</Parent> <Subject>The Prophecy Keepers</Subject> <Title>The Prophecy Keepers</Title> <Year>2008-11-28</Year> <Publisher>Empyrean Hill Books</Publisher> <Pages>320</Pages> <Binding>hc</Binding> <PubType>NOVEL</PubType> <Isbn>9780982049600</Isbn> <Price>$15.99</Price> <Image>http://ecx.images-amazon.com/images/I/41hviyN%2B9lL.jpg</Image> <Note>Data from Amazon.com as of 2011-04-09. MODERATOR NOTES: Detailed information available <a href=http://www.amazon.com/exec/obidos/ASIN/0982049609>here</a> Reasons for inclusion - Browse Nodes: Science Fiction & Fantasy->Fantasy->Epic Science Fiction & Fantasy->Fantasy->Magic & Wizards Additional Browse Nodes: Literature & Fiction Reasons for inclusion - SF subjects: Fantasy Fantasy - Epic Fiction - Fantasy Fiction / Fantasy / Epic Science Fiction & Fantasy </Note> <Authors> <Author>Melaine Bryant</Author> </Authors> </NewPub>
which will be unreadable by most users. I am thinking that a new field will be the right way to go in part because it's better to have a fool-proof way to link submissions to the pubs that they created. Ahasuerus 04:50, 12 April 2011 (UTC)
That will be good for general update checks, but if we go with discardable notes then we need to make the raw submissions more readable. I can work on history.py and recent.py but that will take some time, whereas dumpxml is easy to enable in the meantime - although obviously not ideal in the long term. BLongley 05:13, 12 April 2011 (UTC)
More Data Cleanup scripts?
Mike Hutchins is asking for more, based on the "Data Consistency" work we had in the past. I've checked out a few of them where the SQL was supplied and it looks like we could usefully add "Titles that point at themselves", "Titles that point at variants", "Titles which are variants of non-existent titles", "Publess REVIEWs and INTERVIEWs" and "Series with variant titles only" without performance problems. However, you raised concerns about Data Cleanup Scripts Part 3: Shall I reduce that to "Find Extra Editor Records" and we can leave "Unused Publication Tags" till later when we can figure out how we want to clean them up? (There's no point finding them if we can't fix them.) BLongley 17:33, 15 April 2011 (UTC)
I was just about to ask you to delete the scripts that were committed when you were experimenting with tag deletion! Ahasuerus 18:14, 15 April 2011 (UTC)
Will do, but not right now. A) I want to save them aside as that's the first time I've managed to create a new submission type, and B) I haven't needed to delete anything via CVS for five years or so, and can't recall how to do it off-hand. :-/ Will probably figure these out tomorrow. BLongley 00:12, 16 April 2011 (UTC)
Mike's listed his desires on his talk-page, and I've listed my perceived performance problems on mine - feel free to butt in on either. (I think you have some off-line scripts that we could turn into online ones, so feel free to add those to the mix.) BLongley 00:12, 16 April 2011 (UTC)
OK, I think I've reverted all the tag deletion changes. I've removed the new scripts, but for changed scripts I've had to create new versions: biblio\tag.py 1.7, common\isfdb.py 1.31, and edit\TARGETS 1.15. I think an Admin could delete the unwanted version and the reverting version above that, if that's preferable. BLongley 18:06, 16 April 2011 (UTC)

I've also added five scripts that seem to perform well enough. (It would have been six, but "Variant Titles of Variant Titles" also covers "Titles that point at themselves".) Happy testing! ;-) BLongley 18:06, 16 April 2011 (UTC)
looks good, I will poke around. Thanks! Ahasuerus 22:43, 16 April 2011 (UTC)
Tales of Riverworld
I added the author's note for Farmer's "Crossing the Dark River" to your verified Tales of Riverworld. --Willem H. 10:57, 16 April 2011 (UTC)
Find Stray Publication Authors
I think I've fixed it to avoid most mismatches. Trying to get it to still cope with NOVEL/NONGENRE made it perform very badly, so I did those manually and have left them for now. I think people are working from the project page rather than the script, so there's probably no urgency on this. BLongley 17:10, 24 April 2011 (UTC)
Thanks, it looks better. Installed. Ahasuerus 19:56, 24 April 2011 (UTC)
Cheers! I hope this means you're feeling a little better? BLongley 22:20, 24 April 2011 (UTC)
Not by a whole lot. It's been two weeks now, so I'll go harass another doctor in a couple of days. As John W. Campbell once said, nobody is getting any younger... Ahasuerus 00:11, 25 April 2011 (UTC)
And Campbell had the chance to read "The Curious Case of Benjamin Button" first time around. :-/ I'm no worse than previously, but with Summer attacking I'm probably more house-bound than I was in the winter - just hay-fever, not Triffids, fortunately. BLongley 00:34, 25 April 2011 (UTC)
OK, it looks like it was just infection and it seems to clearing up, so if everything goes well, I should be back in business in a day or two. (Famous last words :-) Ahasuerus 02:44, 27 April 2011 (UTC)
Recent Rejects broken?
I know we don't have many types of update coded for, but Author Updates usually get a link. And we do indeed have links for submissions 1590506 - Kevin Emerson, 1573239 - Barry N. Malzberg and 1576341 Charlotte Stone. But those don't work: "No history records available for this submission". Those were rejected though, and submissions 1590561 C. C. Humphreys, and 1590556 Chris Humphreys were accepted and seem to work. I presume there's a bug with Rejected versus Approved Author Update submissions? (No hurry on this, there's a lot of stuff on Sourceforge to look at, as well as recent cleanup projects.) BLongley 22:54, 24 April 2011 (UTC)
It looks like the Reject logic may not be updating the History table. In a way, it makes sense since the History table, as I recall, keeps track of each "before" and "after" state, but the two are identical if a submission is rejected. Ahasuerus 00:13, 25 April 2011 (UTC)
Understandable then. I think I'll look into small but complex changes next - I think you've got quite enough on your plate already, and I should learn some of these more obscure areas. All the stuff that is not available in the backup has put me off a bit, as I can't be sure if I'm recreating a realistic set. I've just obliterated 300+ Submissions that I created for local testing, for instance, to provide the latest projects. I should probably learn how to save those aside for later if I'm going to improve those aspects. BLongley 00:44, 25 April 2011 (UTC)
Bond's Lancelot Biggs: Spaceman
Can you confirm that the author of this pub is credited as "Nelson S. Bond"? Thanks. Mhhutchins 03:45, 28 April 2011 (UTC)
Good catch, there is no "S." anywhere. I must have verified a pre-existing pup without checking. Also, it reads like a fix-up, with individual stories converted to chapters. I have most of the original pulps, so I could probably check, but it would take a very long time since my pulp collection is, um, suboptimally organized. Ahasuerus 05:02, 28 April 2011 (UTC)
You're right. All of the pub records but one gave "S" in the author's name, but research showed me that none of the pubs actually had it, and no research uncovered that odd little pub in your possession. I think we also need to reconsider that it's a collection. There is more than enough evidence that it should be treated as a fix-up novel. We can give the constituent parts in the title record like we handle other fix-ups. I'm not sure it would be necessary for you to check the text against the original pulp appearances, if we convert the pubs to fix-ups. Thanks. Mhhutchins 06:25, 28 April 2011 (UTC)
BTW, it was Bill's "Stray Author" script that uncovered the mismatch. If that single pub had been incorrect (had an "S" middle initial), it would not have shown up on his list, then I would not have looked at the pub record and saw the discrepancy. Mhhutchins 06:28, 28 April 2011 (UTC)
Not well yet?
Thanks for the small change tested and installed. I appreciate there's quite a backlog now, and I don't want to rush you into putting things live that aren't ready - but I did notice that some of the desires mentioned on User:Ahasuerus#Data_Cleanup_and_Scripting are still not covered, and might be within my abilities. If it would help, let me know which are most desirable in your eyes and I'll see what I can do about giving Mike and Willem etc some other stuff to work on. ("Stray Authors" has begun to frustrate me and I'd really like the software improvements for that project to go in before I continue on it - but as you know, there's always something else to do here!) BLongley 00:31, 30 April 2011 (UTC)
Well, the good news is that the doctors think that they finally have a fairly solid diagnosis. However, the first antibiotic was not working, so now I am trying a different one. If it works out, I should be more functional in the next day or two -- fingers crossed.
As far as cleanup scripts go, I don't have a preference. You, Mike, Kraang, Willem and other folks who handle these issues on a daily basis are probably better judges of what's more important.
Re: "Stray Authors" improvements, are there any that have been checked in and haven't been installed? Ahasuerus 03:44, 30 April 2011 (UTC)
No, there's no further improvements to the Stray authors script, but the unmerge improvement "FR 3294454 Unmerge Titles should preserve publication authors" and fix "Bug 3293839 Unmerge titles offers you the chance to unmerge Variants" would make dealing with the results easier. BLongley 13:38, 30 April 2011 (UTC)
Ah, I see! OK, I will work on them next although I am a little hesitant to do anything non-trivial until I feel better. You never know what kinds of bug you may miss when you can't concentrate for more than 10 minutes at a time. We'll see how it goes... Ahasuerus 18:02, 30 April 2011 (UTC)
OK, I have tested and installed the first one. It can occasionally result in an empty table when the only title record is for a VT, but we can add an error message later. The second one is in the pipeline. Ahasuerus 06:10, 2 May 2011 (UTC)
The other unmerge script has been tested and installed as well. Sorry it took so long, but you are generally more SQL-savvy than me, so I spend more time testing non-trivial SQL changes that update the database. Moderator scripts are much less intensive since even if they go awry a bit, it doesn't affect the underlying data.
Anyway, the new antibiotic is having an impact, although it's not as effective as the doctors hoped. I even managed to write a new script during the day. It finds what stories (and only stories) have been reprinted for a particular author, e.g. for Robert Abernathy it found:
1 Heritage (1942) 2 Peril of the Blue World (1942) 0 Mission from Arcturus (1943) 1 Saboteur of Space (1944) 1 The Canal Builders (1945) 0 When the Rockets Come (1945) 1 Failure on Titan (1947) 0 Hostage of Tomorrow (1949) 0 The Giants Return (1949) 1 The Dead-Star Rover (1949) 1 The Ultimate Peril (1950) 0 The Tower of Babble (1950) 1 Strange Exodus (1950) 0 Righteous Plague (1951) 0 The Four Commandments (1953) 3 The Rotifers (1953) 0 Lifework (1953) 0 The Captain's Getaway (1953) 0 Professor Schlucker's Fallacy (1953) 2 Axolotl (aka Deep Space, 1954) 0 Tag (1954) 0 Deep Space (1954) 0 The Record of Currupira (1954) 0 The Firefighter (1954) 0 When the Mountain Shook (1954) 0 The Thousandth Year (1954) 4 Heirs Apparent (1954) 5 Pyramid (1954) 0 The Marvelous Movie (1954) 0 The Fishers (1954) 2 Single Combat (1955) 0 World of the Drone (1955) 0 The Guzzler (1955) 4 Junior (1956) 0 The Year 2000 (1956) 1 Grandma's Lie Soap (1956) 0 One of Them? (1956) 0 The Laugh (1956) 0 Hour Without Glory (1956)
It only counts unique reprints, so if a 1950s anthology went through six printings, it will only count it once. I am thinking that if we clean it up (e.g. it currently doesn't handle VTs well), we could put a link to it on the Summary page as a "value added feature".
We could do something similar for magazines to show how much of a given magazine's fiction was later reprinted. Perhaps add a breakdown by year, an overall percent of reprinted stories, etc. Do you feel like working on it in your plentiful spare time? If you do, you'll have to start from scratch since my script was "proof of concept" and uses Fixer's code and structures, which are not Python/SQL. Ahasuerus 06:20, 4 May 2011 (UTC)
Glad to hear you're improving! I made a start on the SQL: this gives the same counts as yours, with the exception of "Axolotl".
select t.title_title, count(distinct t2.title_id) - 1 from authors a, canonical_author ca, titles t, pub_content pc, pubs p, titles t2, pub_content pc2 where ca.author_id = a.author_id and ca.title_id = t.title_id and t.title_ttype = 'SHORTFICTION' and pc.title_id = t.title_id and pc.pub_id = p.pub_id and pc2.pub_id = p.pub_id and pc2.title_id = t2.title_id and t2.title_ttype in ('EDITOR','ANTHOLOGY','COLLECTION') and a.author_canonical = "Robert Abernathy" GROUP BY t.title_title
I'm not sure how much complexity you mean by "doesn't handle VTs well" - should there be an entry for "Deep Space" if it's a variant? If the Container Title goes through variations, should they still be counted as one edition ("unique reprint")? BLongley 17:08, 4 May 2011 (UTC)
(unindent) Here is the logic that I had in mind:
Find all canonical (i.e. no VTs) Shortfiction title records for a given author

For each Title record identified above, find the pubs that it and its VTS have appeared in. Skip Project Gutenberg and possibly other publishers

For each identified pub, find the pub's "reference" title which matches the pub's type. Exceptions: if the pub type is MAGAZINE or FANZINE, get the EDITOR title

If this reference title is a VT, replace it with its canonical title

If this is the first time that this reference title has been found, increment the counter for this canonical title. If this is NOT The first time this reference title has been found, skip it
The end result will be a count of "unique reprints" for each title, which we can use to derive stats, e.g. "N% of this author's stories have been reprinted", and reports, e.g. "List of stories sorted by reprint frequency". Do you think this logic will work? Ahasuerus 06:13, 5 May 2011 (UTC)
Step 5 looks like the logic is the wrong way round as to when to increment, but I see the sense. "Librivox" is another publisher we probably want to skip. I guess "unique reprint" will depend very much on how and why we've created variants - I was wondering whether "distinct Pub Title" would be enough to eliminate re-printings. I assume the aim is to match Bill Contento's list? (I find it amusing that his number one finding is wrong - it's "A Sound of Thunder" not "The Sound of Thunder".) BLongley 14:43, 6 May 2011 (UTC)
I was thinking that it would be nice to show each author's most frequently reprinted stories and also what percentage of his/her stories have been reprinted at all, but we could also do it at the Title level a la Contento. If you encounter someone like Robert Abernathy, who most people have never heard of, Author level information will help you determine whether the author may be of interest. Magazine level data will help identify the most influential magazines. Ahasuerus 01:03, 7 May 2011 (UTC)
Speaking of magazines, how do we cope with titles like Victory? The British magazine reprint doesn't seem to me to be a unique reprint, but the magazines are not variants of each other. BLongley 16:13, 8 May 2011 (UTC)
Hm, yes, that's a bit problematic. I agree that ideally it wouldn't count as a reprint, but there are too many differences between the US/UK versions of _Astounding_ to make them variants of any kind. We could ignore all magazine reprints, but then we wouldn't be counting authentic reprint magazines... Ahasuerus 17:43, 8 May 2011 (UTC)
Well, we could always knock-out a simple starter script and see what people complain about most. It's always easier to get feedback on a change after it's been made, than before. Even when I post screen-shots of proposed improvements and try to explain them, 90% of the comments come afterwards. :-/ BLongley 22:19, 8 May 2011 (UTC)
Sounds like a plan! Ahasuerus 22:22, 8 May 2011 (UTC)
OK, I'll give it a stab tomorrow maybe. Tonight I've been pleased with feedback on the latest software improvements (thanks for those, they're going down well!), the latest clean-up project updates (almost done on yet another one!) but seem to be having major problems with the Series Sub-Ordering FR. I'm not sure why one simple addition is going me so many problems, so I guess I'll have to sleep on it. BLongley 22:50, 8 May 2011 (UTC)
Dynamic SF, October 1953
Can you check the record type for the contribution by Orban on page 65 of this issue? Looks like it should be interiorart. Thanks. Mhhutchins 20:06, 30 April 2011 (UTC)
Yes, indeed. Thanks! Ahasuerus 02:10, 1 May 2011 (UTC)
Bug 3293839
Are you sure this is installed? I'm getting different results on Live and my Test system, e.g.
Live:
Test:
Can you double-check which version of edit\tv_unmerge.py is in? BLongley 15:54, 4 May 2011 (UTC)
Ouch, that was pretty bad! I tagged mod/tv_unmerge.py rather than edit/tv_unmerge.py :-( I will fix it in a couple of hours -- thanks for catching it! Ahasuerus 02:45, 5 May 2011 (UTC)
OK, we are back in business. Sorry about that! Ahasuerus 05:51, 5 May 2011 (UTC)
No worries - I don't think anyone messed things up in the meantime, and even if they did then it was just down to the existing bug, nothing you or I did. The latest changes are really appreciated, we may yet finish "Stray Authors" before the next backup. BLongley 23:53, 5 May 2011 (UTC)
Elsewhere: I have to confess that I've gone a little overboard on ISFDB:Moderator_noticeboard#Some_little_things_to_make_your_life_easier - I've committed several changes without an FR or Bug reference. I think there's four improvements made with the main change, three more with individual changes, and one change that just makes it a little bit more standard. Apologies for being so disorganised - I'm trying to get feedback on what people want, as well as what they need, but better language support is something I'm not going to be able to do without clear direction. BLongley 23:53, 5 May 2011 (UTC)
I know it's not the right place to mention this, but I think I messed with common\viewers.py, mod\sv_update.py, mod\uv_merge.py, mod\kv_new.py, and I adjusted mod\pv_update.py. They can't be destructive scripts, so hopefully they won't be too controversial. I know there's a lot on the back-log of changes though, so feel free to tell me to go and report the changes correctly. That'll at least give me an excuse to not work on Language Support. ;-) BLongley 00:34, 6 May 2011 (UTC)
I have created FRs for these changes and added them to the development page, but please make sure to go through the motions in the future. It can be hard to reverse engineer an FR from the code. Ahasuerus 04:08, 6 May 2011 (UTC)
The Yawning Heights by Alexander Zinoviev
An obituary in Locus July 2006 for this author prompted me to add this "fantastic, surreal dystopia" into the database (here). Having zilch knowledge of Russian and how it should be entered, I copied and pasted the original title from a Wikipedia article. Can you take a look at it and make any corrections? Thanks. Mhhutchins 16:56, 4 May 2011 (UTC)
I have transliterated the title and I believe it looks OK now. I also changed "Kostroma Oblast" to "Kostroma region" since it didn't become an "oblast" until after Zinoviev was born. BTW, "oblast" is one of those slippery terms which mean different things in different places. And yes, the book is a (rather bizarre) dystopia, so it's "in". Thanks for digging it up! Ahasuerus 01:11, 5 May 2011 (UTC)
Thanks. I knew if it had anything to do with Russia you'd know how to handle it. Mhhutchins 02:22, 5 May 2011 (UTC)
We all have our areas of expertise :-) You probably know more about Locus and SFBC than anyone else around here and Bill has become something of an expert on UK paperbacks. I just wish I had more time to work on my other areas of expertise like my pulp collection. Ahasuerus 02:54, 5 May 2011 (UTC)
I'm not sure I've "become something of an expert on UK paperbacks" - I just may have scared off all the other UK editors. :-/ I know what you mean about "wish I had more time" though - even when I send off "Rhodaniens" to another editor, there's at least ten feet of other magazines/fanzines to look at. BLongley 00:13, 6 May 2011 (UTC)
Damien Graves
I've tracked down the remaining Midnight Library books by this house-name, and taken the variants as far as I can. However, it seems that the books mostly appeared first in the UK as by "Nick Shadow". Is Fixer aware of these? The late discovery of prior publications is depressing me a bit, I really would like less work / rework. :-/ BLongley 17:29, 8 May 2011 (UTC)
The Nick Shadow versions were published in 2005-2007 while Fixer has only created submissions for 2008-02 through 2011-06. We are going back in time, though, and will pick them up in due course of time. Once Fixer finishes submitting 2008-01, I will ask him to do 2011-07 and then we'll do 2007. Of course, we will always have lacunae, but we'll do our best to minimize them :-) Ahasuerus 23:51, 10 May 2011 (UTC)
Advanced search changes
Thanks for looking at FR 3279968 "Support for Title and Publication Notes search" and FR 3089039 "Add support for URL search to Advanced Search" - I think Marty's changes, when completed, will be good. (Does that make me the official tester?) I think he's also laid the groundwork to fix Bug 3300042 "Publication Search for Cover Artist doesn't work" - I now know how to fix the SQL, and get it to perform acceptably, I think. I'm just a little confused about who's looking after these changes now - should I merge the FRs and add Marty's fix, or let Marty or you do it? BLongley 21:23, 10 May 2011 (UTC)
If you could take the latest iteration of Marty's changes, merge them into your code and reconcile everything, that would be great! I will then take the final result and test it. To make sure that the CR/LF problem doesn't reoccur, you ay want to copy-and-paste the source code from the attachment straight into the Python editor. The mismatch only occurs if you save the attachment on top of the existing file. Thanks! Ahasuerus 23:54, 10 May 2011 (UTC)
I will move the little titleInfo class out of pp_search.py and clean up a little and then check those things in. Then I'll make the same changes to pa_search.py and then to the other searches. May as well be consistent.... I will fix #3300042 while I'm at it, if someone with sufficient privileges would assign it to me. :-) --MartyD 00:23, 11 May 2011 (UTC)
Done -- thanks! Ahasuerus 01:04, 11 May 2011 (UTC)
Ok, I checked in pp_search.py, pa_search.py, local.mk, and a new tableInfoClass.py. One thing I ran into is that WITH the hints, performance goes to pot on my machine, so what I have checked in has the hints approach commented out. This behavior matches what I saw last night, although then I thought I was able to get good performance with the hints in place, while tonight I am not able to do so. I am running MySQL 5.1.56-community -- I'm not sure if version matters. You two can decide what you'd like to do about it. I tried combinations of notes, author, title, cover artist, and all seemed to work ok for me. I also verified next-paging. I did not yet touch the other two searches, but I will modify them to use the same tableInfo approach if this set passes muster. Thanks. --MartyD 01:57, 11 May 2011 (UTC)
p.s. Since my system is behaving differently from Bill's, you gentlemen should see about which USE INDEX and IGNORE hints are appropriate. Also on the coverart searching -- I did not add any hints there, and perhaps some could be useful. --MartyD 12:26, 11 May 2011 (UTC)
I'm on "5.0.51a-community-nt MySQL Community Edition (GPL)", so I guess version could be important. Ahasuerus, what are the versions of your test system and the live system? BLongley 13:02, 11 May 2011 (UTC)
The test system is "5.0.22-community-nt". The live version is "5.0.45 Source distribution". Performance doesn't seem to be too bad, but anything that has to do with ORs is still shaky. I wonder how often our users take advantage of it anyway? I'll ask on the Community Portal... Ahasuerus 03:23, 12 May 2011 (UTC)
I'll look into regressing my system to the test or live versions, as I'm still finding the unhinted versions of these scripts can really kill my PC. If either of you have tips on running multiple versions of MySQL on the same machine, please let me know. I have several more clean-up scripts that I haven't submitted as they are minutes rather than seconds to run, but they may do rather better on the live server. I'm just not confident that we're testing performance as well as we might. BLongley 03:16, 14 May 2011 (UTC)
No useful hints from me, I am afraid. The only thing I know is that when I tried to migrate the development environment to a 64-bit server running Windows 7, I couldn't get either MySQL-32 or MySQL-64 to work reliably after spending many hours on it -- backup restores consistently generated errors. It's most likely just a settings issue, but it goes to show that I am no MySQL expert. I should probably ask Marty about it since the current server is 8 years old and it's hard to test performance on something that out of date. Alternatively, I could set up Linux on the new 64-bit server and see if it works any better. Ahasuerus 21:27, 14 May 2011 (UTC)
Yes, I'd like Marty's input. My own PC must be almost as old as the server is - I should probably retire it and set it up as a dedicated Linux server - I used to do that automatically whenever I bought (or more often, built) a new PC, but funds are low at present. :-( BLongley 00:55, 15 May 2011 (UTC)
Latest Developments
I've added the Deletion changes to "FR 3042002 Support for multiple web links at title level" but would like some feedback on the way I've dealt with "Bug 3301635 Publisher merge may double-up Webpages" before I tackle any other merges. My thinking is that we shouldn't report differences in webpages as errors, we should just combine them all - but we shouldn't create duplicates. What do you think? BLongley 16:42, 13 May 2011 (UTC)
Hm. Yes, that's probably the best way to handle merging "multi-valued" fields. We do want to display them on the merge page, though, so that editors could see what URLs the merged record will have (and possibly edit them after the submission is approved.) Probably a simple table listing all URLs will do. Ahasuerus 00:14, 14 May 2011 (UTC)
I love the way you say "simple" at times like this. BLongley 03:08, 14 May 2011 (UTC)
You know how Fermat wrote that he had "discovered a truly marvelous proof of [Fermat's Last Theorem], which this margin is too narrow to contain"? :-) Ahasuerus 03:21, 14 May 2011 (UTC)
Yep - one of my favorite pubs (as in "public house", the drinking establishment, not "publication" as we use on ISFDB) had the news of the final proof posted on the noticeboard when it happened. Not your average pub really, which is why I miss it. BLongley 03:36, 14 May 2011 (UTC)
Does that mean you're going to do the changes and show me what you mean? If not, do you want clickable URLs or just text? Do you want differing URLs highlighted? BLongley 03:08, 14 May 2011 (UTC)
I am thinking that either clickable URLs or plain text will be fine for now -- whichever is easier. As long as the editor and the approving moderator can see what's happening, we can install the code and later improve it based on user feedback. I could certainly work on it on my end, but I have a long list of things to test at the moment. Getting ready to install "Capture reason or summary of an edit"... Ahasuerus 03:21, 14 May 2011 (UTC)
Tsk! Who's making you work so hard? Oh wait, that would probably be me. ;-) You do realise that if/when "Capture reason or summary of an edit" goes live, and Fixer uses it, I want to roll it out to many other edit types? BLongley 03:36, 14 May 2011 (UTC)
We'll definitely want to add it to all edit forms since it will help editor-moderator communications. And I am sure Fixer will love it when I show it to him! Ahasuerus 04:35, 14 May 2011 (UTC)
Three more waves of this submitted. The first two should be independent of other changes, the third is built on top of "FR 2823394 Improve handling of foreign language Titles (Part 2)". You might want to push these up the priority list, things like [1] seem to indicate they will be useful. BLongley 17:57, 14 May 2011 (UTC)
(Unindent) OK, a couple of changes made to "Bug 3301635 Publisher merge may double-up Webpages". The editor will see a list of all the webpages involved - not de-duplicated, but also not showing as a conflict. The Moderator will see all the webpages, but the ones in the "DropId" Columns will stay green rather than pink, to show that they will be kept. Will that do? BLongley 16:07, 14 May 2011 (UTC)
Thanks, I'll take a look shortly! Getting Fixer to use Mod Notes at the moment... Ahasuerus 20:32, 14 May 2011 (UTC)
That will indeed be time better spent - Fixer submissions seem to be really unpopular at the moment, and it's not just because of the Fairy books. And Mod Notes for Human use seems to be quite desired too, probably above the Title Webpages. The easier we make life for the Mods, the more time they get to look at other stuff. I hope. BLongley 20:45, 14 May 2011 (UTC)
OK, Fixer has been convinced to take advantage of Mod Notes. I also eliminated " " at the end of regular Notes since it no longer serves a purpose. Hopefully 75%+ of Fixer's submissions won't require editing Notes from this point on. The other 25% contain statements like "Edition statement: Reprint", which perhaps should be moved to Mod Notes as well. For now I'll work on testing your changes. Once I take a nap to pay off the sleep debt, that is :) Ahasuerus 21:09, 14 May 2011 (UTC)
I haven't gotten to the publisher merge yet. I was getting simple FRs out of the way tonight (2 more down) because I didn't want to mess things up while in sleep-deprived mode. I will attack it tomorrow morning... Ahasuerus 05:01, 15 May 2011 (UTC)
Title merge now completed. While working on it, I discovered Wikipedia entries weren't shown, so I fixed that as well as adding webpage merge. BLongley 12:48, 16 May 2011 (UTC)
Thanks! I'll put them through the wringer later today. Ahasuerus 13:33, 16 May 2011 (UTC)
I am still testing them, but they look good so far. For now, I installed a few "low hanging fruit" changes. Ahasuerus 05:47, 17 May 2011 (UTC)
Award Editing
I had a play with this earlier and have some thoughts I'd like to bounce around with you before proceeding much further.
We can fairly easily improve "Recent Integrations" to show when an award has been added to a title.
Sure, if it's easy to do, go for it!

We might want to leave some such edits in the queue for other moderators to practice moderating such.

We might want to reenable award editing for some more people willing to test in anger - it's not until you've done a LOT of edits that you realise how awkward the interface is, and it seems a waste to make people do that on a local test instance first. (Mike Hutchins has volunteered now, and Darrah Chavey wants to fix some Tiptree Awards.)
Since you have found that there are no unspeakable multiverse-destroying bugs in the Award code -- and Al said the same thing when I asked him a few months ago -- I will be enabling it for moderators and Darah shortly. I just want to do a bit more testing locally before I open the floodgates.

Moderating such shouldn't require moderators to know about short codes for awards such as "Award Type: Bs" for "British Science Fiction Award". A DB table of the existing awards should allow us to display a more meaningful name - longer term, New Award Names should be enterable.
Absolutely. I discussed it with Al at one point and we agreed that it's highly desirable.

"Award Category" could be a DB-Table-based drop-down list on "New Award" entry without too much difficulty, although again we might want to allow for new categories.
Yes, everything should be table driven as much as possible. The only question is how much time it will take to implement. We will also want to discuss the new tables before we start since those things can be hard to change once they have been implemented.

"Award Level: 9" is not intuitive and again could be improved.
Agreed. Perhaps change "Level" to "Ranking"? Ask on the Community Portal?

In hindsight, I'd much rather have a bot add these and just let Humans Moderate them.
Specific to the two categories I did today:
Al seems to have skimped on the BSFA awards and only listed the Shortlist for most years, rather than the LongList - I'll have to look into how "Nominated", "Short-Listed" and "Awarded" could be improved, before I tackle the omissions.

We need to confirm how Award years are determined - "year award was presented" or "Award was presented for year XXXX".
This is a known issue which Al mentioned in Help:Screen:AddAward when he created the page. We may want to check how most of our our awards are entered, especially Hugos and Nebulas.

Over a dozen Short Fiction nominations are for things we haven't recorded yet. I'm going to look into those later and see why - leave that with me. We may need a standards discussion about how low we go in nominations and webzine entry.
I'm sure there'll be a lot more thoughts as I play more, and sorry to dump these all on you - but you're the only other person I know is occasionally looking at such. Feel free to spread it out to other volunteers. On the good side - I haven't added anything to "Outstanding changes" for hours now, so you should have a good chance of catching up! ;-) BLongley 19:47, 16 May 2011 (UTC)
Lots of good points, I will have to think about them, but my first thought is that we need to make sure that there are no serious database-corrupting bugs in the Award editing logic. As long as that is the case, we can let moderators access these options and see what kind of feedback they provide. Ahasuerus 03:44, 17 May 2011 (UTC)
I haven't found anything DB-corrupting yet, unless you are concerned about people entering frivolous categories. (Fairly easily fixable, see point 5.) But I've only played with "Add Award to Title" in earnest, and so far only for Novel and Short Fiction for BSFA awards. I was heartened to find that we had all the Novel Nominees already, not so happy with Short Fiction as we missed over a dozen of those and some of them are NOT webzines but real print magazines. But I can call for volunteers to fill in some missing issues or add a few Nongenre Magazine contents myself - make a project small enough and with a definite end in sight, and people do work on it, unlike the never-ending Fixer submissions which seem to have disheartened most Mods. Speaking of Mods, if we do find it safe to open up Award-Editing either we need to let Chavey in as a non-Mod with an interest, or make him a moderator. Or use it as a final test of suitability, and get some feedback on how bad it is for non-self-approvers. BLongley 04:28, 17 May 2011 (UTC)
Darrah Chavey has made some very good comments and led me to try some award deletes and edits. On the downside, I've found we are sadly behind on Awards recently. On the upside - we're finding volunteers to work on them! BLongley 14:53, 17 May 2011 (UTC)
How's the thinking going? I'm looking into some improvements in this area, but it probably needs breaking down into much smaller chunks and of different priorities. For instance, a new DB table of short-codes for awards giving the full name is simple but probably only benefits those of us querying the DB offline. But that would be a good step towards improving the moderator experience, and making the editor interface more flexible, but we'll still want a way to add/edit Award types. Give me a few thoughts and I can get started. BLongley 18:49, 19 May 2011 (UTC)
See my comments above. If we want to start small (which is probably prudent), we may want to improve the moderator review screen to show the full award name rather than the abbreviation. The next step could be to create a table of categories and populate it with what we already have in the database. Ahasuerus 02:11, 20 May 2011 (UTC)
Found a bug in the post-approval screen, fixing it now. Ahasuerus 02:24, 20 May 2011 (UTC)
Big fixed. It also occurs to me that we may want to add an "Award search" to the main search box. At the moment, it's not easy to find all Hugo award lists and it will get harder as we add more obscure awards. Ahasuerus 02:45, 20 May 2011 (UTC)
... and add links to "previous year" and "next year" to award pages. At the moment, there doesn't appear to exist a way to get from 1972 Hugo Award to 1973 Hugo Award. Ahasuerus 02:53, 20 May 2011 (UTC)
It's already there. Look in the menu on the left and there's a section "More Awards" with "Previous Year" and "Next Year" links. BLongley 13:28, 30 May 2011 (UTC)
OK, I added scripts/create_award_types.sql as a small first step. As mentioned above, it's probably not going to help a lot of people on its own, but it's a start toward improved moderator screens and making things more table-driven. And even though it's small, it probably raises a few questions - are the column names suitable? Do we need a Primary Key Index for such a small table? Should we include a few more entries for known desired new Award types such as the Carl Brandon Society awards that Darrah Chavey should be an expert on? Do we want to create lots of small Sourceforge Feature Requests for all the above thoughts, or one or two big ones? (We're not particularly clear on how we deal with partial fixes or incremental improvements.) BLongley 10:31, 20 May 2011 (UTC)
I think the first thing that we will need to do is come up with an outline of the "final state" of the award subsystem. For example, is it realistic to have 700 awards as a single drop-down list? If not, how will we want the user interface organized? While we are thinking about it, we may want to go after the "low hanging fruit" stuff like the Award Directory that is currently being discussed on the Community Portal. Ahasuerus 06:06, 26 May 2011 (UTC)
Final state? Be able to record any sort of SF award, with links to any type of entity we recognise. Author, Title, Magazine, Artist, Publisher, etc.

Low hanging fruit: FR 3305564 "Link to new or edited awards in Recent Integrations" and FR 3308045 "Link to Awards page".

Next steps: make the award type table-driven, and editable - by mods alone, at first. Like and - although we probably want to protect existing award codes.

"Award categories" next - we shouldn't have 700, there's not more than 20 categories for any given award type I think. And some of those are inconsistent over years - you just don't notice that when you can only see them one year at a time. I think I can do a small clean-up project for that though, let's see how many we end up with.

"Award levels" after that - there's special coding for Locus Awards where the Poll position is preserved, other awards don't get that, and some special levels are no longer editable. I can't think of any new levels we need, but if we can cut them down to ones relevant to the award category within the award type it makes things quicker. BLongley 23:28, 26 May 2011 (UTC)
Oh, and if you can review the "scripts/create_award_types.sql" addition, then I have things like ready to go. But you did "want to discuss the new tables before we start". BLongley 23:47, 26 May 2011 (UTC)
Already discovered the first problem - it should be "MyISAM" rather than "InnoDB". I've just wasted 3 hours wondering why updates don't "take" - apparently I don't get an auto-commit if the Engine is wrong. :-( BLongley 18:19, 27 May 2011 (UTC)
((Unindent)) Sorry, me again! I've finished the Stoker Awards as far as I care to take them at the moment, and have some more thoughts. I've left some at Talk:Awards‎ but on the development front I'm already looking at Award Categories within Award Type - if those names are agreeable. When looking at things like this it would be possible to not only align certain categories to certain awards, but check whether such a category is permissible for any given year. If that's too prescriptive, we could at least provide a Mod Warning. BLongley 17:48, 28 May 2011 (UTC)
Ouch, I seem to be finding all sorts of worms. I think we need to provide some sort of sequencing number on Award Categories within Award Type - compare 2003 and 2004 Sturgeon awards. BLongley 18:50, 28 May 2011 (UTC)
This is certainly "incremental progress", but I am thinking about the underlying structure. If we are going to support an arbitrary number of awards, then two letter abbreviations won't work once go beyond 26 times 26. In addition, the current drop-down list which lists all supported awards will become unwieldy at some point. We really need to create an Award class and make awards behave the way Titles, Pubs, Authors, Series, etc behave. I am somewhat concerned that the more we beef up the current system, the harder it will be to convert it to the new system. Ahasuerus 20:37, 28 May 2011 (UTC)
I haven't tried to restrict it to two-letter-codes, just tried to preserve what Al started with. The set-up I'm playing with has 26^8 possibilities, which is probably more categories than we will ever have even publications. (Actually, it's got even more possibilities than that as I haven't actually restricted it to letters.) I'm thinking of just one more award at the moment, ("Carl Brandon Society") to keep Darrah busy a bit longer. It shouldn't be too difficult to put a "proper" primary key on all award types if you're happy to run a mass-update script at some point, but I don't foresee a mass-proliferation of such appearing any time soon. (And if the first set of new award types are the ones "Paranormal Romance" titles get, I may quit in disgust anyway!) BLongley 01:46, 29 May 2011 (UTC)
P.S. It turned out at the last moment that I would have to work this weekend. I am also not feeling very well, so I won't be able to spend as much time on testing as I'd like to. The best made plans of rodents and humans... Ahasuerus 20:37, 28 May 2011 (UTC)
Look after yourself and don't worry about us. We muddle by without you, and the worst that can happen is that you come back to lots of questions. I'll have a bit more of a play with Awards and hopefully you'll come back to some more solutions. I already think I've figured out where Al went wrong with References and Verifications, so I might have a fix for that as well as more ideas for Awards. BLongley 01:46, 29 May 2011 (UTC)
(unindent) I am finally back to normal (for some values for "normal") and working on moving the development/test environment to a new server. Lots of backups to do, multiple hard drives to migrate, OS compatibility issues, etc. With luck, I will also get a couple of changes tested and installed this weekend. Ahasuerus 18:21, 4 June 2011 (UTC)
I'm feeling a bit more "normal" now myself (whatever that is), so would like to get back to coding if you're up for testing - and Albinoflea now seems to be on board for that too. Do you have a clearer idea of what the "underlying structure" should end up as yet? I'm still a fan of "incremental progress" as it means I can do small stuff to help, but obviously don't want to develop stuff that will be rejected. We're really bad at discussing major design changes aren't we? :-/ BLongley 23:14, 12 June 2011 (UTC)
Welcome back to the wonderful world of normal! Pay no attention to the giraffe burning in the corner, it's for the snarks.
I am thinking that we want to fix all known Awards-related bugs first and then make Award editing available to all editors. That should keep you busy for a bit while I am processing the backlog. Now that spammers have been defeated (fingers crossed) and the new development server is mostly operational, I am working on foreign language support, specifically on Tiel display and making Merge Titles compatible with the new data element. Ahasuerus 23:45, 12 June 2011 (UTC)
As Darrah has encountered Bug 3311477 recently, I've submitted a quick fix for that. Quick as in "I've only tested it for the specific problems he and I have found so far" - there may be other issues still. But it's quite a small change overall and seems to be quite urgently required now. BLongley 15:55, 13 June 2011 (UTC)
Good idea! I still have to change Unmerge before the 2nd phase of the language support project can be installed, but I can set that aside and test 3311477 tomorrow. Ahasuerus 05:33, 14 June 2011 (UTC)
Bluesman seems to be frustrated with Chavey's unapprovable submissions too, so I guess that's still high priority. I've done a lot of work on allowing Mike Hutchins to correct award names ("no such thing as 'Locus Poll Award'") and allowing new awards to be created (Darrah still wants "Carl Brandon" next, I think) but I'm already up to 10 scripts that will need changing, so it's not trivial and I haven't even looked at Categories and Levels yet! (Apart from the immediate fix mentioned above.) If you have a clear idea on what other Award Bugs need fixing before it can open up to other editors, record them and assign them to me, I'll have a go. I suspect you'd rather have some SMALL changes to look at rather than a major redesign. ;-) BLongley 01:40, 15 June 2011 (UTC)
Chapterbook series
Would you please assign Bug 3291479 to me? Thanks. --MartyD 00:27, 21 May 2011 (UTC)
Done, thanks! Ahasuerus
Blackjack time again?
I suspect we need to look at Darrah Chavey soonish, while he still has the enthusiasm and before I run out of energy moderating his Award edits. BLongley 01:36, 22 May 2011 (UTC)
Based on the Talk page, it looks like there are still some areas for improvement, but it's getting close. Ahasuerus 03:23, 22 May 2011 (UTC)
The Stolen Sun and Tramontane
I added Emil Petaja's introduction of "Tramontane" and a note about this essay to this verified pub. Thanks, --Willem H. 20:50, 23 May 2011 (UTC)
Confirmed, thanks! Ahasuerus 19:43, 28 May 2011 (UTC)
Four-Day Planet / Lone Star Planet
According to the notes in this second printing of the Ace edition, John J. McGuire is not credited. But this OTHER second printing credits him in the header of Lone Star Planet. How is he credited in your first printing? I'm trying to reconcile the three printings, because the Piper only printing came up on Bill Longley's Extra Authors listing. Thanks for checking. Mhhutchins 15:47, 1 June 2011 (UTC)
The title pages for Four-Day Planet (p. -1) and Lone Star Planet (p. 217) do not mention the author(s). However, every even-numbered page of Four-Day Planet says "H. Beam Piper" at the top of the page and every even-numbered page of Lone Star Planet says "H. Beam Piper and John J. McGuire". I have updated notes and suspect that the other two printings are done the same way. Too bad their verifiers are currently inactive... Ahasuerus 23:39, 1 June 2011 (UTC)
So should I take it upon myself to update DES's verified pub by adding McGuire to the author field and the notes you added to your record? That would solve the conflict, and the notes should suffice to explain the credit. Mhhutchins 00:19, 2 June 2011 (UTC)
Sounds like a plan! Thanks! Ahasuerus 02:43, 2 June 2011 (UTC)
Russian Science Fiction 1969
Hi! Could you please check the order of the authors for this ss in this verified pub. They usually appear other way round. Cheers! P-Brane 06:52, 7 June 2011 (UTC).
I am afraid the software doesn't guarantee that it will respect co-author ordering. At one point this feature was requested, but there are no current plans to implement it -- it's rather labor-intensive and the bang for the buck is low. Sorry! Ahasuerus 20:23, 7 June 2011 (UTC)
Arcturus Landing
New scan, found the artist's signature on the cover so changed the existing notes [very tiny, about 1/2"up and left of "G" in Gordon] for [this] --~ Bill, Bluesman 20:46, 7 June 2011 (UTC)
Let me check... Ahasuerus 16:59, 10 June 2011 (UTC)
Confirmed. And yes, I almost needed a better magnifying glass to read the name. Ahasuerus 04:13, 17 June 2011 (UTC)
Awards Coding
I know, I submitted a LOT of software changes today. And although they're things that I've played with for a week now, I wasn't confident enough to remove ALL references to Award "maps" and "dictionaries" and "reverse dictionaries". It may be time for you to find people that can officially test stuff - I'm at the edge of my comfort zone and want fresh eyes checking what I do. And you shouldn't have to do all the testing yourself. BLongley 00:25, 17 June 2011 (UTC)
When I checked my e-mail, my first thought was that I was under a spam attack :-) No worries, I understand. I just need to get the foreign language changes out of the way first and then we can attack the next round of changes. When we add a new data element like title_language, we have to make sure that it is supported by all other editing tools like Merge and Unmerge, so I had to make a number of changes there. Getting close, though... Ahasuerus 00:51, 17 June 2011 (UTC)
It's amazing how one tiny change spreads, isn't it? I thought I could make one small incremental improvement to Language support and see that that's now grown to thirteen changes. BLongley 06:44, 17 June 2011 (UTC)
We are up to 21 now :-) I think I finally got them all, though. I just need to test the results to destruction before I deploy them. We also need to move the ad hoc "SQL" functions that I created to SQLparsing, but that can wait until after I install all subsequent versions which are currently sitting in the queue. Ahasuerus 03:36, 20 June 2011 (UTC)
And then there were 25... I honestly think this is it, though. Preliminary testing looks good, but I'd like to do another pass, then we can install and watch the fireworks. Ahasuerus 06:41, 21 June 2011 (UTC)
I'm getting worried. I've looked at the submission queue tonight and seen a long list of titles I can't understand without a bit of a Google, and worse, titles that don't even display in my browser except in little squares of Hexadecimal codes. Can I suggest you request more testers before you put this live? As I think this might be the biggest firework ever lit here.... BLongley 23:56, 21 June 2011 (UTC)
It's not as bad as it may appear. There are some display problems when editing non-Latin-1 titles, but that's a separate can of worms. I do have a solution for the display problem (although not a very elegant one because we are stuck with Latin-1 rather than UTF-8 for now), but it won't be implemented in the next patch. There are also policy issue to be resolved, namely whether we need separate fields for transliterated titles and for "literal translations", but that's still in the future. All I am doing at the moment is adding the "Language" field to every screen that adds or edits Titles. The added code is pretty simple and mostly repetitive, so the risk is manageable. (Famous last words, I know...) Ahasuerus 03:13, 22 June 2011 (UTC)
Well, it doesn't seem to have blown up too badly so far (crosses fingers). I don't think we can rest easy till the non-English specialists really use it in earnest, but my limited "speaks one language, can recognise a few more" skill hasn't found anything earth-shattering yet. It has reinforced my opinion that I'm really not fit to enter many non-English pubs - I can't cope with the weird accents - but if we get more experts in such I'm happy to make way for more skilled moderators. We may finally deserve the "The Internet Speculative Fiction Database" title, rather than just being the best English-language Internet Speculative Fiction Database. Thanks for all the effort! BLongley 23:09, 22 June 2011 (UTC)
And when I started on "Award Type" I only expected half-a-dozen or so, not seventeen. I think I might have to step back a little and do some documentation as to which modules need looking at when changing one - I don't think we've got anything that says "this edit script leads to this submission script which goes to this mod script which goes to this approval script" - and there's really obscure stuff like biblio\stats.py that I can't even recall how to get to on the live system! BLongley 06:44, 17 June 2011 (UTC)
That's very true, our documentation "sucks", to use a technical term. We also need to explain where classes are used, where SQLparsing is used and where raw SQL commands are used -- and how we plan to migrate this menagerie to classes. Ahasuerus 03:37, 20 June 2011 (UTC)
I made a start and updated all the Database_Schema pages for recent changes I could remember. I've probably missed a few, but I've added your new tables and my proposed award_types table. I'll have another pass through and see if I can improve on the original notes: but things like "title_ctl - This column contains exception bits which can be used to control the title display under distinct unique conditions" still means nothing much to me. Shame really, as that's exactly what I need to understand to fix the "[also as by]" bug. BLongley 19:20, 1 July 2011 (UTC)
Looks very nice, thanks! I am buried at the moment, but should come up for air by Monday-Tuesday (to mix metaphors in a vaguely SF-ish way :-) Ahasuerus 19:32, 1 July 2011 (UTC)
Development page
I noticed you "Alphabetized" the "Outstanding changes" changes section - should this be documented as desirable? I've normally tried to list them (mostly) in necessary implementation order, so the DDL changes go first. But you're our only tester so I guess what you want is what we should do. BLongley 21:07, 25 June 2011 (UTC)
When a change involves more than a few scripts and (some of) the scripts have multiple outstanding versions committed, it becomes difficult to keep track of what's where. It's not exactly essential, but alphabetizing scripts makes it a little easier to ensure that the right version is committed. Ahasuerus 04:21, 26 June 2011 (UTC)
And "subsumed" notes didn't seem necessary to me, but again, if you want such let us know when we should do it. BLongley 21:07, 25 June 2011 (UTC)
It's the same issue -- making sure that I understand what version does what, which versions can be skipped, etc. Ahasuerus 04:21, 26 June 2011 (UTC)
I'd have cancelled the first submission but it had been there a long time and I don't know how many people have the "bad" version. As a Sourceforge Administrator, can you see such? I can't. BLongley 21:07, 25 June 2011 (UTC)
Hm, no, I don't think so or at least I haven't stumbled on it yet. There is so much going on that I tend to do just the bare minimum on the Sourceforge side, although I really need to learn more about branching. Ahasuerus 04:21, 26 June 2011 (UTC)
I see Albinoflea got tripped up with recent incompatible changes, which is probably why other developers aren't doing too much at present. BLongley 21:07, 25 June 2011 (UTC)
Yes, I saw that and I think that branching may be the answer. Or at least I hope so! Ahasuerus 04:21, 26 June 2011 (UTC)
Russian language issue
Can you join in this discussion when you get a chance? Thanks in advance. Mhhutchins 02:45, 28 June 2011 (UTC)
Hi! here is the question in somewhat more general terms. Imagine an author (for example this chap) whose name and works are in the language that uses the alphabet sufficiently different from Latin. Moreover the English translations of his works use different transliterations of his name. What transliteration of the author's name should be used to record the original (albeit also transliterated) title into the db? My proposal is to always use CANONICAL NAME in such cases. Cheers, P-Brane 12:32, 28 June 2011 (UTC).
I agree that it that would make sense. However, I expect that we will end up with at least some counter-intuitive results and various exceptions due to the number of different transliteration systems for different languages. For example, see this chronology of Cyrillic transliterations. It's a royal pain... Ahasuerus 06:44, 29 June 2011 (UTC)
A step too far?
I played about with Albinoflea's latest feature request: see here but I think it might be a bandwidth-theft too far. :-( I suspect we should drop the links to other sites' images and limit the drain on our own resources. What do you think? (It's a shame, I think such looks quite pretty!) BLongley 20:47, 28 June 2011 (UTC)
I've checked my bandwidth usage for yesterday and it was double my average. :-( So I guess I've satisfied myself that it's too heavy to do as currently designed. Do you think we could cope with showing a year's worth at a time? It won't help if somebody works through all years in one session, and if somebody starts uploading covers for a Weekly magazine we may hit problems, but it might be a reasonable compromise. Alternatively, I can take requests for "Missing Magazine Covers" and "Non-ISFDB Magazine Covers" project Wiki pages. BLongley 16:32, 29 June 2011 (UTC)
The Birthgrave (Tanith Lee)
In your verified copy of this book, George Barr is listed for "Interior Art". Since the only piece of interior art is the frontispiece, I've added a page number to that content item, and submitted a title change from "The Birthgrave" to "The Birthgrave (frontispiece)". Chavey 21:45, 28 June 2011 (UTC)
Confirmed, thanks! Ahasuerus 07:21, 29 June 2011 (UTC)
getpub.cgi
md5i has requested FR 3314867 "getpub.cgi script could include some title information". Is it safe to add such to this script or would it be safer to create a new "gettitle.cgi" or something like that? BLongley 16:14, 30 June 2011 (UTC)
To quote Jack Benny, "I am thinking, I am thinking" :-) Ahasuerus 18:02, 4 July 2011 (UTC)
No hurry. I presume Fixer uses the existing interface, so that would be the major worry if I broke it - we all love Fixer don't we? ;-) I haven't fired up "DataThief" in ages so he'd be OK. And Dissembler seems quiet. But we're getting more and more tech-savvy Editors that might want to use the API - Rick Boatright comes to mind, "Head Geek" for Baen. BLongley 01:53, 8 July 2011 (UTC)
Roesnner or Roessner in The Sword of Winter"?
Would you check the spelling of "M. M. Roesnner-Herman" in your verified The Sword of Winter? A submission would make that a pseudonym of "Michaela Roessner". We already have the pseudonym for the alternate spelling of two Ses and one N. Thanks. --MartyD 10:53, 4 July 2011 (UTC)
Good catch -- it was a typo, now corrected. Thanks! Ahasuerus 18:31, 4 July 2011 (UTC)
broken submission
Check out this submission. You can reject it (or let me) when you're done with it -- I just wanted to leave it in the database for you. --MartyD 00:30, 8 July 2011 (UTC)
Duh, never mind. I thought I was reviewing a merge.... —The preceding unsigned comment was added by MartyD (talk • contribs) .
I know the feeling. After a while, their true nature is revealed and they all look like 0110010111010001010... Ahasuerus 03:31, 8 July 2011 (UTC)
Thornton W. Burgess
I'd reject all of them, except that you let Fixer pass so many through that I presume you had some reason to think some may be relevant? BLongley 23:12, 13 July 2011 (UTC)
Well, they are some of America's "most beloved children's stories", to use a well-worn cliche. However, in retrospect they are also the kind of "[a]nimal books for very young children, i.e. books for preschoolers which depict simple scenes from animal life featuring anthropomorphized animals" that ISFDB:Policy specifically excludes. Reject away! Ahasuerus 23:49, 13 July 2011 (UTC)
Thanks! I was worried that we'd opened the floodgates for Beatrix Potter too... and I'm already upset with all the "X, the Y Fairy" books. Can we find a new moderator willing to be harsh on this sort of stuff? BLongley 00:08, 14 July 2011 (UTC)
I would say that the difference is that the Daisy Meadows books are about fairies who, you have to admit, are more speculative than rabbits :-) Ahasuerus 00:23, 14 July 2011 (UTC)
I suppose so. We let Watership Down in though. I'm pretty sure I prefer Rabbit Stew over Fairy Cakes, but I'm not sure the cakes come with real Fairies in them. BLongley 01:05, 14 July 2011 (UTC)
Latest coding
I took your comments on Bug 3161128 and reworked the code - it does make it dependent on FR 3317609 though. If the bug is more important than the feature then we really need to learn how to deal with CVS branching. :-/ BLongley 01:18, 17 July 2011 (UTC)
I suspect there's a bigger benefit (for some editors anyway) in FR 2823394, Part 3 - set and use a Default Language. I'm already becoming aware of some oddities in our Language Support improvements - e.g. "why do we have a language for COVERART?" (INTERIORART may be a different matter, in the case of Cartoons with captions) - but no real showstoppers. If you still have concerns about changing the rules on Foreign Variants without a bit more coding, let me know what they are. (Or if it's the "My Languages" preferences not being active in various Displays, let Marty know - I'm not up-to-speed in that area.) BLongley 01:18, 17 July 2011 (UTC)
I've also proposed ISFDB:Proposed_Design_Changes#Unmerge_Foreign_Title.28s.29 Part 4. The coding was suspiciously easy, but as you can see, it may be built on unsound foundations, which is why I've not committed anything yet. Again, comments welcome. BLongley 01:18, 17 July 2011 (UTC)
I've spotted another wobbly foundation - "merge titles" doesn't spot different languages (yet). So there's a risk of some editors remerging things without warning. Not that many editors seem aware that "Check for Duplicate Titles" doesn't cope with variants, which is yet another underlying problem.... :-/ BLongley 02:50, 17 July 2011 (UTC)
Ah, I see you fixed that already. But "merge titles" still doesn't cope with Wikipedia link differences. I'll see if I can fix that. BLongley 16:32, 17 July 2011 (UTC)
Merges May Cause Self-Referring Titles
I think Bug 1743275 needs reopening - you can keep the "Parent" value that matches the "KeepId". BLongley 17:29, 17 July 2011 (UTC)
Good point - we may need to add a new check to ts_merge.py. Ahasuerus 19:28, 17 July 2011 (UTC)
Working Languages
OK, let's back-track. This:
might be a step 2a or 3a - if either this, or "Default Language", or both, goes in then people can start capturing the data. Is a default of "English" for now OK? I could do a mass-update script to set that for everyone, or just let people add it by default when working on the subject (like we're effectively doing with title language.) BLongley 20:29, 20 July 2011 (UTC)
As I mentioned before, I suspect the Summary Display changes are currently beyond my abilities. (I'm not a UI person, I'm a Data person really.) Please let me know what you think before I rush up another dead end. BLongley 20:29, 20 July 2011 (UTC)
Sorry about the delays, my availability has been spotty lately. When I recently wrote that I often work 80 hour weeks, I wasn't exaggerating, it can get rather hairy. My personal record is over 100 hours a week -- not that I recommend it as a healthy lifestyle. Ahasuerus 03:50, 21 July 2011 (UTC)
Understood. There's something wrong with this planet, that only gives us 168 hours a week. ;-) BLongley
I am acutely aware that my sporadic availability has become a bottleneck on the development side and I am looking for a solution, but nothing has come up as of yet. At one point I thought that I might be retiring in early 2011, which would have given me much more time to work on ISFDB, but it didn't pan out. I currently expect that I will be working for at least another year, possibly 18 months. I'd love to find a solution to this conundrum, but no such luck so far. I don't want to push too hard either because it would have to be done at the expense of what little sleep I do get and that way lie missed bugs and eventually burnout a la Al. Ahasuerus 03:50, 21 July 2011 (UTC)
Well, if cloning you isn't an option then either we wait for you or you find someone trusted enough to take over some of the work. I've made enough mistakes or assumptions that I suspect it won't be me, but my activity seems to be scaring off some of the more competent developers. :-( I suspect Marty understands more of the code than I do, for instance, and Uzume knows far more about our setup than I understand. BLongley 18:51, 21 July 2011 (UTC)
As far as adding "working languages" goes, I have been thinking about it for some time. In most authors' cases this field will never be used: we will always be displaying canonical titles and suppressing any "foreign language" VT for languages that the user didn't select in the Language Preferences screen.
"Working language(s)" only comes into play when you have a translated title with no analog in the original language, e.g. suppose a French editor uses his own preferences to put together a collection of Philip K. Dick stories. We will have to enter the resulting collection as a canonical title since there will be no English language original. When that happens, we will (eventually) want to suppress this Title record on the Summary page unless the user lists French as a "preferred language". Now, this may seem like an obscure and uncommon case that is not worth spending much time on, but there are countries where publishers churn out dozens and dozens of volumes of "Collected works of Famous Foreign Writer XYZ", so over time it will become a problem for popular authors.
To make things even more interesting, there are writers who use multiple languages, e.g. Sam Lundwall has written books in English and in Swedish, Vladimir Nabokov wrote in Russian and English, etc. At first I thought that adding a second "Working Language" field would help address the issue, but it would also complicate the design significantly. I am now thinking that a combination of a single "Working Language" field at the Author level and a Title level override field should work better.
For now, we should be able to get away with one "Working Language" field in the Author table: we will simply display the language of the canonical title as "[language]" next to any canonical titles whose language doesn't match the author's working language. Ahasuerus 03:50, 21 July 2011 (UTC)
OK, I've reduced it to one working language (I've left Working Language 2 commented out rather than removed totally, in case it does turn out desirable.) As it stands, the changes should mean that Author Editors provide the data while doing other stuff - although I could try and default the field with a mass-update script or several. (I know, you're wary of those, so am I when it comes to Languages!) BLongley 18:51, 21 July 2011 (UTC)
Also, we won't have to modify the Summary page aside from adding support for "[language]" whenever if doesn't match the author's Working Language. Everything else can be done by tweaking the SQL query that retrieves titles from the database. Or so I hope :) Ahasuerus 03:50, 21 July 2011 (UTC)
I've deliberately kept this to a "capture the data" change, and not tried to use it elsewhere. Yet. It would be nice to get this in and let another developer with better language knowledge use the data, but only you can prioritise changes. Stonecreek and Willem do seem to like what we're doing though. BLongley 18:51, 21 July 2011 (UTC)
Unmerge titles
I think we messed up with FR 3294454. :-/ When we fixed unmerging of Container Title Authors we broke Content Title Authors. Unmerging of Interviews and Reviews seems to have always been broken. As Deagol and Willem seem to be looking at the language improvements and using the unmerge functionality for foreign language titles (I know, we haven't told them to do so yet!), this may be high priority. BLongley 16:37, 23 July 2011 (UTC)
Please give it a look - it's only one module changed, although I appreciate that there's a lot of use-cases to check. And like you did with my last submission, feel free to ask for further fixes/improvements from me rather than try to do them yourself. (It won't be tomorrow though, I'm off to see my sister that I haven't met for four years now.) BLongley 16:37, 23 July 2011 (UTC)
Will do! Ahasuerus 20:37, 23 July 2011 (UTC)
The Remarkable Exploits of Lancelot Biggs: Spaceman
Please take a look at this discussion. You have a copy of Bond's The Remarkable Exploits of Lancelot Biggs: Spaceman with the publisher listed as "Young Moderns Science Fiction, Doubleday & Company, INc.". I've set up a publication series, Young Moderns, for the two other books we've found in the series. Do you agree that a series is preferable way to handle this rather than an imprint? Thanks. --Ron ~ Rtrace^Talk 14:03, 24 July 2011 (UTC)
Answered on the Verification Requests page. Ahasuerus 20:33, 24 July 2011 (UTC)
Pill spammer?
If you have some time, can you look at the history of this page? It looks like we have a spammer (I think blue pills) with a definite preference for this page. Is there a way to block this one for a longer time? Thanks, --Willem H. 12:08, 25 July 2011 (UTC)
I've protected the page so only Mods can edit it now. BLongley 13:30, 25 July 2011 (UTC)
Thanks! It would appear that Archive pages attract link spammers because the assumption is that they may not be as diligently monitored as regular pages... Ahasuerus 14:46, 25 July 2011 (UTC)
edit/editpublisher.py
This seems to have broken between v1.2 and v1.3? When there's an existing webpage, editing the record now displays it with extra characters around it - e.g. "(’http://www.pspublishing.co.uk/’,)" rather than "http://www.pspublishing.co.uk/". BLongley 17:07, 31 July 2011 (UTC)
It looks like it was broken a while back. I have fixed it, but it's probably the limit of what I can do until I get better... Ahasuerus 01:47, 1 August 2011 (UTC)
Russian publisher
Created a [record] for the Russian first edition of Horsemen from Nowhere, using OCLC's record as the source. Copied the publisher directly from there but it just doesn't look right [quotation marks in a publisher's name??¿¿??]. Could you have a look? Thanks! --~ Bill, Bluesman 21:34, 12 August 2011 (UTC)
There are two types of problems here. First, abbreviations. "Izd-vo" stands for "izdatel'stvo", i.e. "publisher", and "lit-ra" stands for "literatura", which is self-explanatory. Second, "Detskai︠a︡" is a transliteration of "Детская", i.e. "children's", using the ALA-LC Romanization Tables. These tables still use diacritics, which are beloved by some folks at ALA and hated by everyone else.
I have cleaned this mess up a bit and hopefully the record looks better now. I also added these abbreviations to our list of ISFDB:Foreign Language Abbreviations. Thanks for catching it! Ahasuerus 22:02, 12 August 2011 (UTC)
Oops, I just realized that the OCLC record states that this pub is a collection. Here is what the "Contents" line says:
Vsadniki niotkuda; roman [novel]

Chetyre tsveta pamiati; rasskaz [story]

Moshkara; rasskaz [story]

Kheppi end; rasskaz [story]

Glaza veka; povest' [novella]
Unfortunately, it's not clear whether the other four pieces are SF. Let me poke around... Ahasuerus 22:07, 12 August 2011 (UTC)
According to Fantlab and OldSF, all of them are SF, so add away! :-) Ahasuerus 22:15, 12 August 2011 (UTC)
Will wait until Wilem gets back from holidays [next week] as if his verified translation is not a collection then some unmerging will be required. Thanks for looking and cleaning up the record! --~ Bill, Bluesman 20:19, 13 August 2011 (UTC)
My edition contains the novel and nothing more. I did find one little thing wrong however, Alexander is credited on the titlepage as Alexandr. I will add a variant/pseudonym for this. --Willem H. 18:33, 19 August 2011 (UTC)
Impact-20
Hello, if we go by title pages, shouldn't this pub be titled "Impact 20" ? Hauck 17:22, 24 August 2011 (UTC)
Sorry, I missed your note two weeks ago when I was traveling. You are absolutely right, it should be "Impact 20". Have you had a chance to check with the other primary verifiers to see if their copies match? Ahasuerus 22:24, 5 September 2011 (UTC)
Done. Hauck 05:29, 6 September 2011 (UTC)
Bibliography display oddity
Can you figure out why Summit Chase shows up as "[also as by ...]" in Warren B. Murphy or Richard Ben Sapir instead of as "[only as by ...]"? --MartyD 20:31, 5 September 2011 (UTC)
I saw your question... and I think it's due to Author order..... I think if you swap the Author order on the parent title (probably by deleting the first author.. then adding it back as a second edit) it should resolve.. all the other items in that author set have the variant author order matching the parent author order. (Just my two cents... returning to Lurk mode) Kevin 23:10, 5 September 2011 (UTC)
Oh cute. Easy enough to try that. Guess someone could go fix it while he's at it, too.... ;-) --MartyD 00:30, 6 September 2011 (UTC)
Go for it! Although if it stomps over other changes in the pipeline, be careful. BLongley 02:31, 6 September 2011 (UTC)
I feared Ahasuerus or yourself attempting to edit those records at the same time and didn't want to risk stepping (editorially) on anyone's toes. Kevin 00:45, 6 September 2011 (UTC)
This is the only title that I didn't break the variant relationship, because of this question. Once you've determined what causes the display anomaly, let me know and I'll remove the variant, or either of you can. Mhhutchins 00:59, 6 September 2011 (UTC)
We're not very good at letting experts get on with stuff when there's a big project, are we? Perhaps we should have a "Destroyer" project page where people can claim their interests in helping us out? BLongley 02:31, 6 September 2011 (UTC)
Oh, don't worry about that! I'm getting on just fine adding Destroyer pubs while y'all try to figure out why that title's messing up the rules :-) Chavey 02:55, 6 September 2011 (UTC)
Darrah, I've placed your submission on hold until we can figure out why the record is unusually displayed. Mhhutchins 03:13, 6 September 2011 (UTC)
I tested every different combination of credits there are, and it remained displayed as "[also by...]". So I went ahead and accepted Darrah's submission and removed the variant. Mhhutchins 13:55, 6 September 2011 (UTC)
I've found it again and found another reason for items to show up as 'Also as by'. Language. When one variant is of a different language, this shows up... and I bet it shows up when the variant has a language.. and the parent doesn't. See 'Wulfrik' by C. L. Werner. —The preceding unsigned comment was added by Kpulliam (talk • contribs) .
I think you're on the right lines, but suspect that when the English and Foreign titles are the same it gets worse. We've got a huge backlog of Language improvements at present, but less so on the Display front where we know we need more work - are you trying this on current Live software or on the Potential improved software? BLongley 00:10, 7 September 2011 (UTC)
(unindent) I was able to recreate it. In this case, it was the parent's having no language and the variant's having English. I think we should grandfather no language as English. I guess it's also worth looking into whether the "also as by" should display info about the languages.... --MartyD 01:01, 7 September 2011 (UTC)
When you say "grandfather no language as English" are you talking about changing the code to interpret NULL as "English" or updating all or most-of the data (where we could at least exclude the most prominent "foreign" authors, or even update the notables like "Jules Verne"). BLongley 01:29, 7 September 2011 (UTC)
I mean in certain situations treat NULL as equivalent to "English". In a way, that's what happens now when editing a title having no language. Even though the editor makes no choice, the title's language gets changed from NULL to English (the default in the form). That equivalence may not be appropriate everywhere. And we could further restrict it (i.e., NULL <-> English IFF the title text is identical). Just a thought. --MartyD 10:36, 7 September 2011 (UTC)
Data Corruption Issue
Can you look at ISFDB:Community_Portal#Another_Language_issue when you get a chance? I've gone through all the affected pubs I've found in my last loaded backup (which is quite out of date) and fixed those that can be fixed easily (Clone and Delete works on many). But I can't do that on many others as it would lose Verifications. And in some cases the Missing Titles do seem to be ones that we should have still, when comparing them to similar publications. BLongley 23:08, 7 September 2011 (UTC)
I haven't spotted a pattern in the problem - it may just be due to partially completed transactions during server blips - but it has left a few hundred pubs in an unstable state, and I'm a bit worried. BLongley 23:08, 7 September 2011 (UTC)
Understandable. I'll take a look, thanks! Ahasuerus 23:15, 7 September 2011 (UTC)
I think (thanks to some help from Hauck) I've found the problem, or at least one part of it. See User_talk:Hauck#Can_you_help.3F. The underlying problem is that a clone operation includes the content titles in the submission, but approving such doesn't check whether they still exist, so a merge operation in between may lead to a non-existent title being created in the clone. Does that seem right to you? I'm pretty sure I've recreated that problem. BLongley 17:59, 8 September 2011 (UTC)
I don't think we can just add a check to the cloning to make sure titles exist still, or we'd exclude titles that were intended to be included. We could rework cloning so that it works on the current contents at time of approval. But I'm beginning to think that Marc Kupper's one-time suggestion that we record the link between a deleted title and the one that it was merged with might be more useful. BLongley 17:59, 8 September 2011 (UTC)
Whatever the long term solution is, for now I think we should make Moderators aware of what to watch out for, and do a small project page so that people can check their corrupted pubs. That would be best if it was broken down by Verifier, which means only you can do it by Verifier name. BLongley 17:59, 8 September 2011 (UTC)
Have you had a chance to look at this yet? BLongley 17:16, 12 September 2011 (UTC)
Russian spelling: Help appreciated
Hello, Ahasuerus! I had some difficulty in transcribing a Kyrillian title and entered the translated one instead. See these commentaries. The original title is to be found here: it is in the first department of works, the third from the year 1972. I can decipher the first word, Experiment, but after that I am quite lost. Could you lend us a hand, or would it be better to wait for an original entering of the title?Stonecreek 14:49, 11 September 2011 (UTC)
Done! Ahasuerus 18:12, 11 September 2011 (UTC)
Canonical names and diacriticals
I have a submission from Dirk P Broer on hold that will change Stanislaw Lem's name by using a 'mark' through the 'l' in his first name. I can't find anything in the Help covering this, whether it's a no-no or is fine. I can't even make such a letter [that I can tell] and wonder if changing the name will affect anyone searching Lem without such a letter? I seem to remember some discussion[s] on a similar topic but couldn't find any in a quick pass through the various pages [Moderator/Community/Help]. Thanks! --~ Bill, Bluesman 14:29, 12 September 2011 (UTC)
Yes, this is a known issue. At one point we had two problems with the "ł" character. First, it broke author links. Second, it broke searches on, say, "Stanislaw Lem". The first problem was solved a while back when we started using author IDs rather than author names in links. The second problem, however, still exists, so if we change the name, users will have trouble finding Lem. I suggest we reject the change for now -- it will become more feasible once we improve our international support. Good catch! Ahasuerus 15:22, 12 September 2011 (UTC)
Thanks! I'll direct Dirk to this discussion. --~ Bill, Bluesman 15:30, 12 September 2011 (UTC)
My change was solely for the Legal name, NOT for the Canonical. --Dirk P Broer 15:52, 12 September 2011 (UTC)
My oops there. I saw the change, had one of those little LEDs that blink sporadically in the back of my mind go off and just went looking for something in the Help. I don't think changing the legal name will affect searches? Apologies. --~ Bill, Bluesman 16:12, 12 September 2011 (UTC)
There is a Legal Name search in Advanced Author Search, but I lack the skills to know if that's affected. BLongley 17:13, 12 September 2011 (UTC)
The advanced search cannot handle diacriticals, I searched for Martensson OR Mårtensson in Canonical name and it came up with nothing. --Dirk P Broer 18:57, 12 September 2011 (UTC)
Some authors already seem to have this problem with Canonical name. E.g. 19869 is not findable in a simple search for "Martensson", or in the Author Directory. BLongley 17:23, 12 September 2011 (UTC)
Why not make it Martensson as Canonical name, and Mårtensson (alt-134 for å) as Legal name? --Dirk P Broer 18:30, 12 September 2011 (UTC)
"Sourcing" changes
I'd appreciate your comments on this: ISFDB:Moderator_noticeboard#Noting_Sources_made_easy__Attempt_3_part_3 and/or the prior ISFDB:Community_Portal#Noting_Sources_made_easy_-_Attempt_2 or even ISFDB:Community_Portal#Noting_Sources_made_easy_-_Attempt_1. People seem to like the start, but of course only you can decide if it's good enough in the end. BLongley 00:28, 2 October 2011 (UTC)
I don't think I can take this further without clashing with other changes in the queue - Add Pub and Clone Pub seem OK, New Pub still has a prior change ahead for Language Support. I can submit one or the other or both if you'd rather test them yourself rather than look at my screenshots. If "New Pub" is most desirable I'll see if I can relearn CVS Branching. (That could keep me out of your hair for a while.) BLongley 20:07, 2 October 2011 (UTC)
I am thinking, I am thinking :-) Ahasuerus 00:05, 3 October 2011 (UTC)
No immediate hurry - I have to be a Pool demon tomorrow night and herd the players that our craptain can't control. I promise not to code anything new before Tuesday. :-) BLongley 00:41, 3 October 2011 (UTC)
Oh, experimental demonology! Sounds exciting, but please be careful!
Unfortunately, the new process used by Fixer requires significantly more manual work than the old one. I am trying to automate it so that it wouldn't get in the way of my testing, but it is still a work in progress. Hopefully I should have most of it coded by Tuesday-Wednesday... Ahasuerus 06:12, 3 October 2011 (UTC)
Gordon R. Dickson - Beginnings
In your verified pub Beginnings I corrected the title of the short story "Danger—Human!" to use an em-dash, instead of the space minus space that was there, to match my copy in hand. (I actually drop-added a new variant, but that's just mechanics). I imagine that most instances are actually em-dashes and this is a typo that has been propagating for many years. Thanks Kevin 16:18, 8 October 2011 (UTC)
Thanks for the heads up! Ahasuerus 20:00, 8 October 2011 (UTC)
Eddy C. Berlin
Hi, just checking Belgian authors: Could you please check whether it is indeed Eddy C. Berlin or Eddy C. Bertin that wrote the story A Taste of Rain and Darkness in Bizarre Fantasy Tales, Fall 1970 No. 1? —The preceding unsigned comment was added by Dirk P Broer (talk • contribs) .
Sure, let me see if I can find it... Ahasuerus 01:18, 26 October 2011 (UTC)
I believe it is the translation of the Dutch story "Uit een gordijn van regen en duisternis", collected in the 1984 Dutch publication "De Griezeligste Verhalen van Eddy Bertin" and earlier in "De Griezeligste Verhalen" (The Most Scary Stories). --Dirk P Broer 23:37, 26 October 2011 (UTC)
Well, there is bad news and then there is good news. The bad news is that I can't find this issue. The good news is that I have been able to find the next issue of Bizarre Fantasy Tales. It contains another Bertin story and the introduction mentions that the author was also responsible for "A Taste of Rain and Darkness" in the previous issue. So it's definitely the same person, but we can't be sure whether his name was misspelled in the magazine or whether I misspelled it when I was entering the data. However, poking around the internet I see this cover scan which shows "Bertin" on the cover, so it's most likely my error. I will merge the two authors and try to find the issue at a later point to double check. Thanks for catching the error! Ahasuerus 02:31, 27 October 2011 (UTC)
P.S. Also, there is something peculiar going on with Edith Brendall. She is entered as Bertin's pseudonym, but the only Title record is a collaboration between Bertin and Brendall. Ahasuerus 02:37, 27 October 2011 (UTC)
That's like John Wyndham and Lucas Parkes. Bertin collaborated with himself. --Willem H. 05:57, 27 October 2011 (UTC)
Ah, I see, thanks! Ahasuerus 06:13, 27 October 2011 (UTC)
Other other sites
I had a look at linking to 3 more: Google Books, Library Thing and Open Library. There's probably more of a political discussion than a technical one to be had for those, they don't depend on ISBN-13s, so shouldn't require the Smashwords changes to go in first. There's no referrer-id issues with them, and should be able to go in in any order. BLongley 19:02, 30 October 2011 (UTC)
Also, they're easily reversible - just delete the entry if people object. We might need to do such for stores going out of business anyway - I'm glad I never looked into Ottakar links, and Borders might have been a waste too. But these don't seem to be tricky and might be a "quick new feature" while you gird your loins for the big stuff. BLongley 23:32, 30 October 2011 (UTC)
Shelfari would be another easy win if we want it? Also Goodreads. BLongley 16:37, 31 October 2011 (UTC)
One thing to keep in mind when adding new "other sites" is that we also need to update the site-specific User Preferences page. Ahasuerus 05:15, 1 November 2011 (UTC)
Unless I'm missing something, that's covered - the new sites are as selectable in preferences as any others. Unless you want them to be OFF by default? I left them as ON by default so people would actually notice the change - it's heartbreaking when you make an improvement and nobody notices! BLongley 12:15, 1 November 2011 (UTC)
Other than that it shouldn't be a problem, although I thought the original request was to add "verifications" for these sites? Or I am confusing this FR with another request? Ahasuerus 05:15, 1 November 2011 (UTC)
There's some overlap with the "Noting Sources made easy" proposals where some verifications could be done automatically, but those discussions seem to have led to "only Primary and Transient automatic verifications are desired". This set comes more from your announcement of how many Kindle books we've got back-logged, and Kevin's agreement that Smashwords is a more useful bibliographic source than Amazon for many ebooks. Plus your "we're part of Web 3.0 already" comments made me think there'd be some support from you for outgoing links. BLongley 12:15, 1 November 2011 (UTC)
On a related note, LibraryThing offers three other useful API entries we might want to look at, but would we keep within the "one request per second" and "1,000 times/day" limits if we leave them as manual options? BLongley 19:02, 30 October 2011 (UTC)
Unfortunately, "one request per second" and "1,000 times/day" are hard to enforce within the confines of our paradigm :-( BTW, at started a little matrix which lists various online APIs, whether they let you search by genre, by ISBN, etc. It also lists each API's technical and administrative limitations. At the moment, Fixer uses the OCLC API which tells you what other ISBNs are related to a given ISBN, but it is not used when creating submissions. With the current Fixer rewrite -- which finally adds support for multiple sources of information rather than just the two Amazon stores -- it should be easier to leverage this data. Ahasuerus 05:15, 1 November 2011 (UTC)
I agree it's hard to enforce but I think we might naturally stay within those limits anyway. The "other ISBNs" is something I looked at before and I even submitted a bookmarklet for such - which, as it came from "Data Thief" seems to have been totally ignored - but will probably become very useful when we open up more to other languages. BLongley 12:15, 1 November 2011 (UTC)
Character sets
Hi. See User_talk:Dirk_P_Broer#Author_updates for background. It looks to me like part of the problem is that the Wiki is using the UTF-8 character set, while the ISFDB is using ISO-8859-1. So when the ISFDB's pages deal with Unicode 769, they end up using "%E9", while on the Wiki side it ends up using two bytes: "%C3%A9". Do you know off the top of your head whether we can change one side's encoding or the other's (not the database character set, the HTML character set)? I will try a little experiment, but I thought this might be something you've already tried in the past. Thanks. --MartyD 01:39, 2 November 2011 (UTC)
I think we can change the HTML character set used by Python fairly easily because it's centralized in one or two places. However, I'll have to test it to see what the impact on the rest of the application may be... Ahasuerus 05:20, 3 November 2011 (UTC)
Smashwords and other "Other Sites"
I think the consensus is that you can take the money from Smashwords links, if you or Al are happy to set up an affiliate ID yourself, or let me redirect mine to your PayPal account. I think the referral would even work without it, but I'm reluctant to throw away free money when it's so easily available. The other suggestions I've submitted (and the ones I've mentioned but not submitted) don't have that problem. BLongley 01:55, 3 November 2011 (UTC)
To be honest, I'd really like to hear what Al has to say about all this before we do anything. He has a much better appreciation of the issues involved. That said, I suspect that we are better off keeping money out of this altogether because introducing money to an all-volunteer operation can cause all kinds of problems. For now, I think I'll just implement a "naked" Smashwords link and we can easily add an ID later. Ahasuerus 05:46, 3 November 2011 (UTC)
OK. I've committed the Shelfari and GoodReads changes for consideration too. (I've kept them all separate in case you want to introduce them gradually.) BLongley 17:12, 3 November 2011 (UTC)
Having said that, there may still be some discussions to be had: you mentioned "third party publication identifiers like LCCNs, OCLC numbers, OpenLibrary IDs, GoodReads IDs, Google Books IDs, LibraryThing IDs, Amazon's ASINs, etc." These changes do address ISBN links that we didn't have before but I suspect the feeling is that Open, Free resources are good, commercial sites not so good unless we take the money (robbing those rich sites to pay for the poor). So LCCNs and OCLC numbers are probably in, ASIN probably out - I've already mentioned that as Kindle books aren't part of their affiliate program we should check those terms and conditions before opening the flood-gates. Maybe you should poll each one before implementing? Or review the complaints afterwards? BLongley 01:55, 3 November 2011 (UTC)
The way I see it, the key question is whether a new link adds value as far as the end user is concerned. For example, Goodreads is a money-making operation, but it also provides a ton of reviews and recommendation; many ISFDB users and bibliophiles in general are likely to have a Goodreads account, so it can be useful. Amazon also provides reviews and is the place where a lot of people go to buy books, so its links also help our users. Other sites may only be of interest to a limited number of users, e.g. most will not care about FishPond -- unless they live in Australia -- so perhaps we could display a FishPond link only if your IP address suggests that you are from Down Under (or perhaps if you explicitly choose to see FishPond in User Preferences.) And so on and so forth. Whether the company on the other end of the link ends up making money because of our referral is not something that I would be worried about as long as our users find the link useful. Ahasuerus 05:46, 3 November 2011 (UTC)
Well, I'm happy to take requests and it should be trivial to set the user_sites entries to OFF as default if it's an obscure site of limited interest to others - but you might have to tell individual users about it, it seems a lot of our users don't notice new features. And if any of the suggestions I've currently submitted would be better OFF by default than ON, let me know and I'll change those scripts. I'm not sure about Shelfari, for instance. BLongley 17:12, 3 November 2011 (UTC)
I've added Deutsche Nationalbibliothek as another option, it looks very useful. However I don't know if that's allowable by their terms of Service, hopefully Christian can find out for us. BLongley 19:27, 7 November 2011 (UTC)
I've also been talking with Bill Kendrick at Smashwords and got an admission that their documented API is broken. I doubt I can get them to use ours, but there might be a halfway point where they provide a data-dump and we catch-up at leisure. I'll let you know how the discussions go, and be assured I'm not going to let "Data Thief" be a Smashwords shill any more than Fixer is an Amazon one. BLongley 01:55, 3 November 2011 (UTC)
Sounds good! I seem to be coming down with something (it's that time of the year), but if I am operational, I'll try to implement the new links in the next couple of days. Ahasuerus 05:46, 3 November 2011 (UTC)
Angela at Smashwords has been asking about progress, and whether Mark Coker or Bill Kendrick has been in touch: I've confirmed that Bill has. We at least have confirmation that linking by ISBN-13 is stable. They seem to have retired the old API and moved to "ATOM", and these days, "OPDS", whatever those are, so I've given up on the idea of pulling the data. Maybe Fixer understands one of those abbreviations? BLongley 03:13, 9 November 2011 (UTC)
They are apparently XML-based, e.g. see this specification. Fixer understands XML and JSON (as well as Z39.50 and some other beasts), so I could certainly update his parser to handle Smashwords' data. It's just a question of man-hours. Looking at the site, they don't appear to have an overwhelming amount of SF-flavored books, so it shouldn't be too bad in terms of moderator overload. However, many of their pubs don't seem to have ISBNs, so it may be a bit tricky matching them with other sources. Ahasuerus 06:22, 9 November 2011 (UTC)
ISBNs seem to have been a late addition - they weren't included in the old API, but now that Smashwords offer them for free I think the uptake is rather higher. Especially as they're required by Apple and Sony. I'd hate to have to trawl those sites to find them though, if we can get them directly from the source. BLongley 17:27, 9 November 2011 (UTC)
Latest backup
Appears to suffer from this bug - additional 'USING BTREE' on the pubs table creation. I'm on "5.0.51a-community-nt MySQL Community Edition" so that step fails unless I remove it. Will see if that has caused any other problems.... BLongley 21:58, 6 November 2011 (UTC)
Thanks for checking! I have created and uploaded a MySQL 4.0-compatible version, could you please try it? Ahasuerus 23:14, 6 November 2011 (UTC)
Will do, but not tonight. Reapplying all the outstanding DB changes from the development queue took some time and testing such makes me think I could improve "FR 2823394 Let users choose a default language" implementation - it's not ideal for people that haven't already chosen some languages, which of course we haven't really encouraged people to do yet. (It's one I think we want, I've spent this weekend pretending to be a German Editor and found some of it to be quite a pain.) BLongley 01:01, 7 November 2011 (UTC)
European Library
While seeing if there was a Dutch equivalent of Deutsche Nationalbibliothek I came across "The European Library" which seems to be able to do an ISBN-lookup in three dozen national libraries. Even the British Library integrated catalogue, which Bluesman has recently found doesn't have stable direct links. This might be the most useful addition to "Other Sites" I've found yet. BLongley 00:18, 9 November 2011 (UTC)
Yes, it looks quite useful. I think I checked them out a few years ago when they were in beta (?) when their capabilities were less robust. My only concern is that we are adding so many choices to the "Other Sites" section that it is crowding out other stuff in the navbar. Perhaps we could use CSS stylesheets so that hovering over "Other Sites" would give you a list? I seem to recall that a pure CSS solution doesn't require JavaScript, but I haven't tried it. Also, with so many options, it may be beneficial to split "Other Sites" into "Bookstores", "Libraries and Catalogs" and "Social Cataloging Sites". Ahasuerus 05:59, 9 November 2011 (UTC)
Well, I expect most people will turn off a lot of sites - I know that I have turned off most US stores as I'm never going to buy from them and don't find them bibliographically useful. But yes, we could look into reducing the list a bit for those that haven't turned off the extras. (Although this might encourage more people to create accounts just so they CAN turn them off...) BLongley 17:21, 9 November 2011 (UTC)
Award Snafu
Can you have a look at ISFDB:Help_desk#Problem_with_Award_snafu_affecting_page_display please? I think it's something only you can fix at the moment. BLongley 18:37, 22 November 2011 (UTC)
Ouch. Looking into it... Ahasuerus 02:21, 23 November 2011 (UTC)
Fixed. The good news is that Fixer's main queue is empty, so I just need to add one more thing to Fixer's logic and then I will be free to work on the Python side. Ahasuerus 02:37, 23 November 2011 (UTC)
Thanks. It's been quite a while since I submitted the latest award changes and my memory of what's covered is weak, but I think I only worked on Award types and Award levels are a separate can of worms. But nasties like this deserve some urgent attention, even if there's more people crying out for further language support options than there are for award improvements. BLongley 02:46, 23 November 2011 (UTC)
Galaxy #9 (1995)
I went over the recently archived Help Desk topics, and was reminded of our discussion about this "missing" issue of Galaxy. I spent a little more time trying to find some information about it. While mostly unsuccessful, I did find a link to the online SF Encyclopedia that discusses it, with a little additional information. I added a link to that in the notes to our pub record. Chavey 01:34, 23 November 2011 (UTC)
Sheesh, I had no idea it (semi-)survived for as long as it apparently did! Thanks for digging up the SFE article! Ahasuerus 02:19, 23 November 2011 (UTC)
So what do we do next?
Recent changes seem to have had a mixed reception. :-/ I don't think they're what Michael expected, but Christian and Willem and Hervé seem to have liked them. Even Kraang, not our most vocal mod, seems to be trying them out (a little). If you have the time for Python changes again, please give me some direction to work on - fixing stuff, beefing up displays, ensuring "Unmerge Foreign Title" will work as expected, continue "Translator" support design, whatever. We're really unclear on what we're doing, for whom, and why. BLongley 02:17, 2 December 2011 (UTC)
And remember, "testing your stuff" is something I can try too. I'm not sure I'd have caught the problem from the last patch, but it's a possibility. BLongley 02:17, 2 December 2011 (UTC)
Played around with it a bit last night and if I understanding him right I agree with Michael about the display or how it's processed and then displayed. A separate section like “Foreign Language” would be better than “Variant Title”. Having the option to display or not on the main page is useful. Overall a good step forward!Kraang 03:04, 2 December 2011 (UTC)
Hmmm... I'd not considered a separate section. We could also separate the genuine variant titles from the variants only credited due to a difference in author name. Plenty of food for thought there, thanks Kraang! BLongley 03:12, 2 December 2011 (UTC)
That's a great idea! We really do have (at least) three kinds of variants: language, title, and author's credited name. And it would be nice to be able to handle them, or see them, slightly differently. But while you're thinking about that, I'll suggest a fourth type of variant: An adaptation, where (say) Trina Robbins adapts Tanith Lee's "Silver Metal Lover" for a children's market, or Tiptree's "Girl Who Was Plugged In" gets adapted as a musical. In these cases we have two choices: Create them as separate titles, with no VT link to the item from which they were created, or else we end up with Stray Publications messing things up. I wonder if a better solution could be found by a "special" type of VT for adaptations? Chavey 04:53, 2 December 2011 (UTC)
"Title" variants are presumably the original. "Author's credited name" seems a bit of a fudge, but we've lived with that for years. (Although recent "writing as" rule changes mean there's a lot of rework to do there...) "Language" is the latest change and doesn't seem to be universally popular - and we haven't got round to different translators yet! :-/ BLongley 05:28, 2 December 2011 (UTC)
The good news is (I think) that we can convert author variants easily, and if people are willing to work on languages we can do that too when the time comes. I think we need to work on the latest improvements a bit first as adaptations are not going to be automatically fixable - the nearest thing I can think of to help is providing a tool to help rework any title with "excerpt" or "revised" or "expanded", but the work would still be mostly manual. BLongley 05:28, 2 December 2011 (UTC)
I confess that I'm satisfied with the way the process goes. There are two things that interest me most in a bibliography : 1) the exhaustivity (we're probably overall the best for works in english perhaps excepting certain auhtor-specific sites) and 2) the global view of the author's reception or audience and this passes by a as-complete-as-possible list of the translations in all the possible ways. For example, it's quite interesting to see the wildly diverging popularities of writers like PKD (aduled in France but less in its own country before the films based on his books) or Harry Harrison (who is extremely popular in Russia per Tomlinson's book). That's why I'm using the full set of the avaliable languages and will stick to the "maximum" (but crowded) display. But I can understand that there are some people who'd like to work on a more restricted (national ?) basis. BTW, perhaps in a far future, a distinction between "english published in the UK" and "english published in the USA" and "english published in the Canada" (etc...) will be useful for some. Hauck 06:24, 2 December 2011 (UTC)
(unindent) No more commits until I clean up the queue, please. Since you are still learning the software, Bill (and heavens knows there is quite a bit to learn about this illegitimate offspring of functional and OO programming), most of your commits require subsequent changes. Thus, if I am testing version 1.33 and you have already committed versions 1.34, 1.35 and 1.36, I have to change and test not just 1.33, but 1.34-1.35, create new versions, document them, etc. Depending on the circumstances, certain workarounds are possible, but they can be time consuming as well. When I was testing the last big batch of changes, most of my time was spent on these issues, which is quite wasteful. And, unfortunately, my availability is the biggest development bottleneck that we currently have since I usually finish my workday between 12am and 3am. Ahasuerus 06:53, 2 December 2011 (UTC)
No problem, as mentioned before I'm holding back on a lot of stuff already and am happy to incorporate your changes to my changes rather than vice versa. In the absence of any programming style guidelines and solidly-agreed design discussions it's the best way for me to learn what I'm doing wrong. BLongley 18:05, 2 December 2011 (UTC)
I hate to point it out...
... given the current discussions about our language support improvements, but I think there's a bug in the latest change. Look at Wolfgang Jeschke and see what variants (or "translations" as I think Michael wants) are shown. Shouldn't "Der letzte Tag der Schöpfung" show "The Last Day of Creation" as a variant? By "default of English" rather than any other rule? BLongley 05:11, 3 December 2011 (UTC)
Excellent catch, fixed now! Ahasuerus 06:08, 3 December 2011 (UTC)
And to add to your headaches, that title seems to be a good reason to add "Kurd Laßwitz-Preis" as an award type. Which leads on to languages for awards, displays of awards on variant titles (partly fixed in the development queue), and maybe alphabet support - should "ß" be "ss" for now? I'm going to go sleep, my head is already spinning and I won't be available Saturday or Sunday morning. BLongley 05:20, 3 December 2011 (UTC)
Airmont Science Fiction series
I am organizing the Airmont Science Fiction series, and have added your verified pub Invaders from Rigel to this series. —The preceding unsigned comment was added by Chavey (talk • contribs) .
Thanks for the heads up! Ahasuerus 16:33, 4 December 2011 (UTC)
First Person, Peculiar
Added a cover image [great artwork!!] to [this]. Would have left this on the 'other' page but it's into it's ninth printing over two years...... ! ;-) --~ Bill, Bluesman 03:13, 5 December 2011 (UTC)
Dunsany Gods, Men and Ghosts verified pub ToC questions
Just getting started here, so apologies if I am missing the mark or being too simplistic. In Gods, Men and Ghosts (hope that worked, am copying examples I see) I have two questions, please: first, on page 219 is the Preface, which is labeled ESSAY -- since it is explanatory fictional material dealing with wholly fictional subjects, why is it ESSAY instead of SHORTFICTION? Second, in the Gods of Pegana section the sub-stories Of Skarl the Drummer (p220) and Of the Game of the Gods (p222) seem to be missing from the ToC, but all other sub-stories are present. Can you check your copy to confirm? Mvhetzel 16:07, 11 December 2011 (UTC)
You are absolutely right about "Of Skarl the Drummer" and "Of the Game of the Gods", I must have missed them because the titles appear at the top of their respective pages. I have added them to the publication and merged the new Title records with their pre-existing counterparts.
As far as the "Preface" goes, it's an "in-universe essay", which we usually enter as Shortfiction, so you are again correct that we will want to change the title type. However, this title record appears in three separate verified publications, so I have left messages on the verifiers' Talk pages to make sure that we have a consensus. Thanks for catching it! Ahasuerus 04:41, 12 December 2011 (UTC)
Webpages Bug
I'd like to call your attention to Talk:URL Issues#"Author not known". I searched Sourceforge and didn't find an applicable bug. Do you know if this is a new bug that should be written up or leftovers of an old one that just need to be cleaned up? -- JLaTondre (talk) 20:49, 13 December 2011 (UTC)
Checking mod/common.py, I see that deleteFromAuthorTable doesn't touch the "webpages" table, so presumably it's still an active bug. Good catch, I have created Bug 3459242. Ahasuerus 05:57, 14 December 2011 (UTC)
Yes, good spot. When do we get you developing fixes rather than just spotting our errors? ;-) BLongley 06:22, 14 December 2011 (UTC)
When you convert over to Perl? ;-) I'm not really familiar with Python. But you never know, you might inspire me to learn it as I wouldn't mind picking it up. -- JLaTondre (talk) 12:19, 14 December 2011 (UTC)
I'd never used Python before - but I've picked up enough now to keep Ahasuerus busy. There's quite a few small improvements that can be done while you're learning - and even some pure MySQL fixes could be useful. BLongley 19:52, 14 December 2011 (UTC)
Merging of publication series?
Hello, Ahasuerus! Bill had the idea that you are capable of working miracles; see here for background information. If you are overworked and/or it's not possible: just drop the idea. We have only entered a fraction of the series and renaming the individual publications should still be possible. If it's just a matter of time and there are other priorities then just take the necessary time. There's absolutely no pressure on you! Stonecreek 09:51, 21 December 2011 (UTC)
I think it should be doable. I will post about it tomorrow... Ahasuerus 06:15, 22 December 2011 (UTC)
Back to normal
Glad to hear you're feeling better! Now, remind me - what is this "normal" of which you speak? I don't think I've encountered it recently... ;-) BLongley 06:07, 23 December 2011 (UTC)
I am not sure if not being sick and catching up on one's sleep makes one "normal", but it's a first step :-)
I am in the middle of compiling a list of things that I am behind on and then I'll need to prioritize them. If I can spend a few hours a day on these tasks for the next 10 days, we should be at least partially caught up.
There are also things that I really need to do before the end of the year when Amazon may change its APIs yet again, e.g. capture the data at the 4 EU stores. Thankfully, their SF sections are relatively small, so it shouldn't take long. Ahasuerus 18:25, 24 December 2011 (UTC)

User talk:Ahasuerus/Archive/2011

Unknown December 1939

Development

Continued Fixer discussion

Bug 1820414: Page numbers are lost

Hope you get well soon

Latest FR

Award Editing changes

Publisher Bug?

FR 3279968 Add options to search title and publication notes

FR 2811812 Capture reason or summary of an edit

More Data Cleanup scripts?

Tales of Riverworld

Find Stray Publication Authors

Recent Rejects broken?

Bond's Lancelot Biggs: Spaceman

Not well yet?

Dynamic SF, October 1953

Bug 3293839

The Yawning Heights by Alexander Zinoviev

Damien Graves

Advanced search changes

Latest Developments

Award Editing

Chapterbook series

Blackjack time again?

The Stolen Sun and Tramontane

Four-Day Planet / Lone Star Planet

Russian Science Fiction 1969

Arcturus Landing

Awards Coding

Development page

Russian language issue

A step too far?

The Birthgrave (Tanith Lee)

getpub.cgi

Roesnner or Roessner in The Sword of Winter"?

broken submission

Thornton W. Burgess

Latest coding

Merges May Cause Self-Referring Titles

Working Languages

Unmerge titles

The Remarkable Exploits of Lancelot Biggs: Spaceman

Pill spammer?

edit/editpublisher.py

Russian publisher

Impact-20

Bibliography display oddity

Data Corruption Issue

Russian spelling: Help appreciated

Canonical names and diacriticals

"Sourcing" changes

Gordon R. Dickson - Beginnings

Eddy C. Berlin

Other other sites

Character sets

Smashwords and other "Other Sites"

Latest backup

European Library

Award Snafu

Galaxy #9 (1995)

So what do we do next?

I hate to point it out...

Airmont Science Fiction series

First Person, Peculiar

Dunsany Gods, Men and Ghosts verified pub ToC questions

Webpages Bug

Merging of publication series?

Back to normal

Navigation menu

Search