Difference between revisions of "User talk:Ahasuerus"

From ISFDB
Jump to navigation Jump to search
Line 675: Line 675:
  
 
Please see [http://www.isfdb.org/cgi-bin/view_submission.cgi?5372759 this submission]. In adding the additional contents, the submitter edited the two existing records to be different content. I have seen this happen before with users who think the content must be listed in order of appearance on the edit screen (i.e. are not aware of how sorting works). Generally not an issue, but in this case, the two existing records have associated awards. If this edit is accepted, the awards would now be associated to the wrong records. Is editing a record with an award something that should be flagged in the pub submission screen? Maybe it doesn't happen enough to worry about... I'm going to reject this submission and redo it. -- [[User:JLaTondre|JLaTondre]] ([[User talk:JLaTondre#top|talk]]) 09:37, 26 July 2022 (EDT)
 
Please see [http://www.isfdb.org/cgi-bin/view_submission.cgi?5372759 this submission]. In adding the additional contents, the submitter edited the two existing records to be different content. I have seen this happen before with users who think the content must be listed in order of appearance on the edit screen (i.e. are not aware of how sorting works). Generally not an issue, but in this case, the two existing records have associated awards. If this edit is accepted, the awards would now be associated to the wrong records. Is editing a record with an award something that should be flagged in the pub submission screen? Maybe it doesn't happen enough to worry about... I'm going to reject this submission and redo it. -- [[User:JLaTondre|JLaTondre]] ([[User talk:JLaTondre#top|talk]]) 09:37, 26 July 2022 (EDT)
 +
 +
: Wouldn't that also mess up any reviews associated with the changed title records? [[User:Ahasuerus|Ahasuerus]] 11:16, 26 July 2022 (EDT)

Revision as of 10:16, 26 July 2022

See User talk:Ahasuerus/Archive for discussions prior to 2022.

PLEASE NOTE:

If you're writing to inform me that you've either added a COVER IMAGE or NOTES to any of my VERIFIED PUBS, please follow THIS LINK and add it to the bottom of the list. A link to the pub record would be appreciated. Once the pub has been reviewed, I'll remove your note from the list. Thanks!

Author Merges & Alternate Names

I merged "J. C. H. Rigby" (148444) and "JCH Rigby" (287823) resulting in the "new" JCH Rigby. One of them (I'm not certain which at this point, but think it was the 287823 entry) had an alternate name of Charlie Rigby. When I did the merge, the alternate name was lost. That caught me by surprise. When a merge is done, the normal expectation would all the information is merged. Title merges handle variants so I would have expected author names to handle alternate names. Ideally, it should handle the alternate name. If not, there need to be a big warning that further action is required. I wonder if we have lost alternate names because of this... -- JLaTondre (talk) 09:06, 1 January 2022 (EST)

Let me take a look... Ahasuerus 11:33, 1 January 2022 (EST)
OK, I think I can see the sequence of events. Let me run the weekly/monthly backups and then I will try to recreate the problem on the development server. Ahasuerus 12:17, 1 January 2022 (EST)
I have wrapped up and deployed the cleanup reports which were in a semi-ready state yesterday. Hopefully I can recreate and fix this problem tomorrow. Ahasuerus 20:19, 2 January 2022 (EST)
I have been able to recreate the problem on the development server. Luckily, it doesn't seem to affect VTs, so any lost "alternate name" associates will appear on the cleanup report that looks for "stray" publications. Bug 795 has been created. Thanks for identifying the problem! Ahasuerus 17:11, 3 January 2022 (EST)
OK, I believe the bug has been fixed. Ahasuerus 18:06, 3 January 2022 (EST)
Out of curiosity: aren't we standardizing names anymore to J. C. H. Rigby? Regards, MagicUnk 05:32, 4 January 2022 (EST)
Originally, we always added a period and a space after an initial. As Template:PublicationFields:Author says:
  • Initials should normally be entered followed by a period and a space as "Gordon R. Dickson" or "K. D. Wentworth", even if the period or space is omitted in the publication.
However, we later realized that some authors deliberately use their initials as a "pseudo-name" like "JCH" and that the practice has been getting more common recently. That's when we updated Template:PublicationFields:Author with the following caveat:
  • However, when it is clearly the author's choice to omit the period, or when the author has a single letter name that is not an initial (e.g. "Harry S Truman") the period should be omitted. In the rare case where an author prefers two (or more) initials as if they were a name (such as "TG Theodore"), without a period or space, and is so credited, we follow the author's preference.
Ideally, our search logic would ignore periods and other punctuation in names, but we aren't there, so this s the best compromise we could come up with. Ahasuerus 10:08, 4 January 2022 (EST)
Given we can have alternate names for authors, would it make sense to create an ISFDB-standard name as an alternate for search purposes? ../Doug H 13:36, 1 February 2022 (EST)
It's been known to happen, e.g. see PS Cottier, an alternate name used by P. S. Cottier. I am not sure if this is the consensus approach at this time, though. Something to ask on the Rules and Standards page, perhaps? Ahasuerus 14:58, 1 February 2022 (EST)

Rejection Issue

I screwed up the HTML on this submission which is causing the moderator display to not show correctly. I tried editing the URL to change it to http://www.isfdb.org/cgi-bin/mod/reject.cgi?5187168, but that seems to get the ID from the post and not the URL. Need some help on how to reject this submission. Sorry for the inconvenience. -- JLaTondre (talk) 11:51, 2 January 2022 (EST)

Done. You were close -- the correct CGI script name is "hardreject" :-) Ahasuerus 12:19, 2 January 2022 (EST)

Misaligned

http://www.isfdb.org/cgi-bin/publisheryear.cgi?35762+1991; http://www.isfdb.org/cgi-bin/publisheryear.cgi?35762+1992; Another 2 cases where the ? mark after the last title causes it to be in a different row. I think you're the one who fixes this kind of stuff. --Username 11:43, 1 February 2022 (EST)

That's right, I usually handle software issues. However, I am looking at these two Web pages right now and they appear to be displayed correctly. I have tested it under Firefox, Google Chrome, MS Edge and Tor so far. Which browser and which version are you using? Ahasuerus 12:40, 1 February 2022 (EST)
I recall that months ago I brought this up and I think it was you who discovered it was some issue with covers with that rollover thing next to the title that were causing the problem, and then you fixed it; it's on 1 of these message boards somewhere. Recently, I brought it up again after discovering more of them, but was told you were in charge of that stuff, but I don't think I bothered asking you about it then. Now that I came across these 2 problems by the same publisher (in Poland), I see that the first 2 covers are displayed in 1 row and the 3rd cover (the ones with the ? next to them) are displayed below the first two, in a separate row. I believe this is exactly the same issue as the other times I brought it up, so I guess the answer is to do whatever you did that first time to fix it. I use Google Chrome on a laptop with Windows 8.1. --Username 12:49, 1 February 2022 (EST)
Oh, you are looking at http://www.isfdb.org/cgi-bin/publisheryear.cgi?35762+1991+1 and http://www.isfdb.org/cgi-bin/publisheryear.cgi?35762+1992+1 (note the additional "+1" at the end), which display cover scans for these publications. Now I see the issue. Let me check our archives to see what was causing similar problems in the past... Ahasuerus 13:06, 1 February 2022 (EST)
OK, it should be fixed now. Could you please check if everything looks OK? Thanks for reporting it! Ahasuerus 13:21, 1 February 2022 (EST)
Yes, they're fine now. I found my previous message, http://www.isfdb.org/wiki/index.php/ISFDB:Moderator_noticeboard#Jumbled_Covers, in case you want to look at those, too. --Username 13:26, 1 February 2022 (EST)
Thanks, I have updated that discussion on the Moderator Noticeboard page.
What happened was that I fixed this issue on the Title Covers page back when it was first reported some months ago, but I didn't realize that the Publisher Year page had the some problem. And then I missed your January post on the Moderator Noticeboard.
If you come across any other ISFDB pages which display mouseover help for covers and mess up the display, please let me know. Ahasuerus 14:28, 1 February 2022 (EST)

Author:W._A._Harbinson

It looks like there was only one use made of the Melvyl template. I notice you created the author wiki page Author:W._A._Harbinson and in the last sentence said what he'd been doing for 'the last eight years'. Checking the log, seems that was back in 2008. Bit misleading after 13 years. I wonder what kind of editor let that kind of thing get on the wiki. ../Doug H 14:55, 1 February 2022 (EST)

I have moved the author-provided autobiography to the database proper, clarified that it was posted in August 2006 and deleted the Wiki page. Thanks for catching it! Ahasuerus 15:08, 1 February 2022 (EST)

Dating publications - a thought

My last point in the discussion was that of verified data being sacrosanct and all else to be annotated with source. I've been digging through multiple sources to document some non-genre Asimov most recently OCLC. I dug into entry with 4 sources with a date of [1954] and got c1954, sixth printing (in associated scan only), one doesn't have the book, and the fourth connection just hung. I'm currently wrestling with how to record such references when I create a pub, because if I don't record it, someone else might either use it to create new publications or modify an existing one. With that for context, I can see that the idea of documenting sources will need to be addressed as a community. You're probably already wrestling with it, let me know if I can help. ../Doug H 12:42, 4 February 2022 (EST)

Sorry, I missed this message when it was posted. I will try to respond tomorrow. Ahasuerus 00:17, 8 February 2022 (EST)
Yes, secondary sources like OCLC and other online catalogs present a variety of challenges. At one point we tried to describe them in Help:How to parse data in library catalogs and Help:Using Worldcat data, but it's been a number of years since the instructions were last updated and they are still incomplete. For example, the former says "Need to add an explanation of the MARC family of standards, SUTRS and the OCLC guidelines here", which hasn't been touched since 2008.
If you have ideas/suggestions re: improving these Help pages, creating new ones or documenting third party information in Notes, please don't hesitate to post them on the Community Portal! Ahasuerus 09:01, 8 February 2022 (EST)
I've added some text to the Help Entry for parsing (sans discussion on a Portal as it is background and context) and came to the realization that I don't know why you'd want the MARC family of standards (et al) on there. OCLC won't give you the MARC version (as far as I can tell) and all it does is format what's displayed. So unless one plans on automating the pull, I don't see why one would want the machine-readable format. Particularly since it is notoriously difficult to parse. ../Doug H 15:53, 9 February 2022 (EST)

Mismatched Double Quotes

The Mismatched Double Quotes cleanup report might need an ignore option. A single double quote is a valid abbreviation for inch (which is the usage in the two current records). Or have it ignore word boundary + number + double quote (\w\d+"). -- JLaTondre (talk) 21:37, 7 February 2022 (EST)

Good point. FR 1485, "Allow ignoring records on the Mismatched Double Quotes cleanup report", has been created. Ahasuerus 08:54, 8 February 2022 (EST)
After examining the code, I see that this is a bigger mess than I realized. The report is actually limited to records with notes which contain mismatched double quotes AND the string "http". In other words, it's looking for malformed URLs. The logic works most of the time, but notes which contain a URL plus a separate mismatched quote throw it off. In addition, we have another cleanup report, "Invalid HREFs in Publication Notes", which does something similar -- but not the same -- for Publication notes only.
Let me see if I can expand the second report to cover all record types and then fold the first report into it. Ahasuerus 10:45, 8 February 2022 (EST)
Done. Ahasuerus 17:16, 8 February 2022 (EST)
Thanks. -- JLaTondre (talk) 18:39, 8 February 2022 (EST)

Info for authors

If you were to talk to a group of authors about ISFDB, what would be the first things you'd share with them? I'm going to be doing that, and I'd be interested to know the things you'd think are most important for them to know. Thanks! ···日本穣 · 投稿 · Talk to Nihonjoe 17:20, 8 February 2022 (EST)

Off the top of my head:
  • Eligibility - what cannot be added to the DB. Their site or blog are not producing a "published" story. That magazine in Lithuania is.
  • The fact that we are a historical DB - if you EVER published it, we won't delete it. No matter how ashamed you are of it.
  • They won't see the magazines and anthologies they have stories in on their pages - they will see the stories themselves.
  • If they need help, we will assist them by adding or editing or teaching them how to add to the DB. They are welcome to add their own stories and novels or have someone else add for them or ask for help for someone to add (and that includes translations and we have multi-language editors to assist).
For example... I may have done some of it with some of authors :) Annie 17:34, 8 February 2022 (EST)
Apologies for intruding, but a couple of observations/suggestions, based on stuff I've seen on social media:
* Re. eligibility, I don't think it's obvious to newbies that the way to add stories is via the pubs they appear in. See this exchange for an example
* It's not at all obvious to visitors that the primary means of communication is via the Wiki, hence stuff like that prior example or this one of authors crying into the void for help.
* I've also seen ISFDB described as a wiki, which I guess it is in the general sense of a community of contributed info, but not if you're thinking of stuff like MediaWiki, Confluence, Sharepoint (or whatever MS offer in that vein these days). In particular, it might be worth pointing out that having all submissions be moderator approved means that data should be more accurate than Wikipedia, Goodreads, etc ErsatzCulture 18:42, 8 February 2022 (EST)
How about asking them to make sure their full names, correct birth dates/places of birth, and updated websites (if any) are in their records; today I corrected over a dozen "Calfornia" misspellings for places of birth, plus many records have just a year for birth dates with no day or month (many death dates are also questionable, but you can't ask those authors because they're dead). --Username 18:48, 8 February 2022 (EST)
I would probably present it as two separate questions. The first one is "Why is bibliography important when you are an author?" The second one is "What makes ISFDB stand out among other online bibliographies?"
The primary answer to the first question is "Assistance with discovery". When you finish reading an exciting book/story, one of the first questions that you ask yourself is whether there are more books by the same author or similar books by similar authors. A comprehensive bibliography can answer the first question and help with the second question. For example, Mountaindale Press specializes in LitRPG, so it's a good place to check out if you are interested in LitRPG. Ditto Yen On, Airship, and J-Novel Club for light novels. Tor and Baen have their fans who pick books based on the publisher. Major media franchises like Star Trek, Star Wars, Warhammer, etc are good places for new authors to put their name on one or more bestseller lists. Etc. A bibliography which lets readers hop from author to series to publisher to author ad infinitum is a useful discovery tool.
The answer to the second question is that no single bibliography is perfect, but ISFDB has certain advantages compared to other online alternatives. Goodreads has more ISBNs and ASINs, but they are not as well organized and searching for short fiction can be challenging. Amazon has a LOT more ISBNs and ASINs, but, again, organization and short fiction can be problematic. On the flip side, they have reviews and more robust tag systems.
Finally, I have seen editors comment on how useful the ISFDB database can be when putting together anthologies and collections. It may not benefit authors directly, but it's likely an indirect benefit. Ahasuerus 18:56, 8 February 2022 (EST)
That's a good way to look at it - as long as the discoverability is paired with the "but we keep records of all the books we know of, even if you they are not available or you do not want them to be listed or if they were authorized editions". :) Annie 19:27, 8 February 2022 (EST)
Re: Eligibility: Thus my third point - which pivots into explaining how you add your stories :) Annie 19:27, 8 February 2022 (EST)
This is all great information, everyone. Thanks! ···日本穣 · 投稿 · Talk to Nihonjoe 16:19, 9 February 2022 (EST)

Ossuary

http://www.isfdb.org/cgi-bin/pl.cgi?647581; You entered this more than 4 years ago and today I saw it on FantLab. I added cover and fixed publisher, but there's a photo with contents that differ slightly from what you noted, plus there's a cover art credit but I don't see a name on the cover so I didn't enter it; if you're interested you might want to update some things. --Username 11:50, 9 February 2022 (EST)

Updated, thanks. Ahasuerus 13:24, 9 February 2022 (EST)
BTW, there's only 1 Kotarbinsky on ISFDB, Vasily, http://www.isfdb.org/cgi-bin/ea.cgi?271194, with an interior art credit on nearly the same date, so that's probably who it is. --Username 13:31, 9 February 2022 (EST)
Updated, thanks. Ahasuerus 16:47, 9 February 2022 (EST)

Just double-checking

To make sure my understanding is correct. Thanks! ···日本穣 · 投稿 · Talk to Nihonjoe 14:38, 9 February 2022 (EST)

Response posted. Ahasuerus 16:26, 9 February 2022 (EST)

Fateful Lightning cover - non /P/isbn cover

I've just attached a cover to [1] (when the edit goes through) which might match the printing you've PVed. --GlennMcG 01:26, 10 February 2022 (EST)

Yes, my first printing has the same cover art plus the following thin line:
  • ROC * 451-LE5196 * (CANADA $6.99) * U.S. $5.99
at the bottom. I have also confirmed that it's the same cover as the one used on the fifth printing published in 1998-01, which Locus1 attributes to Sanjulian, so now we have the name of the cover artist. Thanks. Ahasuerus 10:20, 10 February 2022 (EST)

Account Recovery

Hello!

Attempted to login yesterday and today with no success. Granted it has been a long while... Used the Wiki feature to reset my password, and it claims an email was sent, but I've been checking my inbox and spam folder all day to no avail.

From what I recall the Wiki was always... quirky. I suspect that hasn't changed, so curious if this is a glitch of some sort or something actually wrong on my end. Regardless, not sure what my options are, or if I should just plan on soldiering on with a new username. Wasn't sure where else to turn.

Cheers, --Albinoflea-Spurned 21:24, 22 February 2022 (EST) (The user/former moderator once known as Albinoflea)

Welcome back :) Mails are... not working with all servers. Let's hope Ahasuerus can help! :) Annie 22:13, 22 February 2022 (EST)
Welcome back! Let's start with the basics -- are you still using s******.f*****@gmail.com as your email address? The reason I am asking is that I don't see this email address in the ISFDB server's outgoing mail queue, so I wonder if the address may have been changed. Ahasuerus 23:22, 22 February 2022 (EST)
Yes, that email is correct, and I still have copies of emails I received from ISFDB users and even an old password reset email from 2014 that were sent to my inbox. Albinoflea-Spurned 14:41, 23 February 2022 (EST)
I see. I have just changed your password using a Wiki maintenance script and sent the new password to your Gmail account. Could you please let me know when you get the email message and whether the new password worked? Ahasuerus 16:18, 23 February 2022 (EST)
Thanks, that one went to spam but I retrieved it without any trouble. Logged in, updated password and all seems good. Thanks! Albinoflea 16:52, 23 February 2022 (EST)
Excellent! :-) Ahasuerus 16:54, 23 February 2022 (EST)

Importing Content Labeled as Proposed Clone

Please see this submission: This is an import content submission, but the title at the top of the moderator screen is listed as "Proposed Clone Publication Submission". On the public view, it correctly states "Pending ImportExport Submission". Thanks. -- JLaTondre (talk) 08:38, 24 February 2022 (EST)

Yup, it's a known issue on my list of things to fix -- Bug 787, "Import/Export submission review has wrong header". Ahasuerus 11:31, 24 February 2022 (EST)
And fixed. Ahasuerus 17:58, 24 February 2022 (EST)
Thanks. -- JLaTondre (talk) 18:06, 27 February 2022 (EST)

Nav bar: something a bit disorienting for a new self-approver

When I clicked on a "Self-approver view" link (e.g. http://www.isfdb.org/cgi-bin/mod/submission_review.cgi?5243123) this added the "Moderator Links" section to the bottom of the nav bar. I wasn't sure what delights might await on those pages, so I clicked on each of the links, but they all give a "Moderator privileges are required for this option" page. My initial thought was that there was maybe some cookie/session magic involved, so I logged out and back in, and was a bit confused when that Moderator Links section had disappeared.

Having gone back through my browser history, and checked out the code, I can see now that that stuff gets added to the nav bar for any page with code in the /mod/ subdirectory. Assuming there aren't any cases where those sidebar links do work for self approvers, it might be nice to wrap the coder that outputs that section to be wrapped in a 'if moderator:' test, and so not render them for editors who can't make use of them?

This would be a very low priority change, as I guess the number of people affected is in the single digits, and they'll all work it out, but maybe it's a case of adding the if test and indenting the block of lines that output those links?

(I can see the code a bit further down mod/isfdblib.py references moderator, self_approver and SelfApprovalAllowed(userid), so perhaps there are complexities involved in the privileges that make this not as trivial as I might have made out though?) —The preceding unsigned comment was added by ErsatzCulture (talkcontribs) .

The "Self-approver view" link was added to the Pending Submission page just last week, so it's probably an oversight. Let me take a closer look... Ahasuerus 17:55, 25 February 2022 (EST)
OK, I think I got it. Could you please try again and see if everything looks OK on your end? Ahasuerus 18:20, 25 February 2022 (EST)
Thanks - I've just been through the edit/approval cycle a couple of times and didn't see the mod links appear. ErsatzCulture 18:32, 25 February 2022 (EST)
Excellent! :) Ahasuerus 18:35, 25 February 2022 (EST)

Reports Paging

Hi Ahasuerus

I've been working through my list of unstable amazon images and I noticed an entirely inconsequential bug with the paging. I just got the report down to one page, 200 rows, and there was still a link to the second page of records. Clicking the link brings you to an empty page. I've since dropped below 200 rows, and there is no link to the second page, as one would expect. I'm guessing that this is the difference between >= vs >. Although it's completely trivial, I thought I'd bring it to your attention. Thanks. --Ron ~ RtraceTalk 07:12, 3 March 2022 (EST)

Thanks, I'll take a look. Appreciate the heads-up! Ahasuerus 10:20, 3 March 2022 (EST)
OK, I see what's going on. Basically, the SQL query says "Give me the next 200 matches". If the database engine returns less than 200 matches, then the software doesn't display a link to the next page. If the engine returns 200 matches, then a link is displayed.
The problem with this logic is that it doesn't distinguish between "there are exactly 200 matches on file" and "there are more than 200 matches on file". What the software needs to do is request 201 matches. I'll see what I can do after finishing the round of changes that I currently have in the works. Ahasuerus 13:39, 3 March 2022 (EST)
OK, the 3 "My Primary Verifications" tables have been fixed. I still need to fix a bunch of other tables that have the same issue, but it may take a bit since they are all somewhat different. Ahasuerus 16:55, 4 March 2022 (EST)

Top Contributors et al

Hi Ahasuerus

I suspect that the Top Contributors and similar reports are stuck again. The most recent Last User Activity is for March 6. As before, this isn't urgent, but I thought you'd want to know. Thanks. --Ron ~ RtraceTalk 07:10, 8 March 2022 (EST)

All "Database Statistics" reports were moved to a weekly schedule last week -- see Nightly processing getting split for details. Most of them say "This report is generated once a week" at the top of the page, but I see that "Top Contributors" ones do not. Let me add that real quick... Ahasuerus 10:12, 8 March 2022 (EST)
Done -- see Top ISFDB contributors (All Submission Types). Thanks for the heads-up. Ahasuerus 12:01, 8 March 2022 (EST)
I won't say that I'm not disappointed, but I do understand the necessity of the change. I recall when these reports were on demand, and I did adjust after they went daily. Thanks for the explanation. --Ron ~ RtraceTalk 21:41, 8 March 2022 (EST)
In a way, we are victims of our success. The more records and submissions we have, the longer it takes to generate reports. Unfortunately, many reports draw data from multiple tables and lock them while the data is compiled.
Ideally, we would move all of our report-generating processes to a different server and then shuffle the backups and the compiled data back and forth between the production server and the backup server, but it would require a fair amount of juggling. Ahasuerus 22:23, 8 March 2022 (EST)

Patches, testing and a proposal

Hi. Sorry for not submitting any further patches after the last one. Offline life caught up to me and suddenly I have a lot less spare time. /Lokal_Profil 18:08, 13 March 2022 (EDT)

No worries whatsoever. Real life happens. Ahasuerus 19:47, 13 March 2022 (EDT)

I struggled a fair bit with setting up a local development environment. I simply don't have space for a full virtual machine running the right version of sql etc. so I aimed at just getting the python bit up and running. After some hacking I managed to get a virtual environment up and running for Python 2.5.4. My plan was to introduce some unit tests so that I could at least do some initial refactoring with a fairly high certainty that it wouldn't have any undesired effects. What I then came across was that the imports rely on the required python files from the common directory being copied across to the other directories when the system is built. This put a stop to my unit test plans.

What I wanted to check with you is if it would be of interest if I re-arranged the import and folder structure as a package instead. This would make unit testing possible but should also simplify both a future migration to a newer python version and make the codebase easier to understand for new developers. Even if it might be possible to split the patch up into a few incremental changes the end result still touches most of the codebase and as such would require extensive testing. I can therefore see how it might not be a desired change. If you want me to go ahead with it I will try to get it done in my spare time and let you know when it is ready for testing. If you don't then I fully understand =) Cheers, /Lokal_Profil 18:08, 13 March 2022 (EDT)

At one point I looked into restructuring our copy/import system. What I found was that the parts of Python which support packages and imports were significantly changed and expanded in Python 2.6 and 2.7. I figured it would be better to wait until we could upgrade to 2.7 and then revisit the issue. More recently, Al started looking into a possible Python 3.x upgrade -- see Development/HTTPS -- which may add another wrinkle. Ahasuerus 19:47, 13 March 2022 (EDT)
I believe the main changes are to how relative imports are handled. Since absolute imports are generally recommended I would have gone for that (should then still work for python 3). I believe the change is desirable before the python 3 migration (since it allows for easier testing) but can probably happen before or after a python 2.7 migration. Happy to wait until after such a migration but I'm also equally happy to go ahead as a preparatory step for auch a migration. /Lokal_Profil 05:07, 14 March 2022 (EDT)
Would it be possible/feasible to try it with a small, self-contained part of the system? For example, common/sfe3.py is copied to all directories, but it's only used in edit/cleanup.py, edit/sfe3_authors.py and nightly/weekly_job.py. Would it be a good guinea pig? Ahasuerus 09:43, 16 March 2022 (EDT)
I looked into what the file structure looks like once the files land in /www/cgi-bin/ and realised absolute imports won't work there unless bigger changes are made. Relative imports might be possible depending on how the files are executed, but this might also be where python 2.6 would be needed (for the __package__ attribute).
The smallest change I could envision is to change the scripts in rest/. I prepared a set of patches for this (1 2 3) before I spotted the above mentioned issue with absolute imports. Using relative imports instead the first of these patches could probably be dropped and the third would have to be modified somewhat.
But based on the above I'd say your original appraisal was right and these changes should probably await the python bump (to 2.7 at least). Even then since the directory structure in the execution environment doesn't mirror that of the repository (since /biblio becomes the cgi-bin root) there would still have to be some copying of common files left. /Lokal_Profil 19:14, 19 March 2022 (EDT)
Thanks for digging! It sounds like 2.7 is the first step then. I have it on my list of things to do and we need it to be able to do certain other things as well, so I will bump it up. Ahasuerus 09:41, 21 March 2022 (EDT)

Cleanup Reports

The "Cleanup Reports" link off the left menu bar is giving a "Internal Server Error". -- JLaTondre (talk) 15:55, 16 March 2022 (EDT)

Fixed -- sorry about that! Ahasuerus 16:03, 16 March 2022 (EDT)

Database backup files not working

(I wouldn't normally jump on these so quickly, but it looks like you updated the Wiki at the same time I made an unrelated edit...)

The Google Drive link doesn't seem to work for me - the header of the page says "backup-MySQL-55-2022-03-26.zip", but I don't get the usually slightly-crappy UI about the file being too large to virus check, and do I want to download it instead. Tried in latest(ish?) Chrome and Vivaldi so far; both should have Gmail/Google Apps user login/session cookies, should that make any difference.

Just tried last week's backup (which I had grabbed OK last week), and that similarly fails to offer me anything to d/l, although it did seem to have a spinner showing for a few seconds, which I don't recall seeing before.

I did see you'd updated the URLs for the image backups a few days ago, has something changed on Google Drive? ErsatzCulture 12:47, 26 March 2022 (EDT)

Update: If I click on the download icon in the top right of the Google Drive page, it does let me d/l the zip file, but the default UI just shows a largely empty page. Probably an issue on the Google side being funny with zip files, but it's rather disconcerting for an unaware user. Hopefully it's something they fix, but if not, maybe will need some help notes on the Wiki page? ErsatzCulture 12:51, 26 March 2022 (EDT)
Good point -- done. Thanks for the heads-up. Ahasuerus 13:15, 26 March 2022 (EDT)

Weird Record

Take a look at this title - no author and no date and it doesn't display the type. Something weird happened, though from the publication history, not clear what. This seems like an extransous record (there is another record in the pub of the same title for an essay) that can be removed and deleted. But thought I'd mention it first in case you wanted to take a look. -- JLaTondre (talk) 09:12, 3 April 2022 (EDT)

Thanks for reporting the issue. It turns out that this 2022-03-18 EditPub submission errored out, which is why it's not visible in Edit History. Luckily, it errored out while processing the last title record, The Third World: Space Powers of 2005?, and the rest of the submitted titles appear to be OK. I have removed and deleted the offending title. Ahasuerus 09:54, 3 April 2022 (EDT)

HTTPS-related bug when cloning pub without a cover image?

Just tried to clone this pub without a cover image, and get a Python stack trace error - the exception is "<type 'exceptions.AttributeError'>: 'NoneType' object has no attribute 'startswith'" and the relevant bits of the stack trace look to be:

 pub_image = ISFDBHostCorrection(pub_data[PUB_IMAGE])

and

 if value.startswith('http://www.isfdb.org/wiki/'):

I'm too lazy to get my dev box working with the latest code, but it feels like the second line should be "if value and value.startswith..." to protect against pubs without images? ErsatzCulture 14:49, 3 April 2022 (EDT)

That's exactly right. A fix has been deployed. Thanks for reporting the problem. Ahasuerus 17:13, 3 April 2022 (EDT)

Fixer and "recycled" ISBNs

Hi, please see this convo - any idea if Fixer will pick up the new pubs if they are using the older, apparently never-published, ISBNs? Thanks! ErsatzCulture 19:03, 9 April 2022 (EDT)

I am afraid Fixer has no way of telling that an ISBN has been reused. Once Fixer notices an ISBN in the main database, he makes a note of that ISBN's record in his data store so that it would never be submitted again. Ahasuerus 11:04, 10 April 2022 (EDT)
Thanks - at least this one is on the radar now, so it can be manually dealt with closer to the pub date. ErsatzCulture 15:36, 10 April 2022 (EDT)

(BTW, unrelated to this, but related to a talk item earlier on this page, when I d/led today's database backup, the Google Drive UI behaved as it used to, rather than presenting a blank-ish page. Guessing they fixed some bug/regression since I grabbed last week's backup...) ErsatzCulture 19:03, 9 April 2022 (EDT)

Updated, thanks. Ahasuerus 11:04, 10 April 2022 (EDT)

CC License update

We should probably update the license linked in the menu sidebar to point to here instead of here. The former is an updated version of the latter, while still being pretty much the same license. ···日本穣 · 投稿 · Talk to Nihonjoe 16:33, 18 April 2022 (EDT)

That's a good point, thanks. Let me ask Al, who knows a lot more about licenses. Ahasuerus 20:12, 18 April 2022 (EDT)

Database Errors

Seeing "<class '_mysql_exceptions.OperationalError'>: (1030, 'Got error 28 from storage engine')" errors all over the place (viewing authors, viewing titles, viewing submissions, etc.). -- JLaTondre (talk) 09:38, 23 April 2022 (EDT)

I am not seeing any errors right now. Given the timestamp of your message -- 09:38am -- I suspect that it was due to the backups which run every morning at 9:30am. I'll keep an eye on things. Ahasuerus 10:08, 23 April 2022 (EDT)

Bureaucrat warning needed?

Perhaps I'm prejudiced, but to me this looks like a personal attack. Despite Nihonjoe's attempts to cool things down, it could get out of control. --Willem 04:39, 27 April 2022 (EDT)

Thanks! Working on it... Ahasuerus 13:35, 27 April 2022 (EDT)

The Songs of Summer

In regards to this publication. If you look closely you will notice a vertical line along the whole of the artwork at the top. The first inch appears to be from a totally different piece of artwork, shows square structures on stilts - it's signed 'Heller' just below the Pan logo (top front cover) so definately the same artist but doesn't match the sea serpent part of the main image - I did an experiment by matching the top part of the book with the bottom in Photoshop and they definately don't marry. I'm going to make a note about this. --Mavmaramis 11:57, 14 May 2022 (EDT)

Interesting. There is this image on Twitter, but I don't think it sheds additional light on the mystery. Ahasuerus 12:40, 14 May 2022 (EDT)
That image comes from Heroic Dreams and is a cropped version of the image that appears on the book. Makes me think that at the composite stage of the book production - the part where they have the image and lay over the text and logos - they had a choice between two Heller imaages that were on seperate layers, added the text to the sea serpent layer and forgot about the 2nd image on the other layer and flattened/merged the layers. The image also appears in Six Fantasy Artists at Work: Dream Makers but I don't have a copy of that book to compare. I've written a message about it on Biomassbob's talk page as he's the only verifier of Dream Makers. Edit: I did find this image which shows the 'error' - whether deliberate or not I can't say but definaely very strange. --Mavmaramis 12:51, 14 May 2022 (EDT)

Preserving the PVs of a to-be-deleted pub record

Hello Ahasuerus. When you've got time, could you please have a look at this discussion re the problem evoked above ? I don't know whether it is solvable. Thanks a lot ! Linguist 11:08, 21 May 2022 (EDT).

I am afraid the software doesn't support the ability to move verifications from one publication record to another :-( Ahasuerus 13:13, 21 May 2022 (EDT)
Thanks for your response, anyway ! Cheers, Linguist 04:30, 22 May 2022 (EDT).

Request: WatchCover template similar to the WatchDate one

There are several upcoming pubs that have placeholder covers e.g. [2], [3], and I've fixed a number of older ones as I've encountered them. To try to make it easier to catch and fix these, could we have a { { watchCover } } template and associated report, so that these dodgy covers can be noted when they're first added, and fixed as-and-when better images come along?

I haven't looked at the code, but I'm guessing/hoping it should be a fairly straightforward copypaste from the existing { { watchDate } } functionality? Thanks ErsatzCulture 13:24, 31 May 2022 (EDT)

We have a spanking new bureaucrat option, "Add a New ISFDB Template" -- see Help:Screen:BureaucratMenu -- so a new template would be trivial to add. A cleanup report wouldn't take long to code either. I suggest you post a proposal on the Community Portal and see if there are any objections (unlikely) or alternative ideas.
Personally, I am all for it because it would help with light novels. US publishers frequently use original Japanese covers as placeholders for forthcoming translations. Ahasuerus 17:50, 31 May 2022 (EDT)
I wonder if we should not make that a more generic "watch (parameter)" template (where parameter can be date, format, cover, author name, title, subtitle, pages and anything else that looks suspicious and may need a second check (I use tags for that now but as they are up on the title level, it is not always easy to remember which pub they were about - especially with reprints). That way we don't have too many templates people keep forgetting to use. The wording will be the same (with the parameter notes in there and the cleanup report can show what is the watched element in a separate column (for example). Annie 18:06, 31 May 2022 (EDT)
That's an interesting idea. Currently "WatchDate" expands to "Publication date is based on questionable pre-publication information and may be incorrect." Will the parameter taken by the proposed "Watch" template be expected to be "Publication date", "Subtitle", "Page count", etc? And will the static text continued to be "is based on questionable pre-publication information and may be incorrect."?
If so, we may want to consider adding another template, "MultiWatch", which will expand to something like "The following values -- [list of field names like publication date, subtitle, page count, etc as the parameter value] -- are based on questionable pre-publication information and may be incorrect." Ahasuerus 18:42, 31 May 2022 (EDT)
Pretty much. We may want to tweak it to be "questionable pre-publication or other secondary sources information" to expand the scope a bit or we can have two separate templates (one for pre-pubs and one for other cases - they require different handling sometimes so having separate ones may make more sense) but that's the idea. That will also allow for new fields to be just added into the list without the need for a new template and without people trying to remember 10 different template names (and that way it can even be used on a title record for notes on translators, being or not genre (a common problem when adding early), author name form, the & vs and and so on).
The idea is to make these pubs (and eventually titles) easily foundable by editors who can assist with finding the data later (or elsewhere). We may even want to come up with a wording of the expanded sentence that will allow the reason for the suspicion to be part of the template (so we can say (watch|author name (title page not available), subtitle) but just being able to list the suspect values will be enormous help - especially on pre-publication records. For example "The following data is based on questionable pre-publication information and may be incorrect: (parameter here exactly as written by the editor)." or something along these lines. Then you don't even need two patterns (single and multi). Either way will give is better flexibility than what we have now. Annie 19:37, 31 May 2022 (EDT)
I like "The following data is based on questionable pre-publication information and may be incorrect: [parameter]" to cover both scenarios. The other template could expand to something like "The following data is based on information from secondary sources and may be incorrect: [parameter]". The new template names could be something like "Preliminary" and "SecondarySources". Ahasuerus 20:45, 31 May 2022 (EDT)
I am a bit worried about the second one becoming a default if it expands to that - just because you use secondary sources does not mean that the data is unreliable but a new editor won't know that when they see it (or a casual visitor). Tagging everything that does not have primary sources with it will be counterproductive and coming up with usage rules that discourage that is going to be hard (not to mention that if you do not know why the warning is there, it sounds like we do warn because the data is only from secondary sources). That's why I prefer to have the word "questionable" or go for something like "The following data is based on suspicious information from secondary sources and may be incorrect: " which makes it clearer that there is something iffy in this data - not just that the data comes from secondary sources. I'll think a bit more on the wording.
I kinda like "watch" more than Preliminary (easier to remember, less likely for someone to decide that it needs to be there for every book that is added pre-publication just by looking at the list). But I can use it either way. Annie 21:08, 31 May 2022 (EDT)
Upon reflection, I can see how both of the issues with my proposed wording would be problematic. Let's see what John thinks and then we can move this discussion to the Community Portal. Ahasuerus 21:27, 31 May 2022 (EDT)
(dedent) Thanks for responses. I don't have any particular opinions/preferences, but I agree about needing to be careful that whatever is built doesn't get applied to all advance listings. e.g. I'd guess that 99% of page counts listed by Amazon are only approximations compared to a count that strictly follows ISFDB rules, and lots of listed publisher values aren't accurate when you drill down to the imprint level, such as pretty much everything listed as "Cornerstone Digital" has a different PRH UK imprint shown on the title page.
Whilst my original request/proposal was primarily to have something to make it easier for editors to note and be reminded which records need cleaning up, there was a secondary benefit to having something that explicitly flags to a casual user of the site that it is known that the information shown is definitely incorrect. This is clearly the case when a image has text saying "Not final cover", but less so for other fields I think?
Anyway, I'm more than happy for this to be thrown out to a wider audience on Community Portal. ErsatzCulture 12:19, 1 June 2022 (EDT)
OK, let's summarize the consensus so far:
  • We would like to create two templates as a replacement for "WatchDate", which is currently associated with 21 publication records.
  • One template will be for "questionable pre-publication data" and the other one will be for "questionable data from secondary sources".
  • Each template will take a parameter indicating which field's data is questionable.
  • Each templates should expand in a way that supports both the name of a single field and the names of multiple fields, e.g. "The following data is based on questionable pre-publication information and may be incorrect: [parameter]".
  • Template names should contain the word "Watch" in them in order to differentiate records with particularly questionable data from regular records which include data from pre-publication and secondary sources, e.g. "WatchPrePublication".
Is this everything?
Re-reading the list now, I am beginning to wonder if we may want to limit the current discussion to the first proposed template, which I will temporarily dub "WatchPrePublication" for the sake of convenience. A "WatchSecondarySources" template raises additional questions -- like "What exactly counts as a questionable secondary source?" -- and may need a separate discussion. Ahasuerus 14:58, 1 June 2022 (EDT)
I had been wondering the same about the secondary warnings and I agree that we should not try to fry the whole fish -- let's get the pre-publication templates set and then we can brainstorm for post-publication data issues.
We may also want to add that the parameter can be a list of fields, a reasoning or a combination of. Or words to that effect (so you can explain why you think something is suspicious or just list the fields.
Number of pages is often suspicious but tagging all pre-publications for that makes no sense - these will clear from a primary/secondary source. But that is about usage of the pattern and that will come from practice. Annie 15:30, 1 June 2022 (EDT)
I appreciate that the proposed functionality is a superset of WatchDate, but one benefit of the current setup is that the current cleanup report shows the date in question, which is potentially useful for identifying things that should have been published by now (which are worth investigating) vs future pubs (which are probably worth leaving until pub date, or close to that). It strikes me that a more flexible template may be harder to produce a cleanup report for, that isn't either hard to read, or relying on you to click through to the publication in question.
(BTW, I've just cleaned up one of the WatchDate records, and another one should become clear in ~2 weeks, so that'll get them down to 19 :-) ErsatzCulture 15:38, 1 June 2022 (EDT)
Show the parameter as a second column in the report. Then you can search for the word "date" or "cover" if this is what you want to work on only - or read through the strings and see if you can spot something else you may be able to work on. We are not expecting to have thousands of records tagged, do we? If we end up with too many tagged, these books maybe should not have been added before more data was available. :) Annie 15:46, 1 June 2022 (EDT)
We could enhance the cleanup report in 2 ways:
  • Add a column for the template parameter
  • Let the user re-sort the report by:
    • Publication date
    • Parameter name
    • Publication title
It would be similar to the way we currently let our users re-sort Publication Series tables by year/series number, e.g. see this publication series. Ahasuerus 12:40, 2 June 2022 (EDT)

(unindent) To summarize, here is what I think we currently have:

  • Create a new template, "WatchPrePub" as a replacement for "WatchDate", which is currently associated with 20 publication records.
  • The new template will be used for all types of "questionable pre-publication data".
  • The new template will take a parameter indicating which field's (or multiple fields') data is questionable.
  • The new template will expand to "The following data is based on questionable pre-publication information and may be incorrect: [parameter]".
  • The associated cleanup report, which currently displays publication title and publication date, will have a "Parameter" column added. It will also let editors re-sort the table by publication name, publication date or parameter name.

If this looks about right, I can post the proposal on the Community Portal. Ahasuerus 14:58, 3 June 2022 (EDT)

This looks fine to me. ErsatzCulture 16:42, 3 June 2022 (EDT)
Done. Ahasuerus 11:24, 4 June 2022 (EDT)

Python Script Error?

'Show All Titles' for author Cindy O'Quinn results in 'A problem occurred in a Python script.' John Scifibones 14:26, 8 June 2022 (EDT)

Thanks, I'll take a look. Ahasuerus 14:58, 8 June 2022 (EDT)
Fixed. Thanks for reporting the problem. Ahasuerus 15:55, 8 June 2022 (EDT)

Needed Bureaucrat warning?

Perhaps I'm prejudiced, but to me this looks like a personal attack. Despite my attempts to cool things down, it could get out of control. --Username 08:34, 13 June 2022 (EDT)

Thanks, I'll take a look. Ahasuerus 11:07, 13 June 2022 (EDT)
Please see Stonecreek's answer here and his remark here. I do not think he has received the message. Thanks, --Willem 15:19, 15 June 2022 (EDT)
Yes, I saw them. When a request for self-approver privileges is posted after June 30, Stonecreek's submission history, his Wiki responses and other editors' feedback will be used to decide whether to approve it. Ahasuerus 16:42, 15 June 2022 (EDT)

Username

As you can see from last comment here, I've had it with Username. I have only limited time to spend working on ISFDB stuff, and he causes me too much annoyance and stress for me to deal with him anymore. So, I've told him I will not be handling his submissions in the future due to all the crap he heaps upon me and anyone else who tries to work with him. I'll continue working on the other submissions, but (despite the many good things Username does) I find he is more a detriment than a help here. I've bent over backward trying to break through his cantankerous shell, but I'm at a complete loss as to how to handle him. Sorry if I come off as rude here, but I've had it dealing with Username, so I'm choosing to not deal with him in the future. ···日本穣 · 投稿 · Talk to Nihonjoe 16:01, 16 June 2022 (EDT)

I wrote my (very long) reply to him already; as I explained there, he never handled most of my edits, anyway, so him not doing so anymore would hardly affect me. It happened at least once before with another mod last year, and I've made many thousands of edits since, so whatever. --Username 16:54, 16 June 2022 (EDT)
We have had a few similar situations over the years. In extreme cases most moderators stopped working on an editor's submissions and they lingered in the queue for such a long time that the editor lost interest and moved on to other projects. It's unfortunate, but there isn't much we can do about it. After all, we are all human and we are all volunteers. (Except Fixer, of course.) Ahasuerus 17:30, 16 June 2022 (EDT)

Amazon URL warning question

Hi. Can you tell me what's wrong with the URL in this submission? It looks ok to me, and I would ignore the warning and accept it, but I figured maybe I'm missing something? Thanks! --MartyD 14:16, 26 June 2022 (EDT)

The software expects "images/S/amzn-author-media-prod/" images to have a "ssl-images-amazon.com" domain. Let me see if there is an equivalent *.ssl-images-amazon.com URL or if we need to adjust the software check. Ahasuerus 14:30, 26 June 2022 (EDT)
It turns out that images-na.ssl-images-amazon.com had the same image -- fixed. Ahasuerus 14:33, 26 June 2022 (EDT)
Ok, thanks! --MartyD 10:31, 27 June 2022 (EDT)

HTML rendering issue - ea.cgi

As a non-logged in user, the author page - e.g. http://www.isfdb.org/cgi-bin/ea.cgi?206767 - shows the HTML button markup for "Never display translations". It doesn't do this for me as a logged-in user, not sure if that's just because I'm logged in, or if it's because of my personal prefs. I've not looked at the code (yet), but this feels like it might be a reasonably high-priority fix requirement, not so much in terms of overall functional impact, but it'll obviously appear on a lot of pages. ErsatzCulture 07:26, 30 June 2022 (EDT)

Yup, it's an unintended side effect of the latest patch, aka a "bug". Let me see if I can fix it real quick... Ahasuerus 08:22, 30 June 2022 (EDT)
Fixed. Thanks for reporting the issue! Ahasuerus 08:50, 30 June 2022 (EDT)

A much less important rendering issue I accidentally stumbled across are the page counts on a couple of pubs here. (Interestingly, the individual pub pages don't render the break, either as a visible HTML tag or as a break - are they being stripped out on that page perhaps?) One niggle is that one of those pubs is verified, so whilst I don't think there should be any issue with removing the HTML (and personally I'd argue it should never have been there in the first place), in theory I should contact the PVer to see that they are happy for the edit to be made. (In this particular case, it's moot, as the PVer is no longer active.) Depending on how much unwanted HTML is around, is it maybe worth relaxing the "contact PVer before making edits" for cleanup edits like this? I'll happily fix stuff like this as I come across it - and maybe even proactively query my local database to find offending records - but if the workflow has the overhead of checking with the PV(s) in order to do mundane cleanup, then I personally am going to be less inclined to spend time on this sort of thing. ErsatzCulture 07:26, 30 June 2022 (EDT)

Help:How to change verified publications makes a distinction between "significant changes" and "adding data", but it requires notifying primary verifiers in both cases. What it doesn't do is define "significant changes" or describe the process for "insignificant changes" like formatting tweaks. Back in January I proposed clarifying and expanding the language, but we were unable to reach consensus.
Perhaps a less ambitious clarification of the Help language would be easier to adopt. Something like "Insignificant and formatting changes can be made without notifying primary verifiers. What is deemed *insignificant* is left to the reviewing moderator's discretion". Not that it would it affect this particular scenario, of course, since it was a code issue, now fixed (see below.) Ahasuerus 09:10, 30 June 2022 (EDT)
I think the latter problem is code, not data. It's showing up for other titles with multiple pages (e.g. Verne, Wells. ../Doug H 08:02, 30 June 2022 (EDT)
Yup, it's another (related) bug. Looking into it... Ahasuerus 08:50, 30 June 2022 (EDT)
And fixed. Ahasuerus 09:02, 30 June 2022 (EDT)

Daughter(s) of Earth

Please see this pending edit. The note being added seems to suggest that we have the incorrect form of the title for the Merril story. I'm adding this same note on the other verifier's pages. Thanks. --Ron ~ RtraceTalk 09:55, 30 June 2022 (EDT)

My copy doesn't have its dust jacket any more, but I can confirm that it says "Daughters" in the table of contents but "Daughter" on page 198. Ahasuerus 12:30, 30 June 2022 (EDT)

Question re. views columns in titles table

Schema:titles (which I appreciate may be out of date) says:

title_views - The number of times this title has been viewed.
...
title_annualviews - The number of times this title has been viewed in the current calendar year.

However, I've just been looking at a handful of stories from the most recent database backup, and for those stories, the two columns are much closer than I'd expect:

   MariaDB [isfdb]> select title_id, title_title, title_views, title_annualviews from titles where title_id in (41692, 41300, 58836);
   +----------+-------------------------------------------+-------------+-------------------+
   | title_id | title_title                               | title_views | title_annualviews |
   +----------+-------------------------------------------+-------------+-------------------+
   |    41300 | I Have No Mouth, and I Must Scream        |       51937 |             51557 |
   |    41692 | "Repent, Harlequin!" Said the Ticktockman |       82185 |             76523 |
   |    58836 | The Ones Who Walk Away from Omelas        |       36974 |             32744 |
   +----------+-------------------------------------------+-------------+-------------------+
   3 rows in set (0.00 sec)

I would expect title_views to be many times bigger than title_annualviews? I could probably find this out for myself by reinstalling a backup from early January, but is it possible that title_annualviews isn't getting zeroed each year as the schema docs implies? Grepping for references to that column name, I could only find it used in SQLparsing.py, when incrementing for a view, so there doesn't seem to be an "official" annual script/job to reset that column.

Of course, that grep result implies there's nothing that exposes that figure to users of the site either, so if it isn't getting reset annually, no-one will see it :-)

(Spotted whilst looking into exactly how the counts are done, due to this blog post, which makes a minor error about the time period the counts cover.) ErsatzCulture 17:01, 30 June 2022 (EDT)

IIRC, at one point I discovered that we didn't have anything in the software that would actually reset the "annual view" count. We could probably add it to mod/marque.py, which is usually run at the beginning at each calendar year. However, it's a manual process which can be run "on demand" from the Bureaucrat menu, so there would be no guarantee that it doesn't get re-run at some random point. I guess we could add a warning to the menu option. Ahasuerus 17:54, 30 June 2022 (EDT)
Thanks. I don't think this is anything worth prioritizing/doing any time soon, I just thought it was worth mentioning after I'd stumbled across it. ErsatzCulture 10:28, 1 July 2022 (EDT)

Wikipedia template {isfdb name}

Hi. Last decade at English Wikipedia, I added hundreds of biography footer links to ISFDB author pages using Template {isfdb name}. Always I identified the ISFDB author by numeric ID, such as 'id=3845' for Jane Gaskell (and I do so again this spring at a slower rate). Probably it seemed best to start that way, because I was familiar with Wikipedia page moves --or name changes-- and considered them frequent. The alternatives are

  • A. use such as 'id=Jane_Gaskell', which is adequate if it is correctly derived from the ISFDB canonical author name;
  • B. rely on the default, which is to derive a similar value from the Wikipedia page name, which is adequate at "Jane Gaskell" only because and so long as her EN.wiki pagename matches our canonical name.

This spring in merely dozens of visits to biography footers, I have noticed extremely heavy use of the same template with alternative A, 'id=Jane_Gaskell' and so on. I suspect someone's steady effort during the last few years, if not a cooperative project or a robot, that adds biography footer links using template {isfdb name} with alphabetic ID values.

Do you know that the numeric ID is a better choice, as it seems to me? Or that the alpha is a better choice? (If the latter, please explain briefly.) Nowadays I do not add many new footer links from EN.wiki biographies to ISFDB authors; the template is in place, for most appropriate biography pages that I edit. But I would routinely change the ID value from alpha to numeric, or vice versa, if I were sure that one is a better choice. And I would explain the reason to prefer one or the other, if I would meet the right person at Wikipedia or Wikidata. --Pwendt|talk 13:04, 1 July 2022 (EDT)

In the past, linking by name didn't work if the name contained non-English characters. For example, http://www.isfdb.org/cgi-bin/ea.cgi?Stanisław Lem would have failed 2 years ago. I assume that's why Wikipedia editors preferred linking by ID since it was more reliable. However, the software was fixed in June 2021, which made linking by name more reliable.
At this point I would say that linking by name is slightly better. Our internal author IDs can potentially change, although it's fairly unlikely. For that to happen you'd need to either merge the author record with another record, or delete the author and then re-enter it. Linking by name doesn't have the same issue, as unlikely as it is. Hope this helps! Ahasuerus 18:30, 1 July 2022 (EDT)

Multi User Write Access Problem

Could you please take a look at this discussion and let me have your thoughts. Thanks. Teallach 18:43, 3 July 2022 (EDT)

Sure thing! Ahasuerus 21:17, 3 July 2022 (EDT)

Another page with exposed HTML markup

Bottom two items on the cleanup reports page have exposed button markup, although the links for both work fine, so it's only a cosmetic issue. ErsatzCulture 07:54, 4 July 2022 (EDT)

Fixed, thanks! Ahasuerus 12:29, 4 July 2022 (EDT)

TBH, I've never really understand why that pages uses buttons, so if it was up to me, they'd all just become vanilla HTML links to the reports ;-) ErsatzCulture 07:54, 4 July 2022 (EDT)

If I recall correctly, it originally used regular links. However, as the number of cleanup reports grew, it became increasingly difficult to click the right link. Hence the change to the "button" look. Ahasuerus 12:29, 4 July 2022 (EDT)

Crystal Warriors

http://www.isfdb.org/cgi-bin/pl.cgi?37703; I replaced unstable "P" cover with OL cover, which looks the same. --Username 10:38, 4 July 2022 (EDT)

Approved, thanks. (I remember reading this series shortly after it was published and thinking that it was gloriously pulpy. I wonder if it holds up...) Ahasuerus 12:33, 4 July 2022 (EDT)

New cleanup report

One of our very active editors has communicated that they are seeing abbreviations other than the six allowed under special designations in the Page field. He has offered to go in and correct them. A cleanup report to locate them is needed since the Page field is inaccessible with advanced search. I know you're busy, just want to get it on the list. Thanks, John Scifibones 22:24, 4 July 2022 (EDT)

Thanks, I'll take a look. Ahasuerus 22:58, 4 July 2022 (EDT)
It turns out that we already have a cleanup report that looks for invalid page numbers. As of yesterday, it was limited to "del" and invalid Unicode pipe characters. Earlier today I updated the report to look for invalid numbers following the pipe character. Anything before the pipe character is trickier because Help:Screen:NewPub says that "Uncommon page numbers like A-1, B-2, etc." are allowed. I could probably come up with another cleanup report which will look for suspicious values but allow moderators to ignore false positives. Ahasuerus 12:29, 5 July 2022 (EDT)
Looking at the first run, it appears the primary 'offender' is post pipe roman numerals. What Username was looking for is a way to find all the times he has used or seen 'fp' for frontispiece. He would like to correct them. This came up while I was reviewing some of his submissions. I have also seen 'N/A' and a few other oddities. Perhaps the answer is not a cleanup report, but the ability to search the Page field via advanced search. John Scifibones 15:10, 6 July 2022 (EDT)
I see. Yes, "fp" is invalid, so we can safely add it to the current report. I have changed the software; the new data will become available tomorrow morning. I expect around 90 additional publication records. Ahasuerus 18:29, 6 July 2022 (EDT)
Thank you for working on this. I'll let him know. John Scifibones 18:58, 6 July 2022 (EDT)
Sure thing! Ahasuerus 19:12, 6 July 2022 (EDT)
After working with the updated report, I suggest changing the description " 'fp' used instead of 'fep' ". While it is sometimes used instead for 'fep', the most frequent use was for frontispiece title records. Maybe " Nonstandard Special Designations (fp)". This is the title of the bullet point listing the allowed abbreviations in the help section. We can then add others as we discover them. John Scifibones 15:36, 9 July 2022 (EDT)
Makes sense. I have made the change. Thanks for working on the report! Ahasuerus 17:18, 9 July 2022 (EDT)

Same Image

See this submission: The image URL is exactly the same. Based on the pub history, it looks like the editor created a double submission by mistake. It would be nice if the proposed changes comparison recognized that as no change vs. the image already on file warning. Thanks. -- JLaTondre (talk) 09:09, 9 July 2022 (EDT)

Let me make sure that I understand the issue correctly. Normally, re-entering the same image URL (or any other field value) won't have an effect because the submission creation process will discard the new value after determining that it is the same as the current value. Thus this scenario only occurs if the same image URL change is submitted more than once, right? If that is the case, perhaps we still want a yellow warning, just a differently worded one. Something like "Submitted value is the same as the value already on file. This may be a duplicate submission." Does this make sense? Ahasuerus 09:50, 9 July 2022 (EDT)
Okay, looking at it closer I see what happened:
  • At 15:53:22, Rosab618 submitted just the image change ([4] [5])
  • At 15:53:38, Rosab618 submitted just same image change & updated pub notes ([6] [7])
They self-approved the second, but forgot to cancel the first.
I had assumed it was one of those cases where the submission button was clicked twice creating a duplicate entry. This all leads me to three thoughts of items that would be nice:
  1. That the submission page showed the comparison to the current state of the record, not the comparison against the state at the time of submission (for some reason, I always think it is the former, but your description reminds me that I should know it is the later). This would help when people submit the same updates, especially with the long queue times lately.
    Let me make sure that we are on the same page. When a submission is created, only the affected record IDs and any changed values are saved within the XML payload. Unchanged values are not stored in the XML payload. When a submission review page is displayed, the software shows what is stored in the XML payload on the right hand side. The data on the left hand side is retrieved from the database using the record IDs stored in the XML payload. If one or more of the record IDs are no longer stored in the database, the submission is displayed as "non-approvable" and needs to be hard-rejected.
    The only exception to this rule is the "Subject" line of the submission. It is built by the submission creation process based on what was in the database at that time and is stored in the XML payload. Typically, the "Subject" line contains the title of the main record. If the title changes between the time the submission is created and the time it is reviewed, there will be a discrepancy between what's displayed and what's stored in the database.
    Are you suggesting that the software should be changed to build the "Subject" line dynamically? That shouldn't be too hard to do on submission review pages, but keep in mind that the "Subject" line is also used on all "Pending"/"Rejected"/"Approved" pages. If the software had to go to the database to get the current record title, it would slow things down. In addition, it would mess things up for approved/rejected submissions whose main record is no longer on file. Ahasuerus 12:54, 9 July 2022 (EDT)
    On reflection, what I'm suggesting is that the table be created as is currently, but the color coding for changes be based on a comparison to the current values. When the values on both sides are the same, show them both as green vs. the left side as red (similar to what is done with the merge screen). A warning note could also be added that there is no change. -- JLaTondre (talk) 14:40, 9 July 2022 (EDT)
    OK, that makes sense. FR 1514, "Enhance submission review pages to look for unchanged submitted values", has been created. Ahasuerus 17:52, 9 July 2022 (EDT)
  2. That self-approver submissions were identified in the moderator queue (maybe another color?) and possibly skipped by the Next Submission button. I wouldn't have bothered with this one had I realized it was a self-approver and instead done another submission.
    Makes sense. I have created FR 1512, "Next Submission link to skip submissions by self-approvers". Re: color-coding them, I'll need to know which color to use since I am color-blind and can't tell which colors stand out to most people. Ahasuerus 12:32, 9 July 2022 (EDT)
    FR 1512 has been implemented. Ahasuerus 13:19, 12 July 2022 (EDT)
  3. That the rejected submission page had the "View the submission as raw XML" link like the approved submission page does. Very minor, but since I assume it is also very easy I will toss it on the list.
    Re: #3, don't we already have this functionality in place? I am looking at this rejected submission and I see "View the submission as raw XML" above the "Submitted by" line. Ahasuerus 12:28, 9 July 2022 (EDT)
    Ah, it's the difference between the public view and the moderator view. See [8]. Not a biggie, but maybe add a link to get from the moderator view to the public view on approved & rejected submission pages? The unprocessed ones have a "Public View" button on the same line as the approve / reject buttons. Having a way back to the public page also from the approved & rejected moderator views would be useful when wanting to leave a follow-up message on the submitter's talk page with a link to the submission. -- JLaTondre (talk) 14:40, 9 July 2022 (EDT)
    Sounds reasonable. FR 1513, "Link 'moderator view' pages for processed submissions to public view pages", has been created. Ahasuerus 17:23, 9 July 2022 (EDT)
    FR 1513 has been implemented -- see the Moderator noticeboard for details. Ahasuerus 17:30, 11 July 2022 (EDT)
Thanks. -- JLaTondre (talk) 10:31, 9 July 2022 (EDT)
Thank you for making the two changes. Appreciated. -- JLaTondre (talk) 08:51, 13 July 2022 (EDT)
Sure thing. The last requested change will have to wait until I finish the rewrite of the way submissions are displayed. I am currently working on it in order to address Bug 673, "HTML tags in titles", so the timing is fortunate. Ahasuerus 08:59, 13 July 2022 (EDT)
Color-coding of self-approvers' submissions has been implemented -- see Moderator Noticeboard. Still working on rewriting the submission review pages. Ahasuerus 12:57, 22 July 2022 (EDT)

Cleanup report showing Python error/stack trace

http://www.isfdb.org/cgi-bin/edit/cleanup_report.cgi?288 reports "<type 'exceptions.NameError'>: global name 'SQLVersion' is not defined"

A very quick check of the most up-to-date code checkout I have from February doesn't have the offending line - I'm guessing it's a Python 2.7 or similar change? ErsatzCulture 13:34, 9 July 2022 (EDT)

It was a copy-and-paste error introduced while working on making the software support MySQL 8.0, now fixed. Thanks for reporting the problem! Ahasuerus 13:44, 9 July 2022 (EDT)

Peter McLean's Priest of Crowns - Fixer submission doesn't match current Amazon.com data

Hi, I see you've had the dubious honour of doing Fixer approvals the past week or so ;-) ErsatzCulture 16:58, 9 July 2022 (EDT)

Yes, indeed. Annie is currently on vacation. Ahasuerus 18:13, 9 July 2022 (EDT)

I've just been syncing up today's database backup with my local tools, and I was a bit surprised that this UK pub had already been added, albeit only a couple of days ago.

That in itself is no issue, but it seems there are differences between what Fixer submitted (apparently based on data from Amazon July 6th) versus [https://www.amazon.com/exec/obidos/ASIN/1529411343/isfdb-21 what Amazon.com is showing right now, notably the pub date:

  • Fixer submitted 2022-07-19
  • Amazon.com says right now 2022-11-08 (as does B&N)
  • The UK publisher site (and all the UK retailer sites I've scraped) says 2022-08-04 (and has done since early February when I first grabbed any data for that title's pubs). (NB: these are the same pubs/ISBNs; the Jo Fletcher Books imprint seems to be licencing for both US and UK markets for many of their titles.)

Perhaps this is simply the product data being updated at an unfortunate time, but I'm just mentioning it in case there's something weirder afoot. Are you able to trigger Fixer to re-grab the data for this pub from Amazon's API, to see if it now matches what is on the public Amazon.com site, or if it serves you the wrong data? I know Amazon UK acquired a bug about a year ago on the customer wish-lists, where sorting based on price criteria seemed to use prices that might have been several days out of date.

NB: I'll update the data for this pub to what I believe is correct some time in the next few days, but I'll leave it as-is for now to make it easier for you to see what Fixer submitted. ErsatzCulture 16:58, 9 July 2022 (EDT)

Checking Fixer's database, I see that Fixer last queried the Amazon API for this ISBN on 2022-07-06, right before creating the submission. At that time the API returned "2022-07-19" as the expected publication date. Re-querying the Amazon API, I receive "2022-11-08", so it looks like Amazon.com's expected date has changed in the last 72 hours.
Re: Amazon UK, it's not uncommon for cancellation/delay notifications -- and especially transoceanic notifications -- to be lost in the shuffle. Moreover, experience suggests that a publication that has been delayed one time has much higher than average chances of being delayed again.
Given the above, I don't think there is a whole lot that we can do except add "WatchPrePub" warnings to Notes whenever we encounter something strange. When I come across really shoddy pre-publication records I tend to cancel the submission, "unsubmit" it in Fixer's internal database and change Fixer's "expected publication date" value to a month after the supposed publication date. That way Fixer will try to resubmit when the ISBN is already available for purchase -- or so we hope. Ahasuerus 18:13, 9 July 2022 (EDT)
Thanks for taking the time to look into this one. One of my long wanted to-do's is to add functionality to my tools for pubs that are in the database, to report on any inconsistencies compared to the data I've scraped. Unfortunately those tools are a big heap of spaghetti code, built on a bunch of incorrect assumptions about how I thought at the time how the ISFDB database worked, and naivity about how badly vendors and publishers can mangle the data on their sites, so I really need to do a major rewrite before embarking on anything more than the relatively simple task of writing scrapers for new sites. One day... ErsatzCulture 08:18, 10 July 2022 (EDT)
Sounds painfully familiar :-\ Ahasuerus 10:03, 10 July 2022 (EDT)

Invalid price check - decimal halfpenny

Just looking through the cleanup reports for any mess I might have created, and in the invalid price check report, I see there are a couple of false positives for the pair of issues of the Sfinx fanzine from the early seventies. These both have prices of "£0.075", which matches the cover images showing a price of "7 1/2p". Post decimalization, there was a halfpenny, but 1970s inflation did it in. In theory it might be worth extending the logic/regex for that cleanup report to cover these prices, but I suspect there won't be any other pubs entered with halfpenny prices, so perhaps it's just easier for a mod to mark those cleanup errors as to be ignored? ErsatzCulture 08:44, 10 July 2022 (EDT)

Thanks for the reminder! I came across these types of exceptions early on, but decided to wait for the cleanup report to be whittled down to a manageable subset of exceptions before deciding whether to add the ability to "ignore" records. And then I forgot about it...
Let me see if the remaining exceptions can be accounted for programmatically. Ahasuerus 10:24, 10 July 2022 (EDT)
You may remember this conversation, but I will call you attention to it to be on the safe side as $0.125 is also valid. -- JLaTondre (talk) 12:13, 10 July 2022 (EDT)
Yes, indeed! I have updated the software to skip "$.0__5" and "£0.__5" prices.
Of the remaining 5 records, "£1 4s" and "£5 5/-" presumably need to be converted to "24/-" and "105/-" respectively as per Template:PublicationFields:Price. On the other hand, prices like "-/4 1/2" are presumably legal and need to be skipped by the cleanup report. I'll see what I can do. Ahasuerus 13:11, 10 July 2022 (EDT)

Edit XML confusion

I'm slightly wary of mentioning this, given my confusion over the edit bug yesterday, but I've just had a notification about a change to a pub I PV that I'm a bit perplexed by.

As far as I can tell, the only material difference in #5362000 over what I originally submitted is adding the cover credit, but the XML view seems to indicate (nearly?) every field has changed. (NB There was an edit between those to fix a page number, and that does show as I'd expect).

So, the actual data in the database seems to be fine, but the XML view seems to be misleading about what actually got changed. Any ideas? I wonder if it's some sort of ASCII vs Unicode thing that isn't visible when the edit is rendered in a browser? (Perhaps it might be helpful to be able to download the XML as a file, so that you can run tools like diff or od/hexdump on?

Hopefully this is just something simple/obvious that I'm being clueless about... ErsatzCulture 10:48, 14 July 2022 (EDT)

UPDATE: I just looked at a different edit by a different editor, and I'm wondering if this is merely a weirdness of ImportExport including a bunch of fields in the XML that aren't actually relevant to the edit? If so, don't waste any time on looking into this. 10:56, 14 July 2022 (EDT)
it does seem to be the issue here. Unfortunately, ImportExport was originally coded to piggy-back on "NewPub" submissions, which is why it shares a significant amount of code and data structures with the latter -- note the "NewPub" tag in the raw XML view of 5362000. It causes all kinds of headaches as the same code needs to handle two very different types of submissions. I would have separated them a long time ago if not for the fact that we would still need to be able to display pre-change submissions. I am hoping that the current rewrite of the submission display logic will make it easier to get rid of some of this ugliness... Ahasuerus 11:58, 14 July 2022 (EDT)

Tom Watson

Thanks for catching and fixing the errors I created on this author's record. In my defence, the "other" Watson isn't an implausible person to have written an SF novel: "He was, by his own account, a nerdy geek with long hair, devouring Robert A Heinlein’s science fiction..." ;-) ErsatzCulture 09:35, 18 July 2022 (EDT)

The credit goes to John Clute who discovered that they are two different people while working on the new SFE article :-) Ahasuerus 11:14, 18 July 2022 (EDT)

Sunset Warrior

http://www.isfdb.org/cgi-bin/pl.cgi?244493; I replaced the unstable Amazon image with Bookscans.com image. --Username 09:13, 20 July 2022 (EDT)

Approved, thanks. Ahasuerus 09:17, 20 July 2022 (EDT)

Masters of Space

http://www.isfdb.org/cgi-bin/pl.cgi?273905; I replaced unstable Amazon cover with Bookscans.com's cover; the LCCN on the OL page gets nothing (and OL says Jove even though the Archive copy is Orbit) and neither does searching LCCN site for this title. Maybe your PV copy has a LCCN on the copyright page. --Username 10:56, 20 July 2022 (EDT)

I have approved the cover change. I have also checked the copyright page of my copy and added the LCCN number and other details. Thanks for the heads-up. Ahasuerus 12:14, 20 July 2022 (EDT)
So the LCCN listed on OL is correct but it's not on the LCCN site? That's odd; I wonder why that is. --Username 12:35, 20 July 2022 (EDT)
It's been a long time since I dealt with the Library of Congress, but back in the day they treated mass market paperbacks as "red-headed stepchildren". They stored them in boxes (crates?) at a remote facility and there was no easy way to access them. If memory serves, their explanation was that they needed to rebind all received books and there was no cost-effective way of rebinding mass market paperbacks. Perhaps it also meant that they didn't catalog them as thoroughly as they cataloged other books. Ahasuerus 13:12, 20 July 2022 (EDT)
Also, a minor point I see in a lot of records here, copyright symbol, ©, was entered as @ symbol. --Username 12:35, 20 July 2022 (EDT)
I suspect that it goes back to the typewriter era. Back then many typewriters didn't have the copyright symbol and "@" was used instead. That's how I got into the habit of using it. Ahasuerus 13:12, 20 July 2022 (EDT)

Possible Faulty Logic

There might be faulty logic in the Duplicate SHORTFICTION in Magazines/Fanzines cleanup report. It is reporting a publication that has two titles with the same name 'Summer Break' but by different authors. The duplicate finder on the publication level correctly reports no duplicates. John Scifibones 16:55, 21 July 2022 (EDT)

This report is basically trying to find (among other things) an interior art being coded as a story by mistake (it will look exactly like that - same title, different authors). In case when the story is a legitimate story with the same title as another one in the issue (as is here), we ignore (that's why this report have ignore). :) Annie 17:12, 21 July 2022 (EDT)
Makes sense, John Scifibones 17:22, 21 July 2022 (EDT)

Reddit admin blockage discussion

FYI in case you hadn't seen it... —The preceding unsigned comment was added by ErsatzCulture (talkcontribs) .

Thanks for letting me know. I first ran into this problem a couple of years ago and was unable to have our domain removed from the spam list. Hopefully this iteration will be more successful. Ahasuerus 14:31, 25 July 2022 (EDT)

Submission Warnings for Awards?

Please see this submission. In adding the additional contents, the submitter edited the two existing records to be different content. I have seen this happen before with users who think the content must be listed in order of appearance on the edit screen (i.e. are not aware of how sorting works). Generally not an issue, but in this case, the two existing records have associated awards. If this edit is accepted, the awards would now be associated to the wrong records. Is editing a record with an award something that should be flagged in the pub submission screen? Maybe it doesn't happen enough to worry about... I'm going to reject this submission and redo it. -- JLaTondre (talk) 09:37, 26 July 2022 (EDT)

Wouldn't that also mess up any reviews associated with the changed title records? Ahasuerus 11:16, 26 July 2022 (EDT)