User talk:Ahasuerus/Archive/2018

See User talk:Ahasuerus/Archive for discussions prior to 2019.

PLEASE NOTE: If you're writing to inform me that you've either added a COVER IMAGE or NOTES to any of my VERIFIED PUBS, please follow THIS LINK and add it to the bottom of the list. A link to the pub record would be appreciated. Once the pub has been reviewed, I'll remove your note from the list. Thanks!

Show link to imported titles in submission page

Hi. When a submission imports existing titles from other records the ids of the imported titles are known when accessing the submission's page. It'd be great if these titles could be rendered as a link in the "Title" column. Jens Hitspacebar 15:35, 12 January 2018 (EST)

I think it's a good idea. Come to think of it, we may want to link all "auto-merged" titles in Clone Pub, Export Content, etc. Ahasuerus 17:25, 12 January 2018 (EST)

That'd be even better. It would make moderating imports and similar submissions easier because you can easily navigate to the existing titles to see if the submission is ok. Jens Hitspacebar 17:53, 12 January 2018 (EST)

OK, FR 1121 has been created. Ahasuerus 18:05, 12 January 2018 (EST)

ISFDB-SFE3 Author Mismatches

Is this report so we can add the links to the author pages, and then click the "Click Once Resolved" link? If not, what is it for? None of the entries have an author listed. ···日本穣 · 投稿 · Talk to Nihonjoe 19:31, 16 January 2018 (EST)

The data for this report was originally generated by scanning SFE3's list of authors. The software then created a subset of SFE3 authors who didn't have an ISFDB entry. For example, SFE3 has an entry for Rudolf Brunngraber, the author of the novel Radium: Roman eines Elements (1936), but we don't have a record for him. Some SFE3 authors may not be eligible on our side, e.g. if their only SF works were in the comics field, hence the "Resolve" links.

The biggest problem with this report is that the data was generated a few years ago and is out of sync with the SFE3 author list. Once the current data has been sorted out, we'll need to add a button to perform "on demand" list reconciliation. Ahasuerus 19:51, 16 January 2018 (EST)

That makes sense. Is there a way to add an explanation of what the report is at the top of the report? For ones like this one, where it's not immediately obvious, it may prove helpful. ···日本穣 · 投稿 · Talk to Nihonjoe 20:14, 16 January 2018 (EST)

Done! Ahasuerus 21:21, 16 January 2018 (EST)

Duplicate Catalog Id

With this submission, there is already a D-537 catalog id in the database. It would be really nice if (when there is no ISBN), the software warned of duplicate catalog ids the way it does with duplicate ISBNs. Thanks. -- JLaTondre (talk) 08:47, 17 January 2018 (EST)

Good point, FR 1134 has been created. However, I am not sure the presence of an ISBN should make a difference. Ahasuerus 10:32, 17 January 2018 (EST)

It would have when they both used the same field. Now that they are separate, it shouldn't. ···日本穣 · 投稿 · Talk to Nihonjoe 13:12, 17 January 2018 (EST)

Done -- see the Community Portal announcement. Ahasuerus 19:51, 30 January 2018 (EST)

The Glory Game by Keith Laumer

Hi. You PV'd this publication in 2007: http://www.isfdb.org/cgi-bin/pl.cgi?189513. It's strange that, unlike the listings for the January 1973 first edition (http://www.isfdb.org/cgi-bin/pl.cgi?419327) and the later printings, the record you PV'd does not cite a gutter code, nor does it note "First Edition" on the title page, yet includes the note, "This may be the first edition". I don't think that's correct. Suggest you check for a gutter code on page 186, then move your PV to a record for that printing. I'll also post this note for the other PV of the record you verified, Biomassbob. Thanks. Markwood 19:12, 19 January 2018 (EST)

Thanks for the heads-up! Unfortunately, I am not in a position to check this book at the moment -- most of my hardcover Laumers are in boxes. My copy is apparently a later reprint since the pub record says that, unlike the true first edition, it doesn't say "First Edition" on the copyright page. For now, I have updated the record to indicate that it's not the first edition and that the publication date is currently unknown. Ahasuerus 21:06, 19 January 2018 (EST)

Display issue

Greetings in the New Year! Belated but ... I am currently working on an in-depth 'assessment' of Ace books and the varieties therein that I like to refer to as "The Mess". While the current publisher pages are ideal for printing out lists of Ace editions [dated or not] there is one major sidebar to that. Perry Rhodan [US] is listed as a magazine and I can't find any way to display/print the series with ISBNs/dates etc. as is available through the current Ace publisher pages. If there is I'd love to know how. If there isn't, is there a way to temporarily change the series to a form that can be printed with the relevant data? [I'd change it back right away, of course, this isn't about the 'rightness' of the series being a magazine]. Any help would be appreciated!! --~ Bill, Bluesman 20:31, 24 January 2018 (EST)

And a happy belated New Year to you too! If I understand the goal correctly, you'd like to generate a list of every Ace-published Perry Rhodan paperback with the stipulation that the list should use the standard publication table format, right? If so, then the following Advanced Publication Search may be helpful: Publication Type is exactly MAGAZINE, Author's Name contains Ackerman, Sort Results by Date. It takes a few seconds to compile, but it seems to include all eligible pubs. Is it sufficient for your purposes? Ahasuerus 20:56, 24 January 2018 (EST)

Most appreciated! It seemed that no matter what I chose in Advanced Search just couldn't come up with "Magazine" to start with. But since you did it I knew it just had to be there [unless you were hiding something ... less said the better ...]. Sub-menus drive me crazy! Unless you know where to start you can't get there from here [there's more than a few jokes about that line]. All printed, back to the abyss. Cheers! --~ Bill, Bluesman 21:35, 24 January 2018 (EST)

I wonder if you may have tried Advanced Title Search instead of Advanced Publication Search? Either that or you forgot to sacrifice a goat to Cthulhu this morning, but that's too scary to contemplate. Cheers! Ahasuerus 21:50, 24 January 2018 (EST)

It takes goats!!¿¿!! The farm next door has a herd ................ :-))))) --~ Bill, Bluesman 21:53, 24 January 2018 (EST)

What's the best way to submit corrections for author names?

On January 7, I posted to the moderator noticeboard pointing out that the author name "Jodi Renee Lester" should be corrected to "Jodi Renée Lester." On January 20, I posted again, with a reminder. The change has STILL not been made. Such neglect is quite usual: numerous author name corrections that I've requested have been made only after a reminder, or never. Clearly, the moderator noticeboard isn't the right place to post them. But where should I do it then? --Vasha 16:02, 31 January 2018 (EST)

I have changed "Renee" to "Renée", but I am afraid I can't think of a better place to post these types of requests :-( Between Fixer and software changes, I haven't been following the Moderator Noticeboard closely lately, so I am somewhat out of the loop. As a general rule, author name changes can take some time if there are verified publications involved, but this was a straightforward case. Ahasuerus 16:11, 31 January 2018 (EST)

Thanks! Two more straightforward fixes: "Mandem" to "MANDEM" (verified only by me); "Julián Diez" to "Julián Díez" (no verified publications). BTW, if there are verified publications I do ask the verifier how the name is spelled in the pub before I post to the moderator noticeboard (although I don't always get an answer, of course). --Vasha 16:15, 31 January 2018 (EST)

OK, "Mandem" has been changed. Ordinarily, we set up house names and collaborative credits like Ilona Andrews as pseudonyms because individual contributors can also create works on their own. However, in this case I am not sure if we know whether all three members of the "MANDEM" trio are responsible for all of the titles that we have on file. Would you happen to know?

It turns out that the earlier works were by only two of them. OK, I will work on changing that, then. --Vasha 17:09, 31 January 2018 (EST)

As far as "Julián Díez" goes, his review of "Los Tejedores de Cabellos" was published in a verified publication. The rule is that we enter reviewer/interviewer names as specified in the pub and then create a variant if needed. We use canonical names for reviewees/interviewees because the software doesn't support variants for revieews/interviewees. Ahasuerus 16:48, 31 January 2018 (EST)

Oh, sorry, I don't know how I overlooked that verification (I've left the verifier a note now). But if I am going to be entering publications for the Ignotus awards, there will be a ton of magazine issues and anthologies edited by Díez (with the name diacritic-ed). --Vasha 16:54, 31 January 2018 (EST)

Award years for Ignotus

Confusing! Here's what happened, turns out. The 1992 awards were indeed for work published in 1992 (so, that cover art). BUT there were no awards in 1993. And when they reorganized in 1994, they started giving them for work in the previous calendar year!

In sum: 1991 pub.=1991 award; 1992 pub=1992 award; 1993 pub=1994 award; 1994 pub=1995 award; etc. --Vasha 19:27, 31 January 2018 (EST)

Aha! Now it all makes sense. I have added a note to the award type description -- does it look OK? Ahasuerus 19:38, 31 January 2018 (EST)

Yep! --Vasha 19:41, 31 January 2018 (EST)

Translation data at La Tercera Fondación

I noticed that La Tercera Fondación (who list much the same amount of data about the works they catalog as we do, and also have verifiers) has solved the problem of how to credit translators: they have a title type Translation, with the translator as author. With the way they display data, it isn't confusing (or not very). See here for a publication record containing translated stories, here for a translation title record, and here for a master title record ("ficha") of a work that has translations. I wonder if some version of that would work for us? Although I find the Tercera Fundación site hard to read, the basic idea of a translation title type seems like a good one. You've no doubt considered it already! --Vasha 14:38, 1 February 2018 (EST)

Yes, their solution is fairly close to what we discussed during the last brainstorming session. We really need a new TRANSLATION title type -- if nothing else in order to support awards give to translations. Our software challenges will likely be twofold: support for translator pseudonyms (including house names, joint pseudonyms, etc) and unified support for other roles like "single author collection editor".

That said, the way they have their data organized is not quite what I had in mind. I will have to think about it. Thanks for pointing me in that direction! Ahasuerus 17:07, 1 February 2018 (EST)

Spanish books with prices in dollars

I found a bunch of unverified books which, although published in Spain, had prices listed in dollars (all sorts of odd amounts). Since there were no sources named for the data, I simply deleted the price. (Presumably it happened because people entered the data, including the price, from Amazon.com, but shouldn't moderators have noticed that?) Could you generate a list of books which have a language of Spanish and a price in US dollars to find more of these? Thanks. --Vasha 23:03, 1 February 2018 (EST)

On second thought, never mind-- I'm busy with other stuff and this is relatively minor. Sounds like the kind of thing that might get a regular cleanup.report though. --Vasha 09:22, 2 February 2018 (EST)

The challenge with our Spanish language pubs is that some of them were actually published in the US. Also, some older pubs were created by robots before the moderator review process was put into place.

As near as I can tell, the easiest way to find these problematic pubs is to run Advanced Publication Search using the following search criteria:

• ISBN starts with 8 (Spain is 84, Brazil is 85, etc) or 6 (Mexico is 607, Peru is 621) or 9 (Argentina is 950, Chile is 956, Colombia is 958, etc) or 9788/9786/9789
• Price starts with $

It's not perfect since some non-US currency signs start with "$", but I think it's as close as we can get. Ahasuerus 11:15, 2 February 2018 (EST)

Invalid Record URLs in Notes

In this report, what is the issue with the two Title Notes? The links in their respective notes field work. Is the report having false positive because of the links to awards? Thanks. -- JLaTondre (talk) 20:59, 6 February 2018 (EST)

This cleanup report recognizes a limited set of ISFDB links: title, pl, ea, publisher, etc. It recognizes award_details and award_category, but not awardtype. I'll try to fix it tomorrow. Ahasuerus 23:12, 6 February 2018 (EST)

Fixed! Ahasuerus 14:28, 7 February 2018 (EST)

Editing 'anomaly'?

[This] publication record was on my list of Changed Primary Verifications this morning. Dirk has corrected a typo. I attempted to move the OCLC # out of the notes but get a warning message re: 'When editing publications, the Regular Titles subsection of the Content section must contain one title whose type matches the publication type' and the edit won't go through. I backed out of the page then retried a submission but made NO changes and got the same message. The Title type is Novel, the only content is Novel. Then tried it with the other edition of the same title with the same result. Bug/gremlin?? At the moment no edits seem possible. --~ Bill, Bluesman 12:19, 7 February 2018 (EST)

Oops! It looks like the last patch introduced a new bug. Investigating... Ahasuerus 12:40, 7 February 2018 (EST)

The immediate bug has been fixed. Some types of pop-up validation are currently unavailable, but it shouldn't prevent editors from creating submissions. Working on it... Ahasuerus 12:55, 7 February 2018 (EST)

OK, the pop-up validation should be fixed now. Sorry about that! Ahasuerus 13:53, 7 February 2018 (EST)

All seems 'normal' ... famous last words ... Thanks! --~ Bill, Bluesman 13:56, 7 February 2018 (EST)

(unindent) Another patch? The pop-up warnings are at it again. "No artists were entered, but other cover data was specified. At least one artist must be entered. See Help for details." --~ Bill, Bluesman 15:32, 9 February 2018 (EST)

OK, the F-5 whammy seems to have worked, for now. ;-))) --~ Bill, Bluesman 15:40, 9 February 2018 (EST)

Hopefully it was the last behind the scenes patch. I am starting to work on something that should be actually visible for a change! Ahasuerus 16:00, 9 February 2018 (EST)

You mean you're taking off the mask??¿¿?? Will my keyboard go up in smoke? --~ Bill, Bluesman 16:02, 9 February 2018 (EST)

Have no fear! Any collateral damage should be negligible, well under a gigaton. Ahasuerus 16:13, 9 February 2018 (EST)

Skyworlds

Hi! Curious about the title for your verified Skyworlds v1 #1 - you've left out the the date in the title, which you've given in the date field. It makes the grid look a little wonky. Was that on purpose? Thanks, Doug / Vornoff 19:49, 9 February 2018 (EST)

Also, on the cover it looks like the title is "Sky Worlds" (with a space). It's referred to as both on websites. Is it one word on the title page? Doug / Vornoff 19:55, 9 February 2018 (EST)

Let me see if I can find my copy... Ahasuerus 11:05, 10 February 2018 (EST)

Unfortunately, I am unable to get to this issue at the moment. A couple of years ago my house sustained a moderate amount of damage, which resulted in thousands of books getting displaced. Most of them are still boxed or otherwise unavailable :-( Ahasuerus 17:21, 10 February 2018 (EST)

That’s not fun. Sounds like a Project is in order. I have a lot of my digest sf stacked in big cardboard boxes and if I have to get at one on the bottom - yikes! Doug / Vornoff 20:01, 10 February 2018 (EST)

Linking to the Internet Archive

Have you had any interaction with the Internet Archive about hot-linking? I was working on a clean-up report and came across The Twilight Land, for which I found a nice page on LibriVox. There they have a download link for the cover image, which is hosted on archive.org. I was wondering about linking to it, rather than downloading it. I looked at their FAQs, and they seem eager to have references/attribution. But the only explicit mention I see of linking is in the Wayback Machine section, where they do grant permission. It's not clear to me if that covers everything or just web.archive.org. What do you think? --MartyD 10:56, 10 February 2018 (EST)

To the best of my knowledge, this question hasn't come up before. I am not sure how to interpret their FAQs either -- perhaps we could ask them directly?

Another thing to consider is that the data that they host is not owned by them, so it's more likely to be taken down due to copyright issues. I don't know how many images may be affected, but it's something to keep in mind. Ahasuerus 11:32, 10 February 2018 (EST)

I'll send a request and see what they say. Thanks. --MartyD 11:49, 10 February 2018 (EST)

The Unholy City

Hi. You are one of two who verified the 1968 Pyramid edition P243345. If I understand correctly, [1] the publication type should be OMNIBUS if the title story is indeed a novel; [2] the novella title should be "The Magician out of Manchuria" rather than "Out of" (we now have "Out of" and "Out Of"). --Pwendt|talk 16:45, 21 February 2018 (EST)

I agree re: [2]. Re: [1], Help is ambiguous on the subject. Here is what is says:

• OMNIBUS. A publication may be classified as an omnibus if it contains multiple works that have previously been published independently, and at least one of them is a NOVEL, ANTHOLOGY, COLLECTION, or NONFICTION. However, generally this category should not be used unless the other categories do not seem appropriate. For example, if a publication contains stories that have previously been published independently in pamphlet form, this should be classified as an anthology. A collection such as Robert Heinlein's "The Past Through Tomorrow" should be categorized as a collection, although one of the works is a novel. "Omnibus" is appropriate for such publications as the Science Fiction Book Club's collections of three independent novels by different authors under one set of covers; or for a single-volume edition of all the Amber novels by Roger Zelazny. If none of the contents have been published before, the inclination should be to classify the publication as an anthology, rather than an omnibus, but this does not have to be an absolute rule. The distinction between "Omnibus" and the other types is somewhat subjective and may require discussion and consensus on the publication biblio wiki page.

(Note the bolded sentence.) There have been multiple attempts to reach consensus re: what should and shouldn't be entered as an omnibus, but we have been unsuccessful so far. If you think that you can come up with a definition that would lead to consensus, please post it on the Rules and Standards page. Hopefully we will slay this beast one of these days! :-) Ahasuerus 17:27, 21 February 2018 (EST)

"Black author" tag

Hello, I'm voicing here my concern about this tag. Here (in France) we're quite touchy about questions of privacy and of data gathering. In a nutshell, to enter and/or to store and/or to give public access to data about the racial characteristics of an individual (or his/her sexual orientation or political opinion) is simply illegal and may be severely punished. I know we're not a french-law-abiding outfit but, to be frank, I'm quite ashamed to be associated with such an endeavour. Why not add such tags as "Muslim", "Leftist", "Gay", "Have cancer", "Of short heigth", "Redhead" etc... (I spare you the Godwin point "To be eliminated") It is the same for Afro-american and even the fact of passing the tag to "private" is not enough (for me and the french law, perhaps the european one), it's the simple constitution of a database with names that is illegal without it being declared. I was already quite worried about our forays into gender typification, but this is worse. Hauck 04:20, 23 February 2018 (EST)

The same is valid for 'African author' and likely some others. In addition to the things Hervé wrote, these labels totally ignore that tags are for titles, and not for authors: it may be okay to have 'Black protagonist', but no thing in this vein to stigmatize authors. Stonecreek 05:08, 23 February 2018 (EST)

I hear your concerns. I started that project last year because of the #BlackSpecFic report. People there were tracking where black authors got published and I wanted to help them. (I was thinking of contributing to Wole Talabi's list of African spec fix also but I never did much with that.) So you think it is not appropriate to do that tracking in a public manner? --Vasha 08:18, 23 February 2018 (EST)

I think it's totally contra-productive. It really smears all relevant information on thematical issues for an author like Samuel R. Delany to have 153 times characterized him as 'Black Author' (who would have thought that?). It has next to no thematic relevance, and I think this is the case for most authors characterized so or in a similar way. Who wants his works issued by color of his skin? Stonecreek 08:39, 23 February 2018 (EST)

The idea is not to tag the author but rather to get an idea of where and when works by black authors have been published--would it sound better to you to change that tag to "work by Black author"? Also, I am in touch with a couple of the people involved in #BlackSpecFic and if you like I could ask them whether they think it is useful or appropriate to use the database in this way.

I would also like to ask Darrah because of her experience trying to use the database to study women Writers. And I know she's given a lot of thought to privacy concerns. Unfortunately she's not around now. --Vasha 08:46, 23 February 2018 (EST)

No, this is a meaningless tag for our purposes. I really think it has to be deleted, or, if that's not possible, to mark it as 'Private'. You really seem to do some things without thinking them through: that's called actionism. A title tag has not to be misused as an author tag: and these examples seem to be a kind of putting authors into a drawing-box, and nobody has asked them if they want to be put in there. Stonecreek 09:03, 23 February 2018 (EST)

2016 #BlackSpecFic report. See surrounding discussion for why it is unfortunately necessary to talk about writers' race. It is not at all irrelevant to their chances of being published; it is the publishing industry that puts people into boxes. --Vasha 09:17, 23 February 2018 (EST)

We're not here to record the plight of black writers. Such tags attached to persons are simply unacceptable and may even be libelous. As for Darrah (who's a he), this page is IMHO already borderline. Both of you can make all the lists of black or feminine or transgenre or martian writers you want but you simply can NOT use the ISFDB which is a public space. Hauck 10:35, 23 February 2018 (EST)

This conclusion has been proven false, if not dangerous. Opening trenches between black / white or Jewish / Arian or any other thought-of dividing line between people has been never a tremendously good idea. You need to talk about the issues & sources of racism, and that what's literature for (foremost I think). But this can only be adressed in works, not by hammering differences into heads by showing this is the most important thing for a given author.

Also, you seem to flea the basic fault of the misuse of tags. Stonecreek 10:20, 23 February 2018 (EST)

(unindent) It looks like there are a number of issues here, including:

1. The legality of certain tags
2. The ISFDB software design which currently lets users add tags without moderator approval/oversight (potential accuracy issues)
3. The ISFDB policy which determines which tags are made "Private" by moderators
4. The feasibility of using title-level tags to enter author-specific data ("female author", "indie author", "Finnish author", etc)

As previously discussed, the laws that we have to abide by are the laws of the jurisdiction where the ISFDB server is currently located. (It's the same for projects like Project Gutenberg whose servers host different works depending on the country where each server is located.) There may be libel laws that we have to abide by, e.g. "written by a thief and a murderer" could be an issue, but at this time our data entry policies are not legally constrained otherwise.

The rest of the raised issues are related to the way the software is currently written as well as to our data entry/moderation policies, so we may want to move the discussion to the Community Portal. Ahasuerus 10:41, 23 February 2018 (EST)

Author update: Julián Díez

Hi, could you please correct Julián Diez to "Julián Díez"? There's only one publication on that page that wasn't added by me, and that one's verified by an inactive user. --Vasha 17:32, 2 March 2018 (EST)

Sure thing. The spelling of the canonical name has been changed and the verifier has been notified. Ahasuerus 17:39, 2 March 2018 (EST)

Plataforma Neo books

Here's a list of the ISBNs of the few YA fantasy and YA magical realism books that Plataforma Neo offers. If it's too much trouble automatically importing books in languages other than English, ignore this. --Vasha 21:03, 6 March 2018 (EST)

978-84-15577-52-2 978-84-15750-23-9 978-84-15750-24-6 978-84-15750-71-0 978-84-15880-43-1 978-84-15880-74-5 978-84-15880-94-3 978-84-16096-64-0 978-84-16256-32-7 978-84-16256-42-6 978-84-16429-37-0 978-84-16429-53-0 978-84-16429-70-7 978-84-16620-43-2 978-84-16620-53-1 978-84-16620-98-2 978-84-16820-16-0 978-84-16820-86-3 978-84-17002-26-8 978-84-17114-20-6

I am afraid Fixer is currently monolingual :-( Ahasuerus 21:29, 6 March 2018 (EST)

Two author updates

Carlos Ortin should be Carlos Ortín and Andre Brink should be André Brink. Thanks! --Vasha 12:37, 13 March 2018 (EDT)

Done! Ahasuerus 12:40, 13 March 2018 (EDT)

Thank you --Vasha 12:56, 13 March 2018 (EDT)

My Pending Edits

Good morning (kinda... let's call it morning) :)

I thought that "My Pending Edits" is for the edits I had submitted. Why do I see a "The current number of pending edits by all editors (not held by a moderator) is 0." line at the top of it? I do not think I had ever seen it before although I may have just missed it :) Thanks! Annie 13:53, 13 March 2018 (EDT)

The count of all pending edits was added 6 weeks ago -- see FR 1125. The idea was that if editors knew the number of outstanding submissions, they would be better positioned to estimate how long the moderator review process would take. Or at least that was the theory :-) Ahasuerus 19:20, 13 March 2018 (EDT)

Yeah, I blinked and missed it and as I rarely open the page, I did not see it until now. I did not even think that it is not a moderator function but is for the editors. Thanks! Annie 19:33, 13 March 2018 (EDT)

Double PV from the same user at the same time

Good evening,

Can you look at this. How did someone manage to PV the same book twice at the same second? I suspect something in some old code but if it can be cleaned (in case some other code relies on one PV per user per book) :) Annie 18:23, 14 March 2018 (EDT)

Let me take a look...

Well, the bad news is that we have 88 pubs with duplicate primary verifiers. The good news is that:

• it doesn't seem to affect the rest of the software
• it's fairly easy to fix
• all but one duplicates were created prior to 2017-04 when the verification system was last revamped

The only exception is this pub which was transient-verified on 2017-06-25. I suspect a Web server hiccup; we used to see them relatively often. I'll see what I can do to clean up the mess -- thanks for identifying the issue! Ahasuerus 18:57, 14 March 2018 (EDT)

Why do I hear sinister laughter out of nowhere when I see/hear a developer saying "it doesn't seem to affect the rest of the software"? :) Annie 19:20, 14 March 2018 (EDT)

What could possibly go wrong?! :) Ahasuerus 19:30, 14 March 2018 (EDT)

Thanks for looking into it. I almost asked Don if he likes this book so much that he has two identical copies before my mind reminded me that he cannot do that on purpose - so I rerouted the question to you instead. :) Annie 19:20, 14 March 2018 (EDT)

After restoring an old backup file and running a few queries I think I know what's going on. The old design had separate verification slots for "Primary1" through "Primary5" as well as "Transient". It allowed a single verifier to (inadvertently) claim multiple primary verification slots. When we converted to the new system, we ended up with the previously mentioned 88 duplicates. Now to fix them... Ahasuerus 18:34, 15 March 2018 (EDT)

This makes sense - thanks for tracking it down! Annie 18:55, 15 March 2018 (EDT)

Fixed! Ahasuerus 12:56, 17 March 2018 (EDT)

Author corrections

Two more author name corrections, a bit trickier. 1. Azorin is Azorín in Spanish (including the publication I just added); I have no way of checking how the name is printed in the English and Romanian publications. 2. Angel Torres Quesada is properly Ángel Torres Quesada. It's true that a number of book covers show the name without an accent over the A, but that's because accents on capital letters are sometimes omitted for stylistic reasons. --Vasha 12:23, 16 March 2018 (EDT)

Unfortunately, I am in the middle of a number of things at the moment, so I'll have to copy this request to the Moderator Noticeboard. Ahasuerus 12:54, 16 March 2018 (EDT)

No need to do so, I'll take over. Stonecreek 13:00, 16 March 2018 (EDT)

Thanks! --Vasha 13:04, 16 March 2018 (EDT)

Language to add: Asturian

Adolfo Camilo Díaz has written a number of speculative works in Asturian. --Vasha 22:49, 16 March 2018 (EDT)

The Asturian language recognized by ISO 639-2, the standard that we use, so adding it shouldn't present any problems. The only question is whether we should call it "Asturian", "Bable" (as per the linked document) or perhaps "Asturian/Babel". We may want to post it on the Community Portal and see what our Romance experts think. Ahasuerus 00:10, 17 March 2018 (EDT)

Well, the English-language article in Wikipedia says that "Bable" was a former designation, and the Asturian-language article says that although "Bable" was sometimes used to refer to the language as a whole in the past, it is now restricted to one of the local variants. But I've posted on the CP to see if anyone has heard the term used. --Vasha 00:33, 17 March 2018 (EDT)

Author name fixes, 2018-03-21

G. De La Ree --> G. de la Ree; Leland De La Durantaye --> Leland de la Durantaye. Vasha 22:59, 21 March 2018 (EDT)

Done! Ahasuerus 23:27, 21 March 2018 (EDT)

Publication Content Limits

I'm posting to your page as this is a rather specific question. I'm going through Science Fiction: The Early Years as a result of its addition as a verification source(Thanks for that, by the way). As you probably know, the publication is split into two publication records([1]) because of the number of titles (mostly reviews) that it contains. I noticed that we are missing a review that should fall in first pub record as it appears on page 23. While attempting to add it, I was unable to add an additional review row. I believe this limit is, at least partially, a result of the code on line 325 of edit_js.js in the GetLastRow function that assumes no more than 999 rows of a given type. As we have, I believe, 17,777 reviews in that publication, we've completely blown through that limit. My question is whether that is the only place that limits the contents for a publication, or are there issues with the server code or the underlying database with handling a large number of content records? I believe that I could work around the javascript issue by adding the new review to a dummy publication and subsequently importing them. However, if that will create problems, we'll need to find another way. I should note that we'll likely have encounter the same issue with the Supernatural Fiction guide. I've slowly been working on adding the reviews to that one, but it's far from complete. Thanks for your time. --Ron ~ Rtrace^Talk 12:47, 24 March 2018 (EDT)

Well, there is good news and then there is bad news. The good news is that your diagnosis was correct -- the immediate problem was being caused by the fact that certain recent JavaScript changes limited the number of titles of the same type (regular, reviews, interviews) per publication to 999. I have changed the limit to 1,999 for now, so the Bleilers should be OK for a while. They contain 1,777 and approximately 1,400 reviews respectively. You may need to do a full page reload (Control-F5) for the new JavaScript code to take effect.

The bad news is that JavaScript is inherently slow. Most browser implementations used to be really slow. Now that they have been optimized, they are simply slow. Checking the status of 1,700+ reviews takes a long time. On the development server, which is admittedly showing its age, it takes 20-30 seconds. I'll need to poke around to see if we can improve it.

In addition, the XML libraries that we use to create submissions have reported memory leaks. In the past it caused the submission creation process to fail when trying to create a New/Add/Edit/Clone Publication submission with a lot of titles -- or at least that's what we believed at the time. It doesn't seem to happen with extra-large submissions any more, presumably because our server has more memory than it had back in the day, but it may become an issue again if we try to put ever more titles into a single pub record.

In addition, there are a couple of places in the Unmerge and Remove Title code which assume that a pub has fewer than 2,000 titles, but that should be easy to fix. Certain other limits also exist, but they shouldn't cause any issues, e.g. we are limited to 200 authors per title. Of course, there may be other hidden limits, but I am unaware of them at this time.

I guess the next step is to review the JavaScript code to see if it can be made faster. No rest of the wicked... :-) Thanks for identifying the issue! Ahasuerus 14:56, 24 March 2018 (EDT)

Thanks. That worked to add the missing review, thought with a couple of glitches. Aside from the delay while the js processed, the server did not return properly. However, the edit was waiting in the queue after that failure, so I was able to approve it and proceed. --Ron ~ Rtrace^Talk 17:14, 24 March 2018 (EDT)

My guess would be that the submission process ran into a timeout either between the browser and the Web server or further down the line. It seems to work OK on the development server, so it's likely a configuration issue on the main server. Ahasuerus 17:34, 24 March 2018 (EDT)

I had encountered the js delay before (with the supernatural Bleiler), which is why I know about the import from a dummy pub hack. Ideally, I'd love to see the two pubs combined into one, but I won't attempt to make that happen until the other fixes to limits are accomplished. I haven't counted how many titles are in the secondary pub, but it would likely push it over 2,000. Anyway, thanks again and there is no hurry on fixing the other limits. --Ron ~ Rtrace^Talk 17:14, 24 March 2018 (EDT)

Sure thing! Unfortunately, our editing tools are not very good at handling massive amounts of information in a single submission, but changing the framework would be time-consuming. Ahasuerus 17:34, 24 March 2018 (EDT)

(unindent) There is still apparently a problem with large submissions; I submitted this, with 84 new content titles, and the last 29 didn't go through. --Vasha 17:55, 26 March 2018 (EDT)

Sorry to hear about the trouble! I see that this submission errored out earlier today, but I am not sure why it did. Only 6 submissions have errored out in 2018, none of them due to the number of Contents titles, so I doubt it's that. I'll try to recreate the problem tomorrow morning after refreshing the development server using the latest live backup. Ahasuerus 18:08, 26 March 2018 (EDT)

After updating the development server with the latest data from the main server, I re-approved your submission in the development environment. The approval process took a long time, probably more than a minute, but it did go all the way through.

I suspect that, as we discussed earlier, the reason why it completed on the development server and didn't complete on the main server is that the latter has different timeout settings. I also wonder if it may be taking longer to approve reviews: for each new review, the approval process tries to find a matching title to link to, which may be more time-consuming than I realized. I'll try to poke around and see what I can do. Thanks for reporting the issue! Ahasuerus 12:28, 28 March 2018 (EDT)

Strangely enough, I'm still working my way through the Bleiler book and along the way I've been adding reviews that were missed when it was first entered. These generally time out, which is fine. When they do so, I jump over to the submission queue and approve my edits are there. Tonight I made three attempts to alter the page numbers where the reviews on a page had gotten out of order. It failed each time, with the last one occurring at about 21:27 EDT. Unfortunately, this error apparently prevented the edit from appearing in the queue. There were no additional content records with this edit, but since this publication has so many reviews, I'm understandably suspicious that it may be why it is failing. The error page has the heading "Internal Server Error" with content suggesting that I email you. However, I decided to post here since it has the prior history above. I'll refrain from making any more edits to that publication for now. I've concurrently made edits in other tabs without any issues. Thanks. --Ron ~ Rtrace^Talk 21:40, 2 August 2018 (EDT)

Thanks for the update! I am not seeing your failed submissions in the list of Errored Out and "In Progress" Submissions, which makes it even odder. I'll try to recreate the problem on the development server once I finish coding and debugging the patch that I am currently working on.

To be perfectly honest, our current setup makes me uneasy. We are allowing up to 2,000 titles per publication, which our software doesn't fully support. I think that ultimately we'll need to either improve the software to make everything work correctly, or lower the number of titles per publication to something more manageable. Ahasuerus 14:13, 3 August 2018 (EDT)

I gave the edit another try tonight and it went through i.e. still timed out but appeared in the queue. I'll let you know if I encounter any other issues. I've been thinking a little about ways to handle high content publications. The obvious solution would be to implement some sort of paging, which is kind of what was done, in a primitive way, by using two pub records for the Bleiler. I'm just not sure how hard it would be to modify the interface to deal with editing a portion of a publication. I believe this is usually done to address retrieval problems. While it does take a while to load this pub, it's not much of an issue for me. Others may think differently. I don't think the database would be an issue. However, it would be a significant change to the UI, and I expect would only change a handful of publications unless the page size was made very small. The cost to benefit ratio for this may not make it a high priority. Thanks for looking into it. --Ron ~ Rtrace^Talk 22:05, 3 August 2018 (EDT)

Project Gutenberg and ISBN

Hi, Last hour I some data to publication P339370 including links to WorldCat and that ISBN-10 which those records report. Perhaps that shouldn't be done for any PG ebooks. Probably it has broken the handy left-marginal link to the ebook, which I no longer see displayed at Other Sites --a link that I believe I followed hours ago. --Pwendt|talk 21:45, 24 March 2018 (EDT)

I have removed the ISBN from that pub & everything is back to normal. Project Gutenberg does not have ISBNs so they should be assigned to their pubs. I don't know why WorldCat is listing an ISBN for them, but WorldCat is not the most reliable. Care should be taken with its data. -- JLaTondre (talk) 21:50, 24 March 2018 (EDT)

Thanks for taking care of this issue! Ahasuerus 22:07, 24 March 2018 (EDT)

Modification to Premio Ignotus

The Premio Ignotus is inspired by the Hugo awards and the voting is the same, with a No Award option. It occurs to me (belatedly!) that we should be displaying the results like the Hugos, that is, ranked. (The ranked results up through 2013 are available in this article.) Hope it won't be much trouble to convert... if you have to delete all the existing award records in order to change the format, that's OK; creating those wasn't much trouble compared to adding records for the works. --Vasha 09:21, 25 March 2018 (EDT)

No worries! I have changed Premio Ignotus to be a "poll", i.e,. ranked. Nominations now appear as "rank 9", which is how nominations are stored internally for non-poll awards. They will need to be adjusted bases on the list that you linked above. Thanks for babysitting this award! Ahasuerus 11:47, 25 March 2018 (EDT)

Is it possible to make the Premio Especial Gabriel an exception? That isn't a poll. --Vasha 12:11, 25 March 2018 (EDT)

I am afraid not. At this time the "poll" flag is set at the award level. There is a Feature Request to change the way it works:

• ... there are three awards ('Lc' for Locus, 'Ar' for Asimov's Readers' Poll, and 'Ca' for Campbell) whose records are considered "poll places" rather than wins/nominations. This is hardcoded in the software. We need to eliminate the hardcoding by allowing award types to have both regular win/nomination award records and polls. This is needed because it's possible for an award to switch from the win/nomination approach to the polling approach from year to year. It's even possible to have an award announce the first three places followed by a list of nominees for places 4-N. In addition, in Locus' case, a list of nominees is announced first and is later converted to poll results.

• It is proposed that the data entry page be changed to have the following choices:

1. Win (button)
2. Nominee (button)
3. Polling place (1-200)
4. Special (drop-down list of choices for Honorable Mention, Withdrawn, Nomination Below Cutoff, Preliminary Nominee, etc)

Unfortunately, the required software changes would be fairly time-consuming. Ahasuerus 12:17, 25 March 2018 (EDT)

OK, no problem. You will have free time for it sometime in the next century no doubt... :-) --Vasha 12:31, 25 March 2018 (EDT)

Any millennium now! :-) Ahasuerus 13:02, 25 March 2018 (EDT)

(unindent) There is a category missing: Mejor antología (Best Collected Work), which was added in 2001. --Vasha 18:02, 25 March 2018 (EDT)

Added. Ahasuerus 18:33, 25 March 2018 (EDT)

Python Error on Merge

When I approved this edit, I received the following Python error:

  <type 'exceptions.IndexError'>	Python 2.5: /usr/bin/python
  Sun Mar 25 19:54:43 2018
  A problem occurred in a Python script. Here is the sequence of function calls leading up to the error, in the order they occurred.
  
   /var/www/cgi-bin/mod/ta_merge.cgi in ()
    371                 merge = doc.getElementsByTagName('TitleMerge')
    372                 if merge:
    373                         MergedTitle = TitleMerge(db, doc)
    374                         submitter = GetElementValue(merge, 'Submitter')
    375                         if debug == 0:
  MergedTitle undefined, TitleMerge = <function TitleMerge at 0x83447d4>, db = <_mysql.connection open to 'localhost' at 810a90c>, doc = <xml.dom.minidom.Document instance at 0x8345b6c>
   /var/www/cgi-bin/mod/ta_merge.cgi in TitleMerge(db=<_mysql.connection open to 'localhost' at 810a90c>, doc=<xml.dom.minidom.Document instance at 0x8345b6c>)
    203         doColumn(db, KeepId, merge, 'Series',     'series_id')
    204         doColumn(db, KeepId, merge, 'Year',       'title_copyright')
    205         doColumn(db, KeepId, merge, 'Storylen',   'title_storylen')
    206         doColumn(db, KeepId, merge, 'ContentIndicator', 'title_content')
    207         doColumn(db, KeepId, merge, 'Juvenile',   'title_jvn')
  global doColumn = <function doColumn at 0x834472c>, db = <_mysql.connection open to 'localhost' at 810a90c>, KeepId = '2333049', merge = [<DOM Element: TitleMerge at 0x834d8cc>]
   /var/www/cgi-bin/mod/ta_merge.cgi in doColumn(db=<_mysql.connection open to 'localhost' at 810a90c>, KeepId='2333049', merge=[<DOM Element: TitleMerge at 0x834d8cc>], label='Storylen', column='title_storylen')
    126         id = GetElementValue(merge, label)
    127         if id and id != KeepId:
    128                 moveTitleColumn(db, column, KeepId, id)
    129 
    130 def retrieveNoteIds(field_name, KeepId, DropIds):
  global moveTitleColumn = <function moveTitleColumn at 0x83446bc>, db = <_mysql.connection open to 'localhost' at 810a90c>, column = 'title_storylen', KeepId = '2333049', id = '2337591'
   /var/www/cgi-bin/mod/ta_merge.cgi in moveTitleColumn(db=<_mysql.connection open to 'localhost' at 810a90c>, column='title_storylen', keep='2333049', drop='2337591')
    107         result = db.store_result()
    108         record = result.fetch_row()
    109         value = record[0][0]
    110         if value is not None:
    111                 update = "update titles set %s='%s' where title_id='%d'" % (column, db.escape_string(str(value)), int(keep))
  value undefined, record = ()
  <type 'exceptions.IndexError'>: tuple index out of range

As far as I can tell, the result looks fine, but thought I'd mention it just in case. -- JLaTondre (talk) 19:59, 25 March 2018 (EDT)

Thanks, I'll take a look... Ahasuerus 20:00, 25 March 2018 (EDT)

Vasha77 did have a pending edit (since approved) to add an award to that title in case that is related. -- JLaTondre (talk) 20:02, 25 March 2018 (EDT)

It looks like there were 2 almost identical Title Merge submissions: 3754452 and 3754446. 3754446 was approved at 19:54:38 while 3754452 errored out, probably shortly thereafter. The submission approval logic is supposed to check that all of the affected records still exist in the database, but it looks like this particular submission type may not perform the check correctly. I'll take a closer look tomorrow morning. Thanks for reporting the problem! Ahasuerus 20:18, 25 March 2018 (EDT)

Wild characters search for external idenfifier

Hi,

Are the */% and _ supposed to work in the external identifier search? Thanks! Annie 00:03, 26 March 2018 (EDT)

<pokes the code with a feather> Apparently not at this time. FR created -- thanks for reporting the issue! Ahasuerus 00:06, 26 March 2018 (EDT)

Fixed -- and look what a search on "ASIN like A%" has found! :-) Ahasuerus 00:19, 26 March 2018 (EDT)

Thanks! I will fix it and do some creative searches through these the next few days :) I was trying to find bad Goodreads references - caught one today from a newbie and he went and fixed one of his old ones after that so I wonder how many we have Annie 00:46, 26 March 2018 (EDT)

Someone hit the wrong field with that one - the cover artist was in the ASIN field :) Annie 01:00, 26 March 2018 (EDT)

Odd quirk with review display

In this publication, on page 79, I have the review title displayed the way it is in the book ("Qué difícil es ser Dios") but linked to the record for the Russian original title "Трудно быть богом." For some reason, the review record displays the mouseover text for the record it's linked to. --Vasha 21:25, 26 March 2018 (EDT)

That's odd. Let me take a closer look... Ahasuerus 09:09, 27 March 2018 (EDT)

I see what's going on. The title of the Spanish REVIEW record is "Qué difícil es ser Dios", but it's linked to the original Russian title, "Трудно быть богом". The way our software works, it displays the transliterated titles associated with the linked title record, hence the mismatch.

The easiest way to fix the problem would be to enter a Spanish translation of this novel -- FantLab has this 1975 translation on file -- and then link the review to it. Might as well enter the 2011 translation, which was apparently done by a different team. Ahasuerus 12:11, 28 March 2018 (EDT)

The fact that there are two different translations is precisely why I linked to the top-level record--that way, people click on the link and then scan down the list of translations. I will get around to entering them both eventually. --Vasha 12:30, 28 March 2018 (EDT)

Well, the review was published in 2001, i.e. before the second translation appeared, so it was presumably based on the 1975 translation. Then again, it's conceivable that the reviewer read the original text or another (French, English, etc) translation, but used the Spanish title of the book in the review. It's hard to be sure with translations -- I have seen all kinds of bizarre permutations.

As a general rule, we link reviews of variants/translations to their respective VT records. That's why the VT page says that the canonical title page "may list more publications, awards, reviews, votes and covers". Ahasuerus 12:51, 28 March 2018 (EDT)

Seems odd to display the transliteration text for the linked record though; what if the review record also had a transliteration? --Vasha 12:30, 28 March 2018 (EDT)

It's a limitation of the current software design. The assumption was that the displayed title would always match the linked title -- which is true in 99.9% of all cases except when dealing with certain linked reviews. I'll create a bug report and see if I can tweak the code. Always something :-) Ahasuerus 12:51, 28 March 2018 (EDT)

(unindent) OK, I think I have fixed it. Ahasuerus 16:11, 8 May 2018 (EDT)

New User Name

Can I modify my user name, or do I have to get a new account (if necessary can I get a new user account). —The preceding unsigned comment was added by Factoficacc‎ (talk • contribs) .

I am afraid there is no easy way to change a user name. However, you can log out on the Wiki side and then create a new account. Ahasuerus 10:13, 28 March 2018 (EDT)

Translation of interviews not displayed?

Hi. I'm not sure right now if it has always been this way or if it is a (possibly new?) bug: the "Interviews with This Author" section on this the author's biography page only displays the Chinese parent record of the interview, but not the German translation, whereas the interview's title record does. There are no pseudonyms involved in this case, so I don't see where I could have missed something when entering the data. Well, hopefully I didn't... :) Is this desired behaviour? Jens Hitspacebar 12:29, 30 March 2018 (EDT)

Hm, that doesn't look right. Let me take a look... Ahasuerus 13:10, 30 March 2018 (EDT)

OK, I remember it now. Originally, only interviews "by this author" were displayed at the bottom of Summary pages. Interviews "with this author" were displayed on a single (potentially very long) line at the top of the page. The latter did not (and still do not) display variants. When I moved them to the bottom of the page, I wrote:

• As per FR 252, the list of author interviews has been moved from the top of Author pages to a new section, "Interviews with This Author". The section appears immediately after the "Interviews by This Author" section. The sorting within the section is chronological or alphabetical depending on the type of the author page. Each line displays the following values: title, year, language (if different from the author's working language), the name(s) of the interviewer(s) and the name(s) of the co-interviewee(s), if any. At this time the section does not display variant titles. Ahasuerus 00:31, 12 August 2016 (UTC)

It's possible to make this section display variants, but it's not a straightforward change. I have created an FR for this task. Thanks for looking into the issue! Ahasuerus 16:11, 30 March 2018 (EDT)

Ok, thanks for checking. Jens Hitspacebar 16:20, 30 March 2018 (EDT)

Both titles (the chinese and the geman) were in the publication. I've removed one. Note that this seems to be the usual way, see Leiber where his translated interview does not appear on the page. Hauck 13:34, 30 March 2018 (EDT)

@Hervé: why did you remove it? They are both in the publication, the original and the translation, as stated in the pub's note (it's a bi-lingual magazine).

In my experience, a "doubled" title in a publication sometimes has unexpected results. So it was worth a try as such configuration (an interview and its variant in the same book) is extremely rare (not to say unique). Hauck 16:56, 30 March 2018 (EDT)

I see. I re-import it again. Jens Hitspacebar 17:01, 30 March 2018 (EDT)

The software assumes that title records are unique within each publication. There are safeguards in place to enforce this rule, but it's hard to be 100% sure because of things like out-of-order submission approval.

That said, the software is not supposed to have a problem with the presence of related (VT-parent or VT-VT) title records within the same pub. If it causes a problem under certain circumstances, then we'll need to narrow it down and create a bug report. Ahasuerus 17:37, 30 March 2018 (EDT)

@Ahasuerus: Hervé's title removal however showed another possible bug: I don't see a notification in "My Changed Primary Verifications" that a title has been removed from it (this one). Jens Hitspacebar 15:00, 30 March 2018 (EDT)

I am afraid "My Changed Primary Verifications" is currently triggered by EditPub submissions only. Import/export submissions, title removal submissions, etc do not affect it at this time. Ahasuerus 15:40, 30 March 2018 (EDT)

Oh, I see. I wasn't aware of that. Thanks. Jens Hitspacebar 16:20, 30 March 2018 (EDT)

Doesn't that list get updated only overnight anyway or does it get updated as soon as someone changes something? Annie 17:19, 30 March 2018 (EDT)

The list is updated as part of the EditPub submission approval process. Sometimes I find it mildly annoying that I see the "New!" message multiple times per day, but I haven't heard any complaints. Ahasuerus 17:39, 30 March 2018 (EDT)

I guess I had been lucky - I've never seen it later in the day (but then I do not have a lot of verifications yet). Thanks for the explanation. Annie 17:49, 30 March 2018 (EDT)

Unneccessary warning message

I changed an ISBN from 10-digit to 13-digit, and when I submitted the change I got the message "ISBN already on file." This actually was the only publication with that ISBN. I guess what's hapening is that the change triggers the software to search for matching ISBNs, and it comes up with the existing version of the same publication. If it's not too much trouble, you could stop that message from being displayed when the match is the same pub. --Vasha 20:11, 31 March 2018 (EDT)

As it happens, I ran into the same issue a few days ago. It doesn't happen very often, but we might as well fix it. Bug 698 has been created. Thanks for the ping! Ahasuerus 20:43, 31 March 2018 (EDT)

Fixed! Ahasuerus 17:52, 4 April 2018 (EDT)

Problematic Verifications by Inactive Editors

The following four works all have secondary verifications with publication notes stating the verification is bogus: 279425, 126421, 248488, & 274104. In each case, the verifier is not longer active. For the two OCLC cases, I have double checked that the publication note is correct. For the Reginald3 & Clute cases, I don't have those references. I assume the only way to remove an inactive verifier's verification would be for you to manually edit the database? Is that worth doing (assuming the Reginald3 & Clute cases are confirmed)? -- JLaTondre (talk) 12:56, 1 April 2018 (EDT)

That's right, I would have to edit the data manually. We may want to consider adding a moderator-only column to the Secondary Verifications table, something like "Remove invalid verification". A topic for the Community Portal? Ahasuerus 13:02, 1 April 2018 (EDT)

Does it happen enough to warrant an interface change vs. just manually handling? If an interface change, would there be a way to limit it to inactive (ex. haven't edited in 60 days) verifiers? See this conversation where, if this feature existed, someone might have used it when they shouldn't have. -- JLaTondre (talk) 13:14, 1 April 2018 (EDT)

That's a good point. Typically, I try to avoid on-off scripts because they are error-prone and because it's usually more productive to spend development time on tools that can be re-used in the future. However, I can see how it could cause other problems in this particular case. Let me see if I can check Reginald3 & Clute. Ahasuerus 14:24, 1 April 2018 (EDT)

I can confirm that the Reginald3 verification is incorrect; Reginald3 refers to the original hardcover edition, not to the Orbit reprint.

The Burning Court is a more complex case. Clute/Nicholls says "1937" while Clute/Grant says "1937 (UK)". My first reaction was to keep the Clute/Nicholls verification and delete the Clute/Grant one. However, there are additional issues at play here. The first edition of the Clute/Grant encyclopedia was published in 1997. A slightly revised version came out in 1999. The online version was further corrected and expanded. Similarly, Clute/Nicholls, which was first published in 1993, was revised in 1995 (twice) and then again in 1999. If different editors use different versions of these verification sources, they are liable to create somewhat inconsistent verifications. I am not sure what (if anything) we can do about these types of discrepancies. Ahasuerus 15:32, 1 April 2018 (EDT)

Ken Kelly new canonical name

Hello,

I am in the process of changing the canonical name for Kelly and in a few books, the note does not specify if the Ken W. Kelly is used because it is credited this way (in which case I will variant) or because it was the canonical at the time (so the coverart needs a new author). Can you please check the following books:

• Vazkor, Son of Vazkor

Thanks in advance! PS: I have to go through a few more of these so I may need to ping you again. Annie 16:44, 2 April 2018 (EDT)

I have checked my copy and updated the record: "Cover signed by the artist; "Ken W. Kelly" is credited on the copyright page." Ahasuerus 16:58, 2 April 2018 (EDT)

Awesome, thanks! Annie 17:00, 2 April 2018 (EDT)

Weird image issue problem

Hello,

When you have a chance, can you look at this? I have no idea what is different between these and the ones that worked properly (and the system accepted them - they are just not visible on the thumbnails). What am I missing? Thanks! Annie 20:17, 2 April 2018 (EDT)

Never mind - had to start posting here so I can see it - it is over 600 px on one side. Shouldn't the software stop it from uploading? Because it works now - just the image is not visible on this page. Annie 20:19, 2 April 2018 (EDT)

Unfortunately, it's a limitation of our version of the MediaWiki software. The only way to change the behavior -- that I know of -- is to upgrade it, which I am too scared to do on my own. Ahasuerus 20:22, 2 April 2018 (EDT)

So I just should pay a bit more attention when my usual source of covers does not have it. Our 600 px on a side limitation is because of that, right? Thanks! Annie 20:33, 2 April 2018 (EDT)

That and because large images take up a lot of disk space. Even after the latest backup optimization our free disk space is at 15%. I'll need to scrounge around to see what else I can delete. Ahasuerus 21:09, 2 April 2018 (EDT)

True... Do we need a bigger subscription/new disks? We seem to be adding a lot of images - which is good but if we are struggling for space, it will get worse... Annie 21:13, 2 April 2018 (EDT)

Let me see if I can find more unused junk floating around. I could have sworn that, given our database size, we shouldn't be that high... <poke poke poke>

Eek! One of our more obscure log files hadn't been purged in 10 years! Off with its head! Now we are up to 44% free disk space. Problem solved :-) Ahasuerus 21:56, 2 April 2018 (EDT)

That's one hell of a big log file :) Annie 22:11, 2 April 2018 (EDT)

10 years is a long time! :-) Ahasuerus 22:43, 2 April 2018 (EDT)

(unindent) I have created a Service Request and adjusted our log file rotation settings accordingly. We'll find out how well the changes work in a few weeks. Ahasuerus 15:46, 4 April 2018 (EDT)

Middle French

Hello Ahasuerus. Would it be possible to add a "Middle French" language tag to the list ? I'll be needing it for some new authors I have recently added (and possibly some older ones who are wrongly tagged "French" or "Old French”. TYIA, Linguist 05:59, 5 April 2018 (EDT).

Sure, Middle French is an ISO 639-2-recognized language, so there should be no issue. Let me see if I can add it real quick. Ahasuerus 08:58, 5 April 2018 (EDT)

Thanks a lot ! Linguist 09:02, 5 April 2018 (EDT).

Done! Ahasuerus 10:10, 5 April 2018 (EDT)

Great ! Thanks again. Linguist 10:14, 5 April 2018 (EDT).

Fuzzy dating

Hello again ! I'm sure the subject has already been raised by some editors, but I can't find any mention of it at the moment : would there be a way for the system to accept (or to be made to accept) an approximate date, say something like 14..-00-00 to indicate "15th century" or 142.-00-00 to indicate 1420-1429 for instance ? This would allow to make the difference between the complete lack of a publication date for a work (e. g. The Odyssey) and an approximate datation for a text known for certain to have been written in a specific century or decade ? This could also be used, of course, in biographic data. This is a problem I have met quite a few times, and so have others, I suppose. There is probably a technical reason why it can't be done, but I can see no harm in asking and learning… :o) Linguist 11:22, 5 April 2018 (EDT).

Dates are a HUGE can of worms in the IT world! Each programming language and each database handles them somewhat differently. The database that we use, MySQL, officially supports dates between 1000 AD and 9999 AD, but it will grudgingly accept values in the 0001-0999 AD range (which we take advantage of.) It does not accept dates prior to 0001 -- see FR 46 -- and it doesn't accept fuzzy dates.

Having said that, there is a way to support fuzzy dates. We don't necessarily have to rely on the database to store dates as "MySQL-compliant" dates. We could change all date fields from the "MySQL date" type to the "arbitrary string of characters" type. We would then have to handle all date operations (e.g. determining which date was earlier during Title Merge operations) in our software, which we have full control over. However, there would be significant costs associated with this approach. First, all databases are optimized to handle dates quickly. If we were to move date handling to our software, many operations involving dates would slow down. Second, it would take dozens and dozens -- probably hundreds -- of man-hours to implement this functionality. Overall, I would say that it wouldn't be feasible given our constraints.

I should also mention that I have seen references to compromise solutions. Typically, they involve keeping all dates "as is" and adding a new field for "unknown date information" information to each record. The latter would list the date components which are dubious or unknown, i.e. the century, decade, year, month or date. It would be up to our software to convert entered values like "14..-00-00" to "1400-00-00/unknown decade and year" and then display them correctly. I don't have direct experience with this approach, but it looks reasonably straightforward. It shouldn't affect performance and it should take less time to implement than a complete date revamp. Still, it would be quite time consuming. Ahasuerus 12:22, 5 April 2018 (EDT)

Considering that there is a note field displayed close to to the dates on both author and title pages, putting date info in notes if it's inexact is probably enough. Maybe if there is both a date note and other notes, put the former alone on a separate line at the start of the notes. --Vasha 13:38, 5 April 2018 (EDT)

Maybe a checkbox for "approximate date" and another for BCE/CE?--Rkihara 14:34, 5 April 2018 (EDT)

Thanks for all the info, and sorry about all those worms crawling everywhere ! I like the idea of a checkbox which would solve part of the problem, for ca. dates anyway. Linguist 04:29, 6 April 2018 (EDT).

I can see how a checkbox for "approximate" dates would be useful. It would let us display the word "[approximate]" on Summary, Title, Publication, etc pages.

On the other hand, a "BCE" checkbox would be difficult to do right. We would need to modify all of our date comparison operations, reports, validation, etc to account for it. That's a lot of work for relatively little gain.

I suggest we copy the relevant parts of this discussion to the Community Portal to see if there are additional considerations that we may be missing before we create a Feature Request. Ahasuerus 10:43, 6 April 2018 (EDT)

Done ! Linguist 11:08, 6 April 2018 (EDT).

Need immediate solution to the problem of non-standard author names

Good morning... I apologize for being demanding, but there's a situation which has been ongoing for too long (I first posted about it on Rules and Standards months ago, and it still isn't resolved)— how to follow author preferences for (e.g.) use of periods or capitalization in their names. Once again today a moderator standardized an author name I had entered (G Benson) even though I was using the form that's in both the book and the author's website. Can I ask you to work on getting this solved?

I love Annie's suggestion of having an extra field in the author record that would contain the author's preferred spelling and cause their name to be displayed that way everywhere, while the main Name field would contain a standardized form. That makes it easy to enter publications, just use the standardized form, no need to look up what the author usually uses. A note in the publication record would be optional--if you know several different spellings are in use, note which one is used in that publication record. This wouldn't be too technically difficult, right?

And if we don't go with that solution, the Help needs to be clarified. Thanks a lot... --Vasha 08:32, 6 April 2018 (EDT)

Sorry, but there's no need to make this urgent, since we do have valid rules, AND we have such a desired field: it's called the note field. Any additional information regarding an author might be put into this nice item. Stonecreek 13:32, 6 April 2018 (EDT)

Christian, you have said many times that you disagree with my interpretation of the current rules, but I haven't heard from anyone else. That's why I'm begging and pleading for an update to the help section that will clarify matters. This can't be just between you and me. -- Vasha 13:54, 6 April 2018 (EDT)

(unindent) I agree that our current lack of consistency re: author initials can be frustrating. I myself have run into this issue on a number of occasions.

I think the underlying problem is that common usage has changed over the last few decades. A hundred years ago, even 50 years ago, it was understood that having a period after an author initial was the standard. Occurrences of author initials without a period were due to typographical conventions (notably at some digest magazines) and not to be taken seriously by bibliographers. The few cases when an author insisted on a "naked" initial -- famously Forrest J Ackerman -- were seen as affectation at best. Over time common usage changed while our data entry standards haven't. It's not an uncommon scenario in the bibliographic field: we have repeatedly changed our data entry standards and the software to support e-books, audio books, links to 3rd party Web pages and so on.

That said, every time the world changes, it takes some time for our standards to catch up. Being too slow is bad, but being too hasty can be bad as well. For example, back when Wikipedia became big (2005-2006) Al and I kind of assumed that it would ultimately become the repository of all knowledge. Our plan was to work closely with it and to move some of our data, especially biographical data, to Wikipedia pages. Within a couple of years we realized that Wikipedia was not necessarily the panacea that we had thought it would be, e.g. their notability policies meant that they would never have Web pages for most of our authors. We also realized that our decision to create a single "Wikipedia URL" for author records had been wrong. We ended up rewriting the software to support multiple 3rd party URLs per author and, later on, per title, per publisher, per award type, etc.

And so, even though I understand the frustration, I think it's important to come up with a comprehensive consensual solution to make sure that we don't have to backtrack and re-do things in the future. Annie's suggestion is an interesting one, but there are other potential approaches. For examples, we could adopt a combined approach:

• state that due to recent changes in the publishing world we would start entering author names as printed on title pages and create pseudonyms/VTs when needed, and
• create a new "standardized author name" field in the author record; it would always include periods after initials and would be used by the search logic so that users could find author names regardless of the way they are "initialied" in books/magazines

Please don't be discouraged by the fact that the last Rules and Standards discussion didn't yield any immediate results. Sometimes it takes a number of iterations before we find a solution that is both feasible and addresses everyone's needs. Then we have to prioritize it, but that's a whole different issue. Ahasuerus 13:58, 6 April 2018 (EDT)

I had been on the record with a similar position as Christian as well - trying to bend the rules causes more problems than it solves and makes both adding books and moderating records much harder. I think that we should just stop trying to follow every fad of US publishing and just follow our rules and use the Notes (and/or new fields) to ride them. I dread working on magazines and small publishers anthologies these days because of creative naming conventions for the authors and us not following our own rules. Annie 14:27, 6 April 2018 (EDT)

That is certainly true, Annie, it is a big messy mess! I'm just trying to figure out what I should do right now, while things are unsettled (and while current rules allow for exceptions but are poorly worded as far as explaining exactly which exceptions are allowed).

I've been compiling quite a long list of authors who use non-standard initials/capitalization/spacing, along with which publications various spellings are used in. I guess I should put that on a Wiki page so that we can all refer to it. --Vasha 14:56, 6 April 2018 (EDT)

How about an innovative approach? Follow the rules. Initial, dot, space. Initial, dot, space. Notes in the author name and in the publications with creative naming conventions and we have all the information that we need. These are fixable downstream if needed anf if we change the rules but in the meantime they do not cause triple work for everyone.

I know you mean well and that you are trying to advocate for the authors but this is a DB and consistency is important. We need a system identification for the author - this is what our canonical name is for the most part - we default on the most used name but... how initials are different from suffixes - or are we going to decide to break that pattern as well? And if we start following this for the US market and keep changing the rules on the fly, what happens when I start asking the same for the Bulgarian ones for example? Let's not go there, shall we? :)

Plus the DB is a scary place for new editors anyway - making it almost impossible to figure put how to enter a name does not help matters. If anything, I think we should go back and clean the mess we had already created by not following our own rules. Annie 15:03, 6 April 2018 (EDT)

Or abandon any pretenses that we are actually following rules and start recording names exactly as they are written in the publication. Which I am even less of a fan of. Annie 15:05, 6 April 2018 (EDT)

I don't disagree with you; standardization would be user-friendly. But there was a decision, years ago, to allow exceptions. And that's still in the rules. So strict standardization isn't following the rules. That's one of the things this inconclusive discussion has been about--should we overturn the former decision that exceptions are allowed? --Vasha 15:08, 6 April 2018 (EDT)

(unindent) In any case, let's stop hijacking Ahasuerus's page and move this back to Rules and Standards. Apologies, Ahasuerus. --Vasha 15:17, 6 April 2018 (EDT)

No problem, I understand that sometimes frustrations boil over :) Ahasuerus 15:53, 6 April 2018 (EDT)

Numbers request

When you have a chance, can you get some numbers on how many LCCN and OCLC we still have in the system that need converting? I am just curious - I know we have tons of them but I like seeing numbers going down now and then :) Thanks! Annie 18:52, 6 April 2018 (EDT)

Sure thing. As of last Saturday morning:

• LOC: 8,667
• OCLC: 54,431 (!)
• DNB: 4,014

I'll check again once I do the backups tomorrow. Ahasuerus 19:35, 6 April 2018 (EDT)

Thanks! We always knew that we have a ton of the OCLC ones so no surprise there. Considering where we started, we are making progress (I know some went over with the automation at some point especially on the OCLC ones). I'll go spam more people (the PV'd change spamming thingie) and move some more now. Annie 19:52, 6 April 2018 (EDT)

2018-04-07 update:

• LOC: 7,940 -- 8% in one week
• OCLC: 52,598 -- 3% in one week
• DNB: 3,884 -- 3% in one week

Ahasuerus 11:20, 7 April 2018 (EDT)

Thanks for getting these! Annie 15:30, 7 April 2018 (EDT)

(unindent) I'd love to see some numbers this week if it is not too much trouble. Thanks! Annie 16:33, 15 April 2018 (EDT)

Certainly:

• LOC: 6,554 -- 1,386 or 17.5% processed
• OCLC: 50,869 -- 1,729 or 3.2% processed
• DNB: 3,692 -- 192 or 5% processed

Ahasuerus 16:51, 15 April 2018 (EDT)

Thanks! Looks like we are getting somewhere with providing new housing for all these poor misplaced IDs. :) Annie 17:18, 15 April 2018 (EDT)

Yes, we are making nice progress. And once we add support for multiple ISBNs per pub, we'll need to create a cleanup report for ISBNs embedded in Notes. Always something! Ahasuerus 17:26, 15 April 2018 (EDT)

When you build the ISBN report, let's make sure we compare the numbers or all of the notes I am dealing with now for the LCCN by Michael will come back again - they have LCCN, ISBN and year in a nice line in the middle of the note (like this one although it is not the best example as this LCCN is for a different edition so it is templated now and not moved to ID and left in brackets)). Or if impossible, that report will need a generously applied "ignore". Annie 17:42, 15 April 2018 (EDT)

True. 17,297 notes include the string "ISBN". That's a lot. Ahasuerus 17:46, 15 April 2018 (EDT)

Compared to the 75K OCLC we started with (and this is just links - we are not even looking for the ones that are just dumped as text - I move these when I see them but getting them all will need a new report looking for the string OCLC and that will be another huge project; same for LCCN btw) and the 100K or so language-less titles I looked through last year when we were doing the semi-automated migration, 18K is peanuts. Still, any that we can discount will be awesome. Annie 17:52, 15 April 2018 (EDT)

And then there are the translators... we need to talk about that sooner or later and then migrate that as well... Annie 17:42, 15 April 2018 (EDT)

The main thing about translators is that we still don't have a solid design which would cover all possible permutations without introducing a metric ton of additional complexity. Now that the '+' functionality has been implemented, we may be in a better position to start doing realistic mock-ups. Ahasuerus 17:51, 15 April 2018 (EDT)

Oh, I know that. So I had been silently doing other stuff. But I will keep nagging about it because we do need it. Or mentioning it now and then anyway :) Annie 17:54, 15 April 2018 (EDT)

(unindent) This week's numbers? :) Annie 16:58, 22 April 2018 (EDT)

Here they are:

• LOC: 5,200
• OCLC: 48,850
• DNB: 3,627

Pretty good progress! Ahasuerus 20:05, 22 April 2018 (EDT)

Thanks! The OCLC ones going down is almost a side effect (some were from their own report but not too many) - I am targeting the LCCN ones and a lot of them carry an OCLC number as well :) Annie 20:12, 22 April 2018 (EDT)

(unindent) Numbers from last weekend (or whenever the last backup was)? (I keep getting distracted...) :) Thanks! Annie 17:41, 3 May 2018 (EDT)

On 2018-04-29 we had:

• LOC: 4,548
• OCLC: 48,103
• DNB: 3,604

Chugging along nicely. Ahasuerus 17:49, 3 May 2018 (EDT)

(unindent) How do the numbers look like after today's backup? Thanks!Annie 16:13, 19 May 2018 (EDT)

• LOC: 1,409
• OCLC: 45,049
• DNB: 3,435

LOC is almost done! Ahasuerus 16:21, 19 May 2018 (EDT)

Yep, although if we just look at the raw numbers, more OCLC numbers got rehoused since early April :) By the way - we will need the LCCN report to acquire an "ignore" option - some books print the link on their copyright page and when an editor copies it in their record, I think we should preserve it. I found ~6 of those so far. Either that or change the report to look for the href as well (which would have missed a few) so I kinda prefer the ignore option. Not urgent - just mentioning it. Annie 16:32, 19 May 2018 (EDT)

At this time the LOC report looks for "lccn.loc" in Note bodies. Notes that use the LCCN template do not appear on this report, so shouldn't we be OK as is? Ahasuerus 17:08, 19 May 2018 (EDT)

Example. The report catches it (because it is lccn.loc) but I do not want to edit it more (I already pulled the number) because it is how it is on the copyright page. Annie 17:38, 19 May 2018 (EDT)

Oh, I see. That's ... unexpected. Thanks. I guess we'll need to add the ability to ignore records after all. Ahasuerus 17:44, 19 May 2018 (EDT)

We have very little of them as of yet but more and more books do print the URL so we may get more. :) Progress I guess - books catching up to the fact that links mean something and can be used... :) Annie 17:49, 19 May 2018 (EDT)

PS: and we have a few thousand non-linked LCCNs that also need moving but that is the next stage of this project (and all the cleanup projects) - we do not need them for the link cleanup but if we do for the search :) Annie 16:36, 19 May 2018 (EDT)

True. We have 4,207 notes with embedded "LCCN:" and 2,594 notes with embedded "LCCN ", which will need additional cleanup. Ahasuerus 17:08, 19 May 2018 (EDT)

Some of the ones you are finding are already moved but left also in the notes because of the pattern that the editor is using (data from copyright page cited even if it in other places on the record as well). We also have some BLIC ones that I keep finding (although the vast majority of the BLIC notes read "No BLIC") - BL being one of our finished cleanups. I usually look for these non-linked ones with "contains LCCN and does not contain {{LCCN". It catches a lot more than it should but there is some creative punctuation out there ("LCCN(", "LCCN;", "LCCN-" that I had stumbled upon. Annie 17:38, 19 May 2018 (EDT)

Other Log of Phileas Fogg

Some time ago you verified (according to the db) that in the 1982 version the novel started on page 5, when it starts on page 7--per the copy I hold here in my hand, having just reread it while on vacation. Page 5 has an interior illustration and page 6 is blank. The introduction, which is really part of the story, begins on page 7. I normally would just fix this, but Hauck insists that this change can only be made if there is a discussion with the PV's. Now I'm not really sure how to transmit such a discussion to make such a change--as there are two PVs. This statement was sent to the first PV appearing on the list of the page in question. Shall I copy it to PV #2 or does a discussion between the two of you take place?

There is sort of a logic breakdown here, however. If someone is detail-oriented enough to submit such a minor correction, the correction should be accepted. The alternative, which is that people who are detail-oriented like me will only submit minor corrections that call for a dialog if they are not introverted and not conflict avoiders. Getting corrections only from ppl who are detail-oriented, not introverted, not conflict avoiders, and with some time severely limits the sources of corrections to a subset of a subset of what is already a small number of people, which ultimately reduces the accuracy of ISFDB more than does putting a PV(s)-check process in place for trivial changes.MartinStever

Thanks for the heads-up! I have checked my copy and it matches your description -- there is an illustration on page 5 with the body of the novel starting on page 7. Ahasuerus 18:57, 8 April 2018 (EDT)

Same here. --~ Bill, Bluesman 12:09, 9 April 2018 (EDT)

So perhaps, even if the publication is not a magazine, the page number "5" may have been chosen in accordance with this part "Exception for works which have illustrations preceding their title pages - If a magazine presents artwork for a story or essay preceding the piece's title page, and it is apparent that the art accompanies the text, the starting page of the story or essay should be the page number of the artwork which illustrates it. If you're creating content records for both the work and its illustration, they would have the same starting page. (See "Sorting" below for multiple works appearing on the same page.) If there is no indication that the artwork is related to the text on the succeeding pages, and no indication in the table of contents that it illustrates the work, then do not count it as the first page of the work. " of our guidelines. Hauck 03:35, 9 April 2018 (EDT)

I was wondering about that myself. I think it only applies to magazines, but we may want to ask our "magazine people" for more background info and perhaps clarify Help. Ahasuerus 09:40, 9 April 2018 (EDT)

I'm not so sure the above should apply to magazines or other forms at all. We are predominantly a print database [electronic or paper], with the artwork a secondary level [excluding graphic pieces, of course]. After all, we are recording the story, at whatever length, as a product of the author - not the artist. How can a piece of art be said to 'start' a story that it only illustrates? Excepting those works where the author has also illustrated the story, and even then I'd bet the print preceded the paint. --~ Bill, Bluesman 12:09, 9 April 2018 (EDT)

As far as our notification process goes, it was created based on our early experiences with editors unilaterally changing verified data. The main issue is that there can be many slightly different versions of the same book: US vs. Canada, subsequent printings, book club reprints, etc. Sometimes an editor may think that the book in his hands matches what's in the database, but, upon inspection, it turns out that he has a different publication. That's why we ended up implementing the current notification process. Although it's not perfect and sometimes cumbersome, it's been our experience that its advantages outweigh its disadvantages. Ahasuerus 18:57, 8 April 2018 (EDT)

Does anyone check with the other PV, or is the issue settled? How does the change get put in now that you've agreed that data should be updated? MartinStever

I have left a note on the other PV's Talk page. He is quite active, so he will hopefully see it soon. Ahasuerus 23:31, 8 April 2018 (EDT)

What if the PV no longer has a copy to refer to?MartinStever

There are 2 types of primary verifications: Permanent and Transient. Transient verifications can be done against library books, books seen at a bookstore, etc. In this case the primary verifications are both Permanent. If one or both were Transient, the fact would be displayed on the Publication page. If one of the verifiers no longer has access to the publication, he can change the verification type from Permanent to Transient. Ahasuerus 23:31, 8 April 2018 (EDT)

ISFDB history question

I know you had been around since the early days so thought I will just ask you. The first ~55,000 books seem to be in alphabetical order (the LCCN and OCLC cleanup lists highlighted that if one was paying attention). Was there a re-IDing process at some point or were they added this way? (I am just curious - I hate not knowing the answer). Thanks :) Annie 16:38, 9 April 2018 (EDT)

The short answer is "I am not sure" :-) The long answer is that my early (ca. 1995-1997) contributions were on the design side while Al did all of the coding. The original, pre-beta, data entry was done by Al using various award lists and such. He may have done it alphabetically, but I don't think he entered anywhere close to 55,000 books before we allowed public submissions. I suspect that the alphabetization most likely occurred when Al migrated the data from the original, custom-built, database format to MySQL in 2004-2006. I wouldn't swear to it, but it would be my best guess. Ahasuerus 17:12, 9 April 2018 (EDT)

Based on the dates of some of the verifications and some other dates here and there in comments, ~2005 sounds about right and I suspect you are right. It is unimportant in the grand scheme of things - it was just something curious that caught my attention and a migration and re-IDing when moving to the DB makes sense. Plus there are some non-English books in those first 55K so I doubt that they were entered at the very start of it. But who knows. All this cleanup had been educational - it is almost like a walk back in time :)

On a somewhat related topic - how are the 500 selected for the OCLC/LCCN lists? Roughly 400-450 are the next 400-450 in the ID ordered list (thus me noticing the pattern - I am inclined to think that it is closer to 50 out of sequence ones if not less than that but unless I clear the whole list, I cannot see the ones that are hiding in the current alphabet order...); the rest are much higher numbers with no rhyme or rhythm for their appearance on the lists (except that once they appear, they will stay there until someone gets them resolved). So it is not random - I just cannot figure out what is the algorithm for them. Annie 17:25, 9 April 2018 (EDT)

The query simply says "select p.pub_id from notes n, pubs p where p.note_id = n.note_id and n.note_note like '%lccn.loc%' limit 500", so there is no explicit ordering. For single-table queries the default MySQL sorting order is typically based on IDs, but this query has a join. Presumably it's the join that throws things off because the engine needs to build a temporary table whose IDs may be in a different order. Ahasuerus 17:33, 9 April 2018 (EDT)

Ah, I see. Yeah, it is the join - my guess is that the node_id is the one that causes these much later works to get into the list so early. I hate writing any queries with no order so was trying to figure out what the order is coded to - especially because it is not random (it stays the same after a regeneration). Thanks for answering :) Annie 17:51, 9 April 2018 (EDT)

Sure thing. And I have no idea why I originally typed "joint" instead of "join" - I blame gremlins! Ahasuerus 17:54, 9 April 2018 (EDT)

Apparently they breed - I somehow managed to make the same typo (I think it was a hands type what eyes see and not what brain knows kinda thing)... Annie 18:02, 9 April 2018 (EDT)

(unindent) It seems like the books were just entered in alphabetical order (with some stragglers added from other people in between) - some weird ones show up in the 27K range that do not belong there if they were reordered at some point. :) Annie 19:25, 14 April 2018 (EDT)

What links here? and code search

Hi. Do we have any functionality equivalent to Wiki "What links here?" from the ordinary database views of Author, Publisher, Series, etc? The report would list, perhaps without subclassification, all of the database pages that contain links to the current one.

What I now know I can do is search the Notes fields of Title, Author, and Publication pages separately for code such as "[full personal name]}", which will report pages of that kind (Title, etc) that contain template links to the author page with matching personal name; and "[full personal name]</a" for handmade links.
(I see that ISFDB Advanced Search hits strings such as "</i" and "&thinsp", which is to search the database itself rather than the views we display at isfdb.org/cgi-bin, if i understand correctly.) --Pwendt|talk 15:35, 12 April 2018 (EDT)

I am afraid the ISFDB software doesn't support anything like "What Links Here?". The best way to approximate it (that I can think of off the top of my head) would be to use Google and limit the search to "site:isfdb.org". For example, a search on "Todd McCaffrey site:isfdb.org" also finds Anne McCaffrey's page because her Author Note field mentions that she was Todd's mother. Ahasuerus 17:05, 12 April 2018 (EDT)

Thanks. Capability to search the Notes field of Publisher, Series, Magazine pages is more likely to be valuable to me and I'll visit the wish list (next section) later this month. --Pwendt|talk 23:18, 12 April 2018 (EDT)

wish list

Does ISFDB.org/wiki have a space dedicated to wish-list matters. At the moment I have in mind technical matters including such things presumably much simpler than What links here?, such as [1] expansion of ISFDB Advanced Search to all Notes fields, or separately Notes of other kinds Publisher, Series, etc; [2] report of the number of hits at the top of search reports, and such pages as "My Rejected Edits", which are search reports I suppose; [3] filter or [4] sort of such search reports. --Pwendt|talk 15:43, 12 April 2018 (EDT)

Yes indeed! You can create Feature Requests here. Ahasuerus 16:53, 12 April 2018 (EDT)

Wiki Cleanup - Fanzines can be killed

I finally finished the remaining fanzines so the Wiki/DB connection for Fanzines can be killed (if it is still around that is...). :) Annie 18:20, 13 April 2018 (EDT)

Many thanks! I am afraid we won't be able to break the database-Wiki link for fanzines until we finish migrating all of our series and magazines -- they all use the same linking logic (although it gets a bit hairy with magazine-series redirects.) Ahasuerus 19:03, 13 April 2018 (EDT)

Right - now I remember - apparently my memory does not work very well :) I had been hitting some series cleanup this afternoon anyway so will let you know when we can kill the whole gnarly web then :) Annie 19:11, 13 April 2018 (EDT)

"Memory"? I am pretty sure I have heard the term :-\ Ahasuerus 19:34, 13 April 2018 (EDT)

(unindent) All the series were either migrated so the whole mess of fanzines, magazines and series can now be unlinked. We probably need to clean the formatting and some of the links in the still remaining wiki pages that are now linked via the URL field (I did some minor fixes here and there but there are still weird formats and redundant information there) but as of right now, the Series wiki cleanup is finished. Annie 15:15, 13 June 2018 (EDT)

Thanks muchly! I'll see what I can do on the software side. Ahasuerus 17:20, 13 June 2018 (EDT)

Based on your priorities - do we need Authors or Publishers cleared first (won't be fast but I occasionally feel like dealing with those)? :) Annie 17:33, 13 June 2018 (EDT)

I don't really have a preference. On the one hand, we have under 350 publisher-related Wiki pages vs. 1460 Author pages (and 1850 Bio pages.) On the other hand, some editors have been blanking out migrated Author pages instead of deleting them, so the actual workload on the Author side may be less than the numbers suggest. Whatever works best for you! Ahasuerus 19:20, 13 June 2018 (EDT)

We also have just 240 or so publication related wiki pages, most of them like the one I linked in the other discussion (although I need to make another pass to see what is easily migrateable) and I am not touching those for now. I will probably hit publishers next then (with random hits into the authors pages). Don't expect quick results though (unless I decide I am sick of them and just tear through them as I did this morning with the series) :) Annie 19:44, 13 June 2018 (EDT)

The thing about publication-related Wiki pages is that they use a different algorithm. There is no "Webpages" field in the publication record, so the report parses publication Notes fields looking for matching URLs instead. Once FR 957 has been implemented, we'll be able to standardize the process. Ahasuerus 19:53, 13 June 2018 (EDT)

So inefficient and taking forever. We need to migrate them (so this parsing can stop) and we cannot because we have nowhere to put them. Until the URL links show up, the only way to get them off the report is to incorporate them in the notes and as these are all verified publications, I'd rather not do that when we will need to edit again anyway. So when is 957 planned for? Annie 20:02, 13 June 2018 (EDT)

At the moment I am working on security; all other non-trivial development is on hold. We really don't want to end up like MAL, an online anime database with somewhat similar functionality. Their whole site was down for many days and then came back with its functionality crippled. The developers are apparently still trying to sort out the mess :-\

I am about half way through the first round of requisite changes, so we are making progress, but we are nowhere near done. Once that's out of the way, we can review the "big" FRs and prioritize them. Ahasuerus 20:29, 13 June 2018 (EDT)

Meant to put a smile at the end of the question. Sorry. :) There are enough things one can work on - so not urgent at all.

Security and stability are way ahead of anything else - anything and everything else can wait. Annie 20:37, 13 June 2018 (EDT)

No worries! One step at a time :) Ahasuerus 20:57, 13 June 2018 (EDT)

Destroyed data

Hello,

Any chance to find the destroyed content of this book from archives? It happened sometime in the last 4 days as it was still there on the 10th. At this point my only guess is that someone decided to "clean" the multilanguage report by deleting data instead of trying to fix it - which I am not even going to comment on. Annie 18:29, 14 April 2018 (EDT)

Sure, we can give it a try. I am currently restoring the 2018-04-10 backup on the development server. It should be done within the next 30 minutes. Ahasuerus 18:36, 14 April 2018 (EDT)

Thanks. If you have a quick way to find out who did this (quick scan through the recently approved did not show it but I will look again), that will also be very helpful (so I can drop them a note not to do it again). I really dislike people destroying data that is not against the ROA that someone took the time to add. Annie 18:45, 14 April 2018 (EDT)

Here is what the Contents section looked like on 2018-04-10:

 Preface 1 (Voices from the Sky) [English] • (1966) • essay by Arthur C. Clarke
 8 • Space Flight and the Spirit of Man [English] • (1961) • essay by Arthur C. Clarke
 20 • The Uses of the Moon [English] • (1961) • essay by Arthur C. Clarke
 39 • To the Stars [English] • (1966) • essay by Arthur C. Clarke
 53 • Beyond Centaurus • essay by Arthur C. Clarke (trans. of Beyond Centaurus 1964)
 71 • The Light of Common Day [English] • (1963) • essay by Arthur C. Clarke
 85 • Ships for the Stars [English] • (1962) • essay by Arthur C. Clarke
 103 • Seas of Tomorrow [English] • (1966) • essay by Arthur C. Clarke
 110 • The Winds of Space [English] • (1965) • essay by Arthur C. Clarke
 123 • Time for the Stars [English] • (1966) • essay by Arthur C. Clarke
 138 • The Playing Fields of Space [English] • (1965) • essay by Arthur C. Clarke
 151 • Preface 2 (Voices from the Sky) [English] • (1966) • essay by Arthur C. Clarke
 153 • A Short Pre-History of Comsats, Or: How I Lost a Billion Dollars in My Spare Time • essay by Arthur C. Clarke (trans. of A Short Pre-History of Comsats, Or: How I Lost a Billion Dollars in My Spare Time 1966)
 164 • The Social Consequences of the Communications Satellites [English] • (1961) • essay by Arthur C. Clarke
 178 • Broadway and the Satellites • essay by Arthur C. Clarke (trans. of Broadway and the Satellites 1966)
 184 • The World of the Communications Satellite [English] • (1964) • essay by Arthur C. Clarke
 199 • Preface 3 (Voices from the Sky) [English] • (1966) • essay by Arthur C. Clarke
 200 • Kalinga Award Speech [English] • (1962) • essay by Arthur C. Clarke
 207 • Memoirs of an Armchair Astronaut (Retired) [English] • (1963) • essay by Arthur C. Clarke
 224 • Science and Spirituality [English] • (1966) • essay by Arthur C. Clarke
 228 • Class of '00 • essay by Arthur C. Clarke (trans. of Class of '00 1966)
 235 • The Meddlers [English] • (1964) • essay by Arthur C. Clarke
 243 • The Lunatic Fringe [English] • (1966) • essay by Arthur C. Clarke
 251 • The Electronic Revolution [English] • (1962) • essay by Arthur C. Clarke
 259 • H. G. Wells and Science Fiction • essay by Arthur C. Clarke (trans. of Introduction (The Invisible Man/The War of the Worlds) 1962)
 269 • "Dear Sir ..." [English] • (1966) • essay by Arthur C. Clarke
 283 • Extraterrestrial Relays [English] • (1945) • essay by Arthur C. Clarke (variant of Extra-Terrestrial Relays)

I'll see if I can determine who deleted it next. Ahasuerus 20:01, 14 April 2018 (EDT)

P.S. I wonder if it may be safer to (temporarily) put this list of essays in the Note field to avoid similar problems in the future. Ahasuerus 20:08, 14 April 2018 (EDT)

That is my plan - yes :) I do not mind the removal (it is in the wrong language and needs to be replaced anyway) - it is the total deletion of it that got me. I could have "solved" a lot of problems in the last year this way instead of doing the legwork to find the proper parents and sources. Annie 20:22, 14 April 2018 (EDT)

OK, we have our answer. The titles were removed by Hauck on 2018-04-11 at 02:16am server time. Ahasuerus 20:30, 14 April 2018 (EDT)

That's surprising - he is not prone to deleting valid data and this had been on the report since forever without anyone even touching the report. Thanks for checking - I will add them as notes for now. Annie 20:40, 14 April 2018 (EDT)

After Annie's "Call to People" which I expected would yield more positive results, keeping data that was completely useless (it was just a import of the original book) and false (some titles in in english were tagged as being in german) was not bibliographically sound. As for the report, I sometimes gave a look at it but was not happy with what was on it as it was (at the time) the result of either "unfinished" (either bt the submitter of by the follow-on moderator) submissions or concerning books (or set of cards!) that were and are outside our scope. Hauck 01:35, 15 April 2018 (EDT)

That one publication had been there for awhile and it was the very last from a very long list of publications that we had started with - a project that I am very happy to be finishing. I could have just deleted the whole mess a few months ago - but I believe in trying to correct data and not remove it :) And it was not just an import - the page numbers do not match so someone did the work to match. I've parked the content in the Notes and hopefully we will find a book to get that sorted out. Annie 01:48, 15 April 2018 (EDT)

Well, in this case the existing data would finally have to be destroyed and was of little practical value (it was the content of the original publication). Not exactly as you, I believe in "no data is better that supposed data". Hauck 02:15, 15 April 2018 (EDT)

It was a leftover from the old way of saving data around here - virtually all the Italian publications were in the same form for example. I would not accept it as valid data now but the old data just needs conversion. Anyway - one of those days we will find these German titles. Annie 02:33, 15 April 2018 (EDT)

That is now the second time that you, Hervé, actively destroyed data. In this case, it may not have been entirley correct, but it is still of interest for some people. What's going on with you? Stonecreek 06:47, 15 April 2018 (EDT)

URL Entry Issue

The logic for validating URLs seems to be missing the case where the URL is entered with a single slash. The URL display logic then doesn't know how to handle it. See Gene von Edler for an example. The link works (or at least Chrome is forgiving of the invalid format). I've cleaned up a couple more of these by adding the second slash. So not a significant problem, but a few did make it through. Thanks. -- JLaTondre (talk) 20:34, 17 April 2018 (EDT)

Good point. Our URL validation checks that URLs start with "http". It should check that they start with "http://" or "https://" instead. Bug 700 has been created - thanks! Ahasuerus 21:14, 17 April 2018 (EDT)

Fixed. Ahasuerus 17:41, 3 May 2018 (EDT)

WorldCat and Fixer

If I remember correctly, Fixer had munched on a lot of non-processes WorldCat records that are still waiting for actual processing. Any Bulgarian ones (and Russian technically but let's start somewhere) by any chance (with Latin titles or not)? Annie 17:59, 19 April 2018 (EDT)

Let me have a chat with Fixer... Ahasuerus 19:45, 19 April 2018 (EDT)

OK, we have good news and we have bad news. The good news is that Fixer is aware of 100+ Bulgarian ISBNs, i.e. ISBNs starting with "954".

The bad news is that the data is partially garbled. Apparently many Cyrillic characters didn't survive the migration from 32 bits to 64 bits and the records are almost unreadable. It shouldn't be too much of a problem as long as we are dealing with 110-120 ISBNs -- we can simply look things up in WorldCat. Russian ISBNs are likely to be another story.

In any event, would you like me to post the Bulgarian ISBNs that Fixer is aware of on your Talk page? Ahasuerus 20:11, 19 April 2018 (EDT)

Yep, send them over. Then I will tackle the Russian ones as well - as long as there is the OCLC number, I can look them up and then need to go to FantLab anyway. Annie 20:36, 19 April 2018 (EDT)

Done. Re: Russian ISBNs, Fixer is aware of 5,989 of them. That's a lot of ISBNs. Given how sparse WorldCat's international data tends to be, I am not sure WorldCat is the best place to start. FantLab and other national bibliographies have much cleaner data, but, of course, they all use proprietary formats. Ahasuerus 21:27, 19 April 2018 (EDT)

I am thinking less of a start and more of using them as a base. Plus once you give me one ISBN, I can get more from the same series and authors and so on. Thanks for the Bulgarian ones. Annie 21:47, 19 April 2018 (EDT)

Well, if the goal is to create a list of Russian SF-flavored ISBNs, there may be other ways to tackle it. Although FantLab doesn't make its backups publicly available, one of our contributors, who is also a FantLab volunteer, has a local copy of their private backups. He could ask their administrators if it would be OK to extract the ISBNs. Unfortunately, FantLab has many non-SF pubs and ISBNs, so it may not be a perfect fit for us.

Another way to approach this issue would be to take advantage of the way they compile bibliographies. As I wrote on Usenet a few weeks ago:

• Fantlab's author bibliographies tend to be comprehensive due to the way they do things. While we (the ISFDB) are book-centric, they are author-centric. Each author is assigned a volunteer "curator" who monitors "his" authors' output and keeps the data up to date. Fantlab won't create an author page until a curator has been assigned.

This means that they have relative few author pages, but they tend to be very comprehensive and could be used as starting points. Ahasuerus 22:20, 19 April 2018 (EDT)

I still have the SFBG records to work through - so at this point I was looking more for something to change into more than really working on the Russian titles systematically. That will be in a couple of years if noone else tackles them. Let me work on the BG ones for now and will ping you if I want to see some Russian ones anyway. Thanks again for posting these :) Annie 01:05, 20 April 2018 (EDT)

Service with a smile! :-) Ahasuerus 08:52, 20 April 2018 (EDT)

BNB Links

I previously put this message on the Help page but did not get any feedback. I though that you might be able to shed some light on my problem.

The record for The Condition of Muzak has an external link to the British National Bibliography (BNB). When I clicked on this link it went to the British Library page which then stated that the reference did not exist. This is true for the Fontana edition of this novel, only the Allison & Busby edition is in the British Library catalogue. However the Fontana edition is in the British National Bibliography catalogue. Is there something wrong with the BNB link? The result when accessing the link seems to be odd. I've just re-selected the link and it did take me to the British National Bibliography page and found the correct record. However when I closed down all the pages and then reopened isfdb, selected the novel and clicked on the BNB external link it went to the British Library page and not the BNB page. --AndyjMo 06:25, 22 April 2018 (EDT)

Let me try this BNB link...

Hm, I see what you mean. When I first accessed the URL, it behaved inconsistently: sometimes it would display the Fontana edition while other times it would say that no matches were found. I then noticed that the page was asking me to agree to its terms and conditions (cookies?). Now that I have accepted it, it seems to display the results consistently. I am not sure if that was the problem, but whatever it is, it doesn't seem to be on our side. It would be nice if BNB supported direct linking by ID, but I don't think it's available yet. Ahasuerus 08:21, 22 April 2018 (EDT)

Thanks for looking in to it. It wasn't just me having a problem. Just have to accept that the BNB Links will not always return the required page. --AndyjMo 08:38, 22 April 2018 (EDT)

Series order for Twayne Triplet

There seems to have been a fairly recent assignment of publication months for the anthologies (one in August, the other in December of 1952), so that I have reversed the title order according to that logic (or is there a continuous flow of narration, which would seem odd, considering the schedule?). Stonecreek 13:23, 3 May 2018 (EDT)

It was more of a publication series than a regular series. The idea behind the series was that each anthology would contain three novellas and that each "triplet" would be thematically linked internally, but there was no connection between anthologies. I think the chronological order should be fine. We could even make it a publication series. Ahasuerus 14:07, 3 May 2018 (EDT)

Yeah, on the other hand it is some unique idea, that also can be seen as a idea for a series of anthologies. Maybe it could be both (though this very seldom). Stonecreek 15:24, 3 May 2018 (EDT)

Omnibus contents

See this submission. I take it he was unable to put all of the requisite values in the Contents field. --MartyD 09:55, 5 May 2018 (EDT)

The field is currently limited to 32 characters while this record would require 36 characters. Let me see what I can do... Ahasuerus 13:57, 5 May 2018 (EDT)

OK, I have made the requisite changes on the development server, but SourceForge is having problems at the moment. I'll try to commit again later this afternoon. Ahasuerus 15:32, 5 May 2018 (EDT)

Done! Ahasuerus 19:28, 5 May 2018 (EDT)

Thank you kindly! :-) --MartyD 11:02, 6 May 2018 (EDT)

Sure thing. I am in no shape to undertake anything major right now, so I am going after the low hanging fruit anyway. Ahasuerus 17:44, 6 May 2018 (EDT)

Sorry to hear that. Let me know if there's anything on the ISFDB front I can help with. --MartyD 20:40, 6 May 2018 (EDT)

Thanks. I guess the fundamental problem is that, as John W. Campbell observed some 55 years ago, no one is getting any younger. Even when I am feeling OK, I just don't have the energy that I had even a few years ago. And, of course, my Fixer-related workload keeps going up. We may want to consider various ways of spreading the load. Ahasuerus 08:38, 7 May 2018 (EDT)

Is there anything I can do as far as taking some of the workload of new additions? --Vasha 10:33, 7 May 2018 (EDT)

Thanks for volunteering! At one point Annie and I worked on making User:Fixer/Public open to all editors, but the project kind of stalled a few months ago. We should make it a priority once Annie is back from her vacation.

For now, would you like to take a look at some of the old mini-projects which have accumulated over the last couple of decades? For example, User:Ahasuerus/OldBiblios contains 2 sections:

• Books on writing SF
• Books about the legend of the Wandering Jew

The first section should be easy because chances are that all (or almost all) of the listed books/ISBNs are already in the database. The second section may take more time to process because it's longer and because the books are more obscure, so they may require additional research. Once everything has been entered into the database proper, we can delete the Wiki page. Ahasuerus 11:53, 7 May 2018 (EDT)

I will look into those once I finish adding all the works nominated for the Premio Ignotus (still need to do 2006-2017). --Vasha 12:01, 7 May 2018 (EDT)

Thanks! Ahasuerus 12:22, 7 May 2018 (EDT)

And please do consider ways to get help with Fixer work. --Vasha 12:01, 7 May 2018 (EDT)

I agree that manually approving/massaging thousands of Fixer-derived submissions is not the best way to spend developer man-hours. Unfortunately, our experience with turning them over to moderators has been mixed. Some types of submissions, notably e-AddPubs, work quite well. Others don't, probably because I have accumulated a great deal of in-depth knowledge about the publishing world which is hard to transfer to other moderators. Hopefully the "Public" approach will help alleviate the problem. Ahasuerus 12:22, 7 May 2018 (EDT)

Back and I will put that on my list of things for this week - it requires a lot more attention than the LCCN cleanup (thus me leaving it on the back burner when tired) but I will go and finish these instructions so we can kick that off. Annie 13:25, 14 May 2018 (EDT)

Welcome back and thanks! Ahasuerus 14:11, 14 May 2018 (EDT)

I had been thinking about it a few times and kept pushing it back. :) Are you feeling better? Annie 17:48, 14 May 2018 (EDT)

I am back to normal for now. Unfortunately, it's "the new normal", which isn't as good as "the old normal", but it is what it is.

There are a number of things that I'd like to start unloading, but it will require additional thought. For example, I am considering rewriting Fixer in Python/MySQL. However, even if I could do it relatively quickly, I am not sure it would be viable. Running Fixer as it currently exists is very time-consuming even without approving/massaging Fixer-generated submissions, almost a full time job. There may be other ways to get the data that we need. Food for thought. Ahasuerus 18:29, 14 May 2018 (EDT)

Time never stops and one needs to get older :) It is annoying but despite the main focus of our DB, we still need to live in the real world... :) Annie 18:55, 14 May 2018 (EDT)

It beats the alternative, at least for now :-) Perhaps I just need some time off. It's been over 12 years since I had a real vacation. Ahasuerus 22:29, 14 May 2018 (EDT)

Part of our problem is Amazon - it is a notoriously ratty source (while at the same time it is the only one that is somewhat complete for new editions). Had you explored using WorldCat as the main source instead? Their data is usually less ratty (for some value if less anyway). Annie 18:55, 14 May 2018 (EDT)

Fixer has had a WorldCat interface for many years. His database includes hundreds of thousands of XML-formatted WorldCat-derived records. Here is the current breakdown:

Total ISBNs: 1,063,329
Already in ISFDB: 343,926 including 228,085 non-English ISBNs
Not in ISFDB: 719,403 including:
  English non-audio: 67,467
  Non-English: 404,367
  Audio/digital: 247,569

The quality varies: some are good enough to create ISFDB records and some are nowhere close. The big difference vis a vis Amazon is that WorldCat's pre-publication data is at best rudimentary and often non-existent. At this time Amazon is our only reasonably reliable source of forthcoming books.

One of the things that I have been considering lately is converting this Fixer subsystem (which I haven't been leveraging anyway) into a separate Python/MySQL application and then turning it over to another maintainer. It's almost entirely independent from Fixer's Amazon interface, so it shouldn't be too difficult to do. Ahasuerus 22:29, 14 May 2018 (EDT)

Yeah, OCLC is good for older stuff but new and forthcoming is... abysmal. I know that you have the records, I was thinking about processing them eventually. Anyway - I was just thinking aloud. :) Annie 23:12, 14 May 2018 (EDT)

Or maybe we should start talking about a 2-step process again (I brought that up last year) - dump the Amazon data in a table somewhere, make it editable and then automate the creation of requests from there - all the massaging will be in that table and the ones that need more work will just take longer. I know it is a new interface but... this way people can pitch in much easier. Annie 18:55, 14 May 2018 (EDT)

In a way, Fixer's "public" pages are just that, only not as spiffy as an automated subsystem would be. Ahasuerus 22:32, 14 May 2018 (EDT)

But it still takes you awhile to clean up the lists enough to post them there as opposed to dropping the raw data. But yeah - that was why they were created :) Annie 23:12, 14 May 2018 (EDT)

It so happens that I further automated the "Public" process a few weeks ago :-) The new version generates publisher- and author-specific files on demand, and then all I have to do is copy and paste them to Wiki pages. I still have to assign ISBNs to the appropriate Fixer queues, but that's part of the standard monthly processing. Ahasuerus 00:05, 15 May 2018 (EDT)

Ah, I missed that. Great! I'll let you know when I am done editing the rules page so we can review and get that kicked off. Annie 12:11, 15 May 2018 (EDT)

Can you point me to merge instructions?

Ahasuerus, can you point me to the page that has instructions on how to merge a double entry? I looked around, but I'm not finding it. Thank you! MartinStever 23:00, 7 May 2018 (EDT)

Certainly. The Help page you are looking for is called Help:How to merge titles. It's listed under Help:How to, our master list of pages which cover more advanced topics and usually involve more than one data entry page and/or multiple execution paths. Ahasuerus 00:09, 8 May 2018 (EDT)

Unmerge Titles Bug

Found an unmerge bug. It happens under the following conditions:

1. Publication has a cover art record
2. Same publication has an interior art record
3. The interior art record is a variant of the cover art record
4. You unmerge the cover art from other titles it is merged with

The result will be two newly created cover art records (rather than the expected one) and both will be in the publication (i.e. the publication goes from having a single cover art record to two duplicate cover art records). I assume this would happen with any case where a parent and a variant are in the same publication and the parent is unmerged, but found it with the cover / interior art case (which would be the most common one) and didn't try others. Thanks. -- JLaTondre (talk) 18:15, 9 May 2018 (EDT)

Thanks, I'll take a look. Ahasuerus 19:17, 9 May 2018 (EDT)

You were right, it affects all title types. Bug 701 has been created. Thanks for identifying the problem! Ahasuerus 16:11, 10 May 2018 (EDT)

Fixed. Hopefully. Ahasuerus 20:46, 11 May 2018 (EDT)

Book by country

Hello Ahasuerus, is possible add the option to separate every book/author by country? Example: book A Máquina do Tempo. Original language and Country: English, UK. This edition: Portuguese, Brazil. And so on (also make a "country directory" by author and "country directory" by book).

And in this and this lists also make explicit from which country every title/author is. Example:

Titles by language

• English: 1,177,543
  • US: X. number
  • UK: x. number

What do you think?

Thanks, ErickSoares3 16:57, 13 May 2018 (EDT)

Let me first copy this question to the Community Portal. Ahasuerus 18:34, 13 May 2018 (EDT)

Author Edit history

I seem to remember that we have the history of the author pages changes. Can you check thisand see if someone did a weird typo while editing the author? Thanks! Annie 13:34, 15 May 2018 (EDT)

Author name correction

Juan J. Gutierrez should be Juan J. Gutiérrez. I have managed to find previews of all of his publications; only one doesn't print the diacritic, and I made a note on that. --Vasha 16:08, 19 May 2018 (EDT)

Updated, thanks! Ahasuerus 16:09, 19 May 2018 (EDT)

+ in author names

I have entered an essay that's credited to "Asociación Cultural Alt+64" and I found that it wasn't possible to have the + in the name. Is there a way around that? --Vasha 20:59, 24 May 2018 (EDT)

Not at this time, I am afraid. It's a design flaw that goes way back and would take quite a bit of work to fix. Not impossible, just time-consuming :-( Ahasuerus 22:11, 24 May 2018 (EDT)

OK, no problem. Time for some notes.

Also, I have found yet another author needing an accent added: Joaquin Moreno Pedrosa should be Joaquín Moreno Pedrosa. Sorry about that. --Vasha 22:18, 24 May 2018 (EDT)

Done - thanks. Ahasuerus 23:54, 24 May 2018 (EDT)

Publications with Invalid Prices

Hello,

What exactly is this report checking? I keep stumbling on incorrect prices (from missing spaces or too many spaces to reversed currencies and numbers("100 din." for example) so I started wondering what exactly we are checking for. Thanks! Annie 16:55, 29 May 2018 (EDT)

This report looks for a limited number of "bad" patterns which have given us trouble in the past. At this time it is limited to:

• "CDN" instead of "C"
• spaces after "$", "£" and "¥"
• commas in US/UK prices
• prices without currency symbols

We could certainly add more patterns to it -- that's what FR 866 is for -- but I suspect that we won't get them all no matter what we try.

I expect that we will re-do the way we enter prices as part of adding "Support [for] multiple prices per publication" as requested in Roadmap 2017. Drop-down lists for currency signs/codes ("$/USD", "£/GBP", "€/EUR", etc) would be a good start. Of course, we would need to include "other" for obscure and obsolete currencies, which would be documented in notes. Ahasuerus 17:26, 29 May 2018 (EDT)

I was just curious (for example this one had a space after € awhile back - your explanation makes it clear why it was never caught).

I agree - that can wait the rewriting (then we can do some cleanup). One thing I would propose for prices is to have the ability to "un-obscure" - when you have 1000 publications with a common currency, having it in a drop down makes a lot of sense. Just saying :) Annie 17:46, 29 May 2018 (EDT)

I am not sure I understand what you mean by "un-obscuring" in this case. Could you please clarify? Ahasuerus 19:03, 29 May 2018 (EDT)

Have a process that allows currencies that are not considered obscured to be added to the drop-down (or whatever we come up with) :) Otherwise, it will keep feeling like English only DB :) Annie 19:24, 29 May 2018 (EDT)

Oh, I see. I assume that we will create a table of currency symbols and codes, so adding new currencies shouldn't be difficult. Determining what constitutes a new currency can be ... challenging, as we discussed last summer :-) Ahasuerus 20:18, 29 May 2018 (EDT)

There is that as well. Although if it get tied to a number of publications, it should be less challenging (maybe) :) Annie 20:27, 29 May 2018 (EDT)

Changing variant relationships

Technical question--when shifting a title A that's a variant of title B to instead be a variant of title C, do you have to break the old relationship with "0" before creating the new one, or can you make the change directly?--Vasha 19:50, 1 June 2018 (EDT)

You can make the change in one step. Each title record has a field where the title ID of its parent is stored (assuming there is one), so it's just a matter of changing the value in that field. Ahasuerus 19:55, 1 June 2018 (EDT)

Just to be perfectly clear: this also applies if you're using the make-variant form to create a new parent record -- you don't have to unlink before doing that either? (I'm sure I've done this plenty of times without unlinking, but the question came up in a discussion, so I want to make sure). --Vasha 21:04, 1 June 2018 (EDT)

That's right, you don't need to break the VT link first. Varianting to a brand new (i.e. previously non-existent) parent title should be fine. Ahasuerus 21:49, 1 June 2018 (EDT)

Errored-out submission

This submission didn't go through. Can you tell what's invalid about it? --Vasha 21:05, 1 June 2018 (EDT)

It looks like the submission errored out half way through the approval process, probably while trying to create a title record for "El arcángel de Nueva York (Part ?? of ??)". I'll try to re-approve the submission on the development server once the backups run tomorrow morning.

Do you happen to remember if you used copy-and-paste to enter this SERIAL title? I wonder if it may contain an odd special character that the software doesn't handle correctly. Ahasuerus 21:47, 1 June 2018 (EDT)

No need to resubmit it. I just re-added the missing titles and now they are all there. --Vasha 22:54, 1 June 2018 (EDT)

Oh, I mean that I plan to resubmit on the development server to see if I can recreate and fix the problem :-) Unfortunately, there are some subtle differences between the development server and the main server, which means that sometimes things don't work exactly the same. Ahasuerus 23:05, 1 June 2018 (EDT)

The submission was re-approved successfully on the development server. Which, unfortunately, leaves us with no easy way to debug the error :-( Ahasuerus 15:08, 2 June 2018 (EDT)

Ed Acuña

There seems to be something funny happening because of a special character: see this record and this record. They are different but because of the special character, they get mixed. See this to see the one that is actually a pseudonym. Changing the name of the main one unlocks them (that's how I managed to edit - then I restored the name) but I am not sure what we should really do here. Any idea how this happened at all? Annie 18:33, 2 June 2018 (EDT)

Looking... Ahasuerus 21:10, 2 June 2018 (EDT)

OK, I think I know what's going on. If I am right, it should be fairly easy to fix. However, it's 9:25pm on the East Coast and I no longer trust myself not to make a mess of things after 6pm :-( I'll take another look tomorrow morning and hopefully fix it. Ahasuerus 21:26, 2 June 2018 (EDT)

Have a good evening. I just noticed it and decided to report it - nothing critical is broken :) Annie 21:45, 2 June 2018 (EDT)

Thank Cthulhu for small mercies! Ahasuerus 22:05, 2 June 2018 (EDT)

Fixed! Ahasuerus 17:51, 3 June 2018 (EDT)

Thanks :) Any idea why we went down again this morning? Annie 17:59, 3 June 2018 (EDT)

First time I have heard of it. I can see that:

• the nightly jobs completed successfully at 1:19am
• the server hasn't been rebooted in 9+ days
• the MySQL server has been up for days

How long did the server stay down? What were the symptoms? Ahasuerus 18:15, 3 June 2018 (EDT)

I decided to go back to sleep so did not wait for it to come back but somewhere around 9 am Eastern this morning, I was getting the good old "too many connections to the DB" error for ~5 minutes when I tried to get to it. I do not know when it came back and don't have exact time (sorry - did not think to make a note of the time:( ) Maybe just an unexpected spike of activity? Annie 18:24, 3 June 2018 (EDT)

Ah, I see. The backups run at 9:30am Eastern, which effectively freezes the server for a few minutes. There is a related issue, Bug 604, which I should probably handle at the same time. Ahasuerus 18:43, 3 June 2018 (EDT)

This was definitely earlier - closer to 9 than to 9:30. Sounds like just a hiccup then so won't worry too much. Annie 18:52, 3 June 2018 (EDT)

I was going to work on Bug 604 anyway, so might as well add a more user-friendly message while the backups are running. That way there will be no ambiguity when it happens again. Ahasuerus 19:07, 3 June 2018 (EDT)

Accented characters in wiki-to-DB links

I'm sure you know about this already. The wiki templates that create links to author names in the DB don't work with accented characters. It doesn't bother me, because I don't use those much anyway; I just have been noticing it lately because of creating a lot of image wiki pages for Spanish artists. What workarounds (if any) do people usually use? --Vasha 19:43, 6 June 2018 (EDT)

Unfortunately, there isn't much we can do at this time: the Wiki and the database store accented characters differently. Eventually the database will be upgraded to use the same way of storing accented characters, but it will take time. For now, I usually use direct links when I need to connect a Wiki page to an "accented" author's Summary page. Ahasuerus 21:39, 6 June 2018 (EDT)

The templates should all support parameters to allow linking by id. Example: {{A|山田太一|107958}} produces 山田太一. The directions will be on the template page (ex {{A}}). If there is a template that doesn't support it, let me know and I'll fix it. For the image templates, you need to add "|ArtistId=number" manually. I think the upload link could be modified to add that (it doesn't hurt anything if present when not needed). -- JLaTondre (talk) 22:10, 6 June 2018 (EDT)

Oh, right. Sorry, I keep forgetting about this functionality. Ahasuerus 22:28, 6 June 2018 (EDT)

OK, that worked for the CID template. And yes, adding it to the upload would be a good idea. --Vasha 22:42, 6 June 2018 (EDT)

Sounds good, I'll review the upload code tomorrow. Ahasuerus 23:00, 6 June 2018 (EDT)

The upload process has been modified -- please see the announcement on the Community Portal. As I mentioned there, the change was limited to publications with one cover artist. Pubs with 2+ cover artists use templates CID1-2/CID1-3, which, as far as I can tell, do not support artist IDs at this time. Ahasuerus 11:14, 7 June 2018 (EDT)

Thanks! I'll take a look at those templates when I have the chance. -- JLaTondre (talk) 20:50, 7 June 2018 (EDT)

Great! Ahasuerus 21:05, 7 June 2018 (EDT)

Russian Science Fiction 1969

Can you dig out your verified and see if there is really a "Robot Humor" story on page 1946 or if it is just a heading for the section? FantLab does not list a mysterious story there... Annie 19:07, 7 June 2018 (EDT)

As per the Notes field in the pub record:

• "B. Zubkov's and E. Muslin's short-shorts on pp. 146-149 have an overall title, "Robot Humor"

I think it should be safe to "ignore" the mismatch, although we may want to update the Note field in the title record. Ahasuerus 19:46, 7 June 2018 (EDT)

I saw the note and it made me wonder - I don't think we should have it as a short story record at all (as it is not a short story - but a section of the book)... but your book, your decision. :) Annie 20:03, 7 June 2018 (EDT)

IIRC, we discussed "composite" stories a while back, but I don't recall the details. I don't have a strong preference one way or the other -- perhaps something to discuss on the Rules and Standards page? Ahasuerus 20:32, 7 June 2018 (EDT)

Let me sleep on it. I did not read the note as "composite story" but as "they grouped them in their own section" :). Annie 20:41, 7 June 2018 (EDT)

Take your time! BTW, here is one possible relevant Help sentence:

• Review columns [...] are entered as ESSAY types in the general content record, but you should also record the books reviewed, and who reviewed them, in [the Review] section.

In a way, composite stories are similar to review columns: they are both "container titles" of sorts. Ahasuerus 21:43, 7 June 2018 (EDT)

Once I start thinking of it as a composite story and not as a "section in the book that contain them", it makes sense and I agree that we should have it there. Maybe that note should be rewritten though - because now it is kinda confusing (well.... it confused me) :) Something like "The short-short stories are presented as a single story (link to the record) in the book. They are indexed here separately as well for bibliographic purposes." Or something along these lines. I am even thinking of listing the contents lists in the title record of the story itself... Annie 22:19, 7 June 2018 (EDT)

Good point -- I have updated the parent title, so it should be less confusing now. Hopefully! Ahasuerus 19:18, 9 June 2018 (EDT)

I like the new note. Thanks! :) Annie 19:54, 9 June 2018 (EDT)

Reports numbers

When you have a chance, can you check the outstanding numbers in the two translators reports in today's archive? I feel less like trying to scoop out an ocean with a tablespoon when I see some numbers moving down. :) And while you are at that - how do the OCLC and DNB numbers look like this week? Thanks! Annie 16:38, 9 June 2018 (EDT)

Sure thing!

• Translations without notes: 37,927
• Translations without the Tr Template in Notes: 30,191
• OCLC: 40,229
• DNB: 1,830

Ahasuerus 19:03, 9 June 2018 (EDT)

Thanks. Numbers are going slowly down :). I had also been thinking about the {{Tr migrations into whatever we come up with later on - some languages will be very easy, some not so much and a few will be almost manual (the non-Latin based ones plus probably Hungarian). I think that we will need to go language by language, clear the unusuals and then automate the clear ones. I know we are away from having a system, I had been just thinking while moving data around and looking all over internet for the data we miss. Still - at least I am clearing the wrong merges and finding the missing data.:) Annie 19:53, 9 June 2018 (EDT)

"Slowly"? At the rate the tablespoon is going, if I were the ocean, I would start panicking ;-) Ahasuerus 20:10, 9 June 2018 (EDT)

Well, slower than I wish they were anyway. And we keep finding new things to clean. :) I hope to have all of the external IDs into their proper fields before the end of 2018 - the linked ones anyway. We have a lot of OCLC numbers that are not linked and we have some not recorded (because people rely on the left menu's WorldCat link, we have OCLC verified books with no OCLC number in them.... - and once we get the links and the non-linked out, I think we need to track down these as well). The good news is that we will have a lot less false alerts than with the LCCN - because OCLC is not printed in the books. And more and more people seem to be moving OCLCs so... we will see. And then the translators by mid next year maybe. And then I will really start pestering you about getting a proper translators system in place :)

PS: I had not forgotten about Fixer's public page - I will see if I can get you a version to check this weekend - I am trying to put together some FAQs to make everyone's life easier (and get us as good quality records as we possibly can). Annie 20:36, 9 June 2018 (EDT)

Regarding translations: somehow I have missed the regulations for a multiple translated work (that is a work translated by more than one translator): can we already handle this case using the template? Stonecreek 09:05, 10 June 2018 (EDT)

{{Tr|A and B}} is what I do if there are two translators A and B. Annie 09:11, 10 June 2018 (EDT)

(unindent) I had been seeing a lot of activity in both the translators and external ID cleanups this week so how are our numbers looking on this week's backup? :) Annie 15:40, 16 June 2018 (EDT)

Here we go:

• Translations without notes: 35,372 (last week 37,927)
• Translations without the Tr Template in Notes: 29,378 (30,191)
• OCLC: 38,585 (40,229)
• DNB: 989 (1,830)

The DNBs are almost done! Ahasuerus 15:54, 16 June 2018 (EDT)

Yep - and most of the DNB ones are two for one deals (OCLC + DNB). Same for the BNF (where we have 725 left). All numbers are going down :) Annie 16:02, 16 June 2018 (EDT)

(unindent) The DNBs are done - how are the other 3 numbers looking this week? Annie 19:30, 23 June 2018 (EDT)

Let's see:

• Translations without notes: 35,182 (last week 35,372)
• Translations without the Tr Template in Notes: 29,219 (29,378)
• OCLC: 36,606 (38,585)

Good progress on the OCLC side. Ahasuerus 19:40, 23 June 2018 (EDT)

Weird format submitted

When you have a chance, can you look at this (or join us here) - somehow a non-standard format is being submitted. I saw a few of those the last few weeks on a report and kinda shrugged it off but we may be having a validation issue somewhere :) Annie 20:14, 15 June 2018 (EDT)

And another one and and one more Annie 20:16, 15 June 2018 (EDT)

Those happened when I attempted to submit a perfectly normal tp format via Microsoft Edge, a few minutes before I stopped being able to submit anything from that browser at all. --Vasha (cazadora de tildes) 20:19, 15 June 2018 (EDT)

Thanks, I'll take a look. Ahasuerus 20:24, 15 June 2018 (EDT)

As I mentioned on the Community Portal a couple of minutes ago, I have added additional validation to prevent confused browsers from submitting publications with invalid formats. At least it will prevent corrupted data from getting added to the database. It had been on my list of things to do for over a year, but it kept getting pushed back. Ahasuerus 21:16, 15 June 2018 (EDT)

Automated import?

Hi -- If I sent you a list of data pairs, one being an ISBN and one being a LTF external identifier, could you import the identifiers into the corresponding publication records? There's 206 of them, so if it was possible to save significant time & keystrokes, that'd be great. --Vasha (cazadora de tildes) 21:38, 18 June 2018 (EDT)

If it's just 206 LTF IDs, then it wouldn't be cost-effective because it would take me longer to code and test everything. However, if we can think of a way to keep the ISFDB database in sync with the LTF database going forward, then it's something that we may look into. Based on our experience with Fixer, it would likely involve a fair amount of development work, but we won't know until we look into it in-depth. Ahasuerus 22:14, 18 June 2018 (EDT)

No, never mind. There wouldn't be any good way to keep the databases in sync, not with multiple editions, uncertain editions, etc. These are just 206 records I happened to know how to match up because of notes I kept during the process of creating all those record for the Ignotus. So, I'll put 'em in by hand & it'll take a couple hours. --Vasha (cazadora de tildes) 22:56, 18 June 2018 (EDT)

Vasha, let me know if you need a hand - I can add some. I had been slowly moving the ones we have as links here and there) but if you have a list, I can as well do that. :) Annie 00:27, 19 June 2018 (EDT)

DNB links in non pub/title notes

When you have a chance, can you run a search in the backup through notes that are not in publications and/or titles for direct DNB links? I do not expect to find any but I found one in a title and I cannot do series and pub series notes via the Advanced search :) Other from that, we are all done with these. Thanks! Annie 14:16, 19 June 2018 (EDT)

Great news! I'll refresh the development server tomorrow morning and see if there are any DNB links in the "webpages" table. Ahasuerus 14:27, 19 June 2018 (EDT)

I am less worried about the webpages ones and more worried about series notes for example. Such as the one I pulled from here :) I can find the ones in authors and titles via the Advanced Search; I cannot do that for the rest of our pages :) Annie 14:38, 19 June 2018 (EDT)

Sounds good, I'll check Notes as well. The reason I mentioned URLs is that Advanced Search doesn't let you search Series, Publisher and Publication Series records, which can have URLs associated with them. Ahasuerus 14:52, 19 June 2018 (EDT)

Agree. Although I am not sure if we want to move them anywhere if they are in the URL field - the whole point in having the URL field is to have them there and not incorporated in the text (unless needed). So let's check the numbers but I do not think that we need to worry about them :) Annie 15:04, 19 June 2018 (EDT)

2 pub series and 1 publisher with "d-nb.info" URLs, all three apparently legitimate. No (!) notes with "d-nb.info". I think we can declare victory! :-) Ahasuerus 19:27, 20 June 2018 (EDT)

Finally a finished project... :) Let's celebrate. Oh, hold on - there are 120 more to work on. Oh well. :) Annie 19:55, 20 June 2018 (EDT)

One project at a time! There are days when I review what has been done over the previous day or week and it feels like we are hardly making any progress. Then I compare where we are now with where we were 5, 10, 15 or 20 years ago and I immediately cheer up :-) Ahasuerus

As I said before - scooping an ocean with a tablespoon (or 3). :) I get bored sticking to a single projects so I rotate projects usually - until one thing really annoys me enough to push it to the end... :) Annie 20:30, 20 June 2018 (EDT)

Fixer Public

Hello, would you look at this and if you like it, we can link it from the main Public page and open that for editors that are bored and would like to help with these. Feel free to edit/change/add and so on. Annie 17:58, 19 June 2018 (EDT)

Thanks, I'll take a look! Ahasuerus 18:22, 19 June 2018 (EDT)

The Infernal ISBNs

I suspect that the 5 books that showed up here today are because of how Fixer pulls future date books from Amazon UK and it seems like he cannot get the correct form of the ISBN for some reason (there had been a few almost daily lately so I tracked down the submissions - they came from you and from the look of them, these are Fixer's). Can you have a word with him? PS: In case someone cleans them before you can see them, here are the ISBNs:

• 0356508587
• 0356509389
• 1405291338
• 147321341X
• 1473217679

Thanks. :) Annie 02:07, 27 June 2018 (EDT)

OK, I see what the problem is. Fixer has had problems with UK ISBNs since the last round of Amazon's API changes. Much of the time he gets the core data from Amazon.com and then I have to correct various fields manually. It's not too bad when the two stores have essentially the same information, but the last month has been problematic with dozens of cancelled ISBNs and other discrepancies. The result was that Fixer was submitting many 0000-00-00 dates. And since Fixer's logic says that pre-2007 books should be submitted with ISBN-10s... I'll have a chat with Fixer and explain to him that "0000-00-00" is not really "pre-2007".

Thanks for letting me know! Ahasuerus 09:25, 27 June 2018 (EDT)

Ah, 0000-00-00 dates - that makes sense. I usually do not mind just fixing these when they show up but as it looked like a pattern from someone lately, I went investigating :) Thanks for figuring it out! Annie 13:21, 27 June 2018 (EDT)

Mail received

Time stamp is 7:12 my time ( 10:12 EDT), I am just not up that early on saturdays. :) So looks like I got it as soon as you sent it. Of course Yahoo decided it is a Junk mail but that is nothing new with test mails. Annie

Thanks! So far we know that:

• I can apparently email other people successfully
• I am unable to send e-mail messages to my own account or to exchange messages between my main account and Fixer's account (they are both email.com accounts)
• Hauck's report that he had sent ISFDB email to me, which I never received

Let's see if Marty gets my email. If he does, then we can hypothesize that I can send ISFDB email, but I can't receive it. If so, then the next step would be to determine if it's a problem with my account settings, email.com or something else. Depending on the answer, the problem may be limited to my account or it may affect multiple ISFDB accounts. (I have double-checked to make sure that I have "Enable e-mail from other users" checked on both accounts.) Ahasuerus 13:33, 30 June 2018 (EDT)

Do you want me to try to send an email to you? Annie 13:38, 30 June 2018 (EDT)

Good point, please do! Ahasuerus 14:43, 30 June 2018 (EDT)

Done. Annie 15:21, 30 June 2018 (EDT)

Thanks! It's been 15 minutes and there is still no sign of the wayward email.

I get e-mail updates from SourceForge, Amazon and third parties all the time, so it's presumably something between the ISFDB server and email.com (and possibly other places.) Ahasuerus 15:36, 30 June 2018 (EDT)

We may be looking at two separate issues here - some servers do not allow the sending; some do not allow the receiving. After your mail, I sent one to myself (I've never used the system to send an email - I had only received through it) - and that one is nowhere to be seen either (neither are the copies I requested for both). Considering that we are sending from the name of the user (and their email), it looks like some servers (yahoo seems to be one) does not like when someone pretends to be sending from one of its mails. We know that email.com allows the sending (I got yours) but it does not seem to like the receiving part (headers don't match the from address would be my guess).

As Marty mentioned that he had both received and sent via the system, he may be able to send to you and we will see if email.com just does not like both directions. It sounds like we are having the usual set of mail relay issues (I deal with that in my daily job and relays had been trickier in the last years (and mail servers had been tightening their security gradually)). I do not know what the system we have is but one option may be to switch to sending from an address that belongs to ISFDB and put the sender's mail in the 'cc'... But if we have only a relay, that won't work unless we find a server we can get he relay to send through. Annie 16:57, 30 June 2018 (EDT)

Fascinating! Not in a good way, but fascinating nonetheless :-) I guess we'll know more once we hear from Marty.

In the meantime, I wonder if there may be something on the ISFDB end which doesn't like e-mails with the same sending and receiving address. Ahasuerus

That would have made this checkbox "send me a copy of my mail" useless. But everything is possible. :) Annie 17:27, 30 June 2018 (EDT)

Hm. Perhaps our e-mail server only objects to the same e-mail address in the "To:" field but not in the "Cc:" field? Pure speculation, of course... Ahasuerus 17:45, 30 June 2018 (EDT)

(unindent) Some more investigating :) I went looking at the mails I had received through the system - one specially set server and the rest were gmail addresses. I happen to have an gmail address so I changed mine here to it and did some more testing:

• Confirmation mail from ISFDB arrived imediatelly
• A mail from myself to myself arrived as well - so the system has no issue with the same addresses.
• I just sent a mail to you - can you check if it arrived?
• I did not get a copy of my mail to myself but I got a copy of the mail I sent to you.
• Gmail marked all those mails as junk (probably because the headers do not match the from they are coming from) except for the confirmation mail which came from an isfdb.org address.

I'll stay here with my gmail address I think for now until we get to the bottom of all that but if you want me to switch back to yahoo for testing, it is easy enough. Annie 18:21, 30 June 2018 (EDT)

Thanks for looking into this! You latest email was apparently lost in the ether as well. I should probably experiment with other email services too: gmail.com, outlook.com, etc. Ahasuerus 18:32, 30 June 2018 (EDT)

Sounds like email.com really does not like a relay that pretends to be sending mails with from addresses not matching the relay address... :) Annie 18:38, 30 June 2018 (EDT)

I am sure they mean well :-\ Ahasuerus 19:19, 30 June 2018 (EDT)

Late to the party, I got mine at 10:12am. That was yahoo. The editor I had the exchange with was using gmail. He sent me ISFDB mail successfully, and I used ISFDB mail to send to him successfully. I delete religiously, though, and no longer have the correspondence to see when that was, but I'm almost certain it was in June. --MartyD 07:23, 1 July 2018 (EDT)

p.s. The problem is the way the email is set up. I can see in the message properties that a scanner thinks it's likely spam because the sending domain and the from domain are different (and, presumably, there's no spf record allowing the mix). Looks like sourceforge is doing the filtering, although maybe that's because my ISFDB email is my sourceforge address. Here's a snippet from the message header (I've XXXed at signs):

X-Mailer: MediaWiki mailer
From: Ahasuerus <ahasuerusXXXemail.com>
X-Spam-Score: 1.5 (+)
X-Spam-Report: Spam Filtering performed by mx.sourceforge.net.
 See http://spamassassin.org/tag/ for more details.
  0.0 FREEMAIL_FROM          Sender email is commonly abused enduser mail provider
                             (ahasuerus[at]email.com)
  0.2 HEADER_FROM_DIFFERENT_DOMAINS From and EnvelopeFrom 2nd level mail
                             domains are different
  0.2 FREEMAIL_FORGED_FROMDOMAIN 2nd level domains in From and EnvelopeFrom
                              freemail headers are different
  1.0 RDNS_NONE              Delivered to internal network by a host with no rDNS
X-Headers-End: 1fZGbU-00Dvn2-1V
Content-Length: 128

I can't right now, but later I will try it out with my accounts and see if the behavior is any different. --MartyD 07:31, 1 July 2018 (EDT)

Thanks, Marty! Now that we know what the underlying problem is, I will post an update on the Community Portal and then we can think of the best way to address it. Ahasuerus 15:24, 1 July 2018 (EDT)

(unindent) OK, so now that we know what's causing the issue, we need to come up with a viable way to fix it. We are using MediaWiki e-mail software, so there are limits to what we can do (easily.)

The first thing that comes to mind is $wgUserEmailUseReplyTo, which was introduced in 1.12.0. Our version is 1.12.0rc1, which presumably means that it supports $wgUserEmailUseReplyTo (Edit: apparently it is included.) I don't have MediaWiki installed on the development server, so I can't test it, but it should be easy to add this setting to the live server's configuration file, send a test message and then disable the setting until we can confirm that it's doing what I hope it should be doing.

What do you think? Will it resolve the problem?

P.S. We really need to upgrade MediaWiki anyway because of all the security fixes in later versions. It's been on Al's plate for many months, but he is effectively unavailable. I'll need to install the software on the development server and see if I can get it to run. Ahasuerus 18:27, 1 July 2018 (EDT)

It may help for some servers - and some may decide that they still not like it. It cannot get worse anyway so we can at least try to see if it unblocks at least some more servers... If that does not help, we may need to look into using one of the isfdb.org mail addresses as a from address... Annie 18:54, 1 July 2018 (EDT)

Right. If the sending server is not something the "from" domain authorizes as a sender, there is a good chance the email will be blocked no matter what configuration is used. It looks to me like the ISFDB's MediaWiki should be configured to send mail from an general isfdb.org address, and $wgUserEmailUseReplyTo could be turned on so that the replies will go to the ISFDB user's configured email. That's only available in 1.12.0 and later. The other thing you should do is set up an SPF record for isfdb.org. Some servers won't accept mail from anything not having a valid SPF record. Let me know if you need help. --MartyD 07:36, 2 July 2018 (EDT)

I added $wgUserEmailUseReplyTo at 10:32 EDT and sent test messages to the two of you and to myself. I then commented it out pending investigation results.

On the plus side, it didn't seem to impact Wiki behavior. On the flip side, I am not seeing anything in my Inbox. I wonder what SourceForge will have to say about the latest e-mail.

Re: setting up an SPF record for isfdb.org, I believe SPF records are part of DNS. Only Al has access to our DNS settings. I can forward recommendations to him, but otherwise it's out of my hands. Ahasuerus 10:45, 2 July 2018 (EDT)

Well - it worked - the mail of the user is nowhere to be seen as expected, the reply to also pulled it from the message. It may be confusing for someone that does not understand that from and replyto can be different. Annie 10:55, 2 July 2018 (EDT)

Progress is good! Once we get Marty's analysis of the SourceForge headers, we can decide whether the potential for confusion ("From" vs. "Reply-To") is an acceptable trade-off for increasing the universe of servers accepting ISFDB e-mail.

For now, I have changed my ISFDB e-mail address from email.com to sourceforge.net, which is forwarded to email.com. I have confirmed that it works with email.com just like it works for Marty. I see that SourceForge says that "From and EnvelopeFrom 2nd level mail domains are different", but that's about it. Ahasuerus 11:31, 2 July 2018 (EDT)

Marty has confirmed the receipt of the test e-mail. The "Reply-To" settings are apparently OK after forwarding.

Too bad the three servers that we have tested behaved the same way "before" and "after". I wish we could find an e-mail server which would drop e-mails sent using the old method and accept e-mails sent using the new method.

I am going to re-activate the new setting and do some more testing. Ahasuerus 14:46, 2 July 2018 (EDT)

Different reasons for erroring-out?

I had a submission error out today, partway through contents, which didn't really surprise me because it had 80+ content titles; I'm used to long submissions not all going through. But last week this one errored out in the middle of adding contents, with only 20 content titles. Maybe that was because there was a html error (now fixed!) in the notes section: </ul> without <ul>. Could that cause a delayed error-out partway through the contents? --Vasha (cazadora de tildes) 16:08, 1 July 2018 (EDT)

Well, "never say never", but based on what I know about the submission approval process, it seems unlikely. Let me restore the latest backup file on the development server and see if I can recreate the problem. Ahasuerus 16:14, 1 July 2018 (EDT)

Just as a data point - I fix at last 2 of those "missing /ul" thingies (either altogether missing, or using /li or using just ul) pretty much daily - either while moderating or via the Cleanup reports so I doubt that it is causing any issues like this. Annie 16:50, 1 July 2018 (EDT)

It was the open-tag not the close-tag that was missing but yeah, I'm sure that happens a lot too. --Vasha (cazadora de tildes) 16:54, 1 July 2018 (EDT)

Ah, yes. Not as often but those pop up now and then as well. :) Annie 17:04, 1 July 2018 (EDT)

(unindent) After restoring the backup file on the development server, I re-approved the "ul" submission and it went through successfully :-\ Checking submission history, I come up with the following statistics starting with 2018-06 and going back in time:

• June: 45,324 submissions, 5 errored out
• May: 42,480 submissions, 5 errored out
• April: 39,524 submissions, 1 errored out
• March: 36,623 submissions, 4 errored out
• February: 26,828 submissions, 1 errored out
• January: 28,919 submissions, 1 errored out
• December: 32,698 submissions, 0 errored out
• November: 30,376 submissions, 3 errored out
• October: 24,341 submissions, 2 errored out
• September: 25,101 submissions, 0 errored out
• August: 27,607 submissions, 1 errored out
• July: 33,093 submissions, 2 errored out
• June: 33,794 submissions, 3 errored out
• May: 33,682 submissions, 5 errored out
• April: 43,470 submissions, 4 errored out
• March: 35,632 submissions, 7 errored out
• February: 33,914 submissions, 9 errored out

I am not seeing a conclusive pattern, which, in a way, is a good thing. It suggests that whatever is causing the errors is probably not related to the software changes which have been implemented over the last year. I am not sure if the fact that the last two months have been busier than the corresponding 2 months in 2017 is significant, but we are definitely getting more submissions:

+----------------+----------+
|           2006 |    30742 |
|           2007 |   150208 |
|           2008 |   171709 |
|           2009 |   215003 |
|           2010 |   211926 |
|           2011 |   248834 |
|           2012 |   298663 |
|           2013 |   235530 |
|           2014 |   312831 |
|           2015 |   308074 |
|           2016 |   370596 |
|           2017 |   390945 |
|           2018 |   439638 (based on the first 6 months) |
+----------------+----------+

In any event, the frustrating part is that many errored out submissions are quite simple, e.g. some add short title notes. I can't think of any reason for them to error out and yet they do :-( Ahasuerus 17:06, 1 July 2018 (EDT)

Advanced Title Search Results and Tags

I don't know if it's always been there and i just haven't run into this before or if tags is a recent addition to advanced title search results. -- JLaTondre (talk) 21:00, 2 July 2018 (EDT)

Sorry, I forgot to answer the first question. Tags were added to Advanced Title Search back when tags were revamped, which was a number of years ago. Ahasuerus 22:32, 2 July 2018 (EDT)

However, when there are a lot of tags (example), it makes the results user unfriendly as the more important fields get compressed on anything but a very wide screen. Could tags be made an option, only a limited number displayed, smaller text used for them, or something else creative? -- JLaTondre (talk) 21:00, 2 July 2018 (EDT)

Hm, I see. The first thing that comes to mind is limiting the number of tags to something modest like 5 and indicating that there are more. We already use similar logic on Summary pages, so it would be consistent. Do you think it would address the issue? Ahasuerus 21:22, 2 July 2018 (EDT)

I think that would probably improve the look of the list. --Vasha (cazadora de tildes) 22:08, 2 July 2018 (EDT)

I would rather have an option that allows me to remove them from the view - even with just 5, they will still take most of the screen and squish the columns I am actually trying to look at... Annie 23:02, 2 July 2018 (EDT)

We could certainly create a User Preference to suppress the display of tags in Advanced Title Search.

I would suggest re-posting this question on the Community Portal. Perhaps other editors may have additional ideas.

P.S. In a way, our software supports two overlapping audiences: people who are looking for "stuff to read" and people who are trying to improve our bibliographic records. In certain cases they may have somewhat different needs and priorities. I know that I find tags more useful when wearing the first hat and less useful when wearing the second hat. Ahasuerus 10:47, 3 July 2018 (EDT)

Will do. In a way they are the opposite to the bibliographic warnings in that regard. :) Annie 12:33, 3 July 2018 (EDT)

Poe: Poetry and Tales

Hi--could you please approve this submission so that I can go ahead with everything I need to do to it (importing and removing titles, varianting newly-created titles, etc.)? Thanks! --Vasha (cazadora de tildes) 14:10, 5 July 2018 (EDT)

I have approved the submission and rearranged the sentences which talk about dates. I also changed "printing date" to "publication date" -- the former refers to the date when the book was finished by the printers, which can be a few months prior to the official publication date, i.e. the date when the book was made available to the public. Hopefully I didn't mess anything up! Ahasuerus 14:53, 5 July 2018 (EDT)

After sleeping on it I realized that you probably meant "the date of the ninth printing". Sorry, I had SFBC on my mind at the time I approved the submission and SFBC is notorious because we often know the printing/manufacturing date (see Gutter code) but not the publication date. Leave it to publishers to use the term "printing" to mean two different things :-\ Ahasuerus 11:37, 6 July 2018 (EDT)

Actually I did appreciate the reminder of the difference between printing & publication. :) Thanks for improving my wording --Vasha (cazadora de tildes) 13:17, 6 July 2018 (EDT)

Need a hand?

I know that it takes time for Fixer to get an ISBN list out but if you want to get some of the newer ISBNs offloaded with little or no processing, I can get some of them in this weekend. (if not, I will pull some lists from the public list and work on them). Annie 14:20, 6 July 2018 (EDT)

Thanks for the offer, appreciate it! In addition, it may be a good dry run for Fixer's "public" process which we have been working toward. Let me see if I can sort out the 83 June ISBNs in Fixer's "Queue 1"... Ahasuerus 15:05, 6 July 2018 (EDT)

Well, it will be similar to what we did last year so no surprises expected but yeah :) Plus it may remind me steps I may have been forgetting when I was trying to write. Annie 15:12, 6 July 2018 (EDT)

75 June ISBNs ready for action! Thanks again. Ahasuerus 15:13, 6 July 2018 (EDT)

Thanks! Will get them cleared over the weekend. Annie 15:14, 6 July 2018 (EDT)

P.S. And if you run out of ISBNs, Fixer will be happy to share more of them :-) Ahasuerus 15:33, 6 July 2018 (EDT)

Will do - I need to clean the remaining ISBNs on my page as well (and a new run through the Deferred submissions is overdue (based on my list; will reconcile with the big one under Fixer at some point as well). Annie 16:47, 6 July 2018 (EDT)

Special characters and Gae͏̈tan Dorémus

Hi,

Can you look at this. Any ideas what is going on (besides our usual awesome handling of non-standard characters)? Annie 15:34, 6 July 2018 (EDT)

Help pages cleanup

This whole subsection needs fixing - we just spent the better part of the year getting rid of these links, teaching people to add more of them looks a bit counterproductive :) Maybe we should leave just the Gutenberg example at the bottom? Or do you have any ideas on what other examples we can add? :) Annie 15:50, 11 July 2018 (EDT)

An excellent point! I have deleted OCLC, LCCN and BLIC; Project Gutenberg is the sole survivor (for now). Thanks! Ahasuerus 16:08, 11 July 2018 (EDT)

While we are on this: also needs massaging - the three statmensts for LCCN, OCLC and BLIC. Maybe we should instead tell people that for these links, they need to use the templates and/or external identifiers? And in the same section, the link to "Linking Examples" needs fixing now that you changed the name :) Annie 16:24, 11 July 2018 (EDT)

Done! Ahasuerus 17:58, 11 July 2018 (EDT)

Why even keep the Project Gutenberg one? The software already autolinks to the record via the catalog number. -- JLaTondre (talk) 19:17, 11 July 2018 (EDT)

Trucidated with extreme prejudice! Ahasuerus 23:44, 11 July 2018 (EDT)

Implementing tables in BBCodes

Currently, the supported HTML allows some attributes to be added to tables. Here is an example of someone finding a use for a table in notes & implementing it just about as simply as possible, but they still used "border=1" and "align=right." And here is one with no borders but using "width," "align," and "colspan." I really can't imagine that tables with no attributes at all will be useful; and yet the most basic form of BBCodes doesn't allow them (I am pretty sure?) When you transition to BBCodes, will it be possible to use an expanded version (I know there are several) which allows table attributes? --Vasha (cazadora de tildes) 14:16, 12 July 2018 (EDT)

Perfect timing! I am currently working on the next phase of security enhancements, specifically the BBCodes migration. I expect that I will post the results of my preliminary investigation later today or tomorrow, but for now let me address HTML/BBCodes tables.

As I wrote on Doug's Talk page the other day, "if we decide that our default tables should have borders between cells, we can do it in less than an hour". All ISFDB notes are displayed by the same software module, so it would require a single change to the parsing/massaging logic which handles template. Changing the default behavior of other BBCodes/HTML elements should be equally easy.

OTOH, adding support for multiple HTML attributes like "align", "width" and "colspan" to the ISFDB version of BBCodes would be a much bigger project. At this time we have 229 attribute-less HTML tables in notes and 11 tables with attributes, so I don't think it's a high priority. We could easily create custom BBCodes like "table2", which would be displayed as table colspan="2", "table3", etc.

Does this make sense? Ahasuerus 15:36, 12 July 2018 (EDT)

Yes... I think the way people have been using tables, we could live without any attributes except the ability to choose whether to have borders or not. Maybe create two versions of the table codes, one bordered & one not? --Vasha (cazadora de tildes) 15:55, 12 July 2018 (EDT)

I guess we could keep the default behavior of [table] (i.e. no borders) and add something like [tableb] which would include borders. It's probably best to discuss it on the Community Portal once I post the results of my investigation. Ahasuerus 16:04, 12 July 2018 (EDT)

More languages to add

The magazine Samovar has published stories in Nepali and Pashto. --Vasha (cazadora de tildes) 14:43, 12 July 2018 (EDT)

Done. Ahasuerus 16:08, 12 July 2018 (EDT)

What is this book?

It looks like Russian to me. Can you help me out? ···日本穣 · 投稿 · Talk to Nihonjoe 17:12, 12 July 2018 (EDT)

It is this one. It is Russian indeed, the first book edition of 3 of the "Alice the Girl from Earth" stories (one of them long enough to be a novel). We even have it in the DB (I am out of the door for an appointment but I will fix our record when I am back (adding more details and so on)) :) Annie 17:28, 12 July 2018 (EDT)

Related: we should make sure all of these are in the database. I don't think they all are. Since I don't know Russian, I'm probably not the best one to enter them. :) ···日本穣 · 投稿 · Talk to Nihonjoe 18:55, 27 July 2018 (EDT)

I am sure we'll get to it, but he was extremely prolific, so it will take some time.

Our coverage of Russian (as well as Polish, Hungarian etc) SF is rather limited at the moment, but at least the software supports non-Latin scripts now. We just need to recruit more editors who would be in a position to enter the data. I hear candy works well :-) Ahasuerus 19:54, 27 July 2018 (EDT)

I have the Gusliar stories on my list of things to sort out anyway, may as well work through the rest of his works when I get around to them (probably won't be very soon all things considered). Annie 20:03, 27 July 2018 (EDT)

Question about Fixer/Public: multiple magazine formats

I will be glad to take some of those ISBNs off the page. I have been trying to double-check new additions of 2018 anthologies, collections, and magazines anyhow, and not infrequently find more info needing adding or improving, so I might as well do the intial submission for those myself. --Vasha (cazadora de tildes) 14:32, 18 July 2018 (EDT)

Thanks! Ahasuerus 15:28, 18 July 2018 (EDT)

I do have one question though. What do we do when Fixer submits a magazine that's already in the database in another format? --Vasha (cazadora de tildes) 14:32, 18 July 2018 (EDT)

It's kind of a long story. The original design did not envision multiple versions of the same magazine issue. After all, magazine issues couldn't possibly have more than one edition or printing, right? Of course, the best laid plans of mice and men and all that...

The first time we ran into this issue was with simultaneous magazine reprints, usually US/UK or US/Canada. We got around this problem by creating separate country-specific EDITOR series.

The second time reality had an unpleasant surprise in store for us was when small presses and POD publishers began reprinting pulp magazines in facsimile editions. We decided that they were really anthologies rather than magazines and treated them as a separate series.

Finally, ebooks became popular; many magazines either switched to e-publication or began running two versions in parallel. There was no easy solution to this conundrum, so we reluctantly agreed that yes, it was possible for a magazine issue to have 2+ "editions", e.g. see the 2014 row in this magazine grid or the 2017 row in this one.

The software still doesn't fully support this functionality. If you try to clone an existing MAGAZINE/FANZINE pub, you'll get an error. However, you can create a bare bones NewPub submission and then use Import Contents to get the Contents titles over. A bit of a kludge, but it works.

Come to think of it, we have had FR 745, "Allow cloning MAGAZINE publications", for a couple of years. Now may be a good time to implement it. Ahasuerus 15:28, 18 July 2018 (EDT)

FR 745 has been implemented -- see the Community Portal announcement. Ahasuerus 19:48, 18 July 2018 (EDT)

I fixed Fixer's Public instructions to reflect the change. Annie 19:55, 18 July 2018 (EDT)

Thanks! Ahasuerus 19:58, 18 July 2018 (EDT)

Especially now that we can have 3 different versions of a magazine (looking at Clarksworld for example that has a printed, an ebook and an web version), we really should decide how we handle these. We probably should also look into a way to connect different series - because it is not very easy to find if we have different formats of the same issue - which we now do in separate series or do something with the grid that allows the different formats to live under one umbrella in an easier way... :) Annie 15:33, 18 July 2018 (EDT)

Having separate EDITOR series for different formats is just a pain in the tuchus. They are always falling out of synch with each other, users may not know that both of them exist, it is not easy to see both of them at the same time, and they create a duplication on the editors' summary pages.

But we already do sometimes have multiple formats in the same EDITOR series, as you pointed out. To me, it seems like that works fine. The only thing that would be needed would be to add an indication of the format to the grid display. Any other objections? --Vasha (cazadora de tildes) 16:09, 18 July 2018 (EDT)

It is the sometimes - we do that mostly when one of the format is used rarely; for most magazines that always have both formats we go for double series. :) As always, the explanation is "history" and having the EDITORs the way they work for magazines does make sense but we need to expand on it. Plus having both series on Neil Clarke's page for example just look weird.

In my ideal world, the grid shows the format in brackets after whatever is there AND we have a way to filter by format. Annie 16:16, 18 July 2018 (EDT)

It would be easy to append the format to each issue. However, wouldn't it be overkill when viewing paper-only magazines like Astounding / Analog (1937-1971)? Ahasuerus 16:43, 18 July 2018 (EDT)

Well, the magazine series could have an invisible checkbox that indicates "multiple formats, needs to be displayed with format in the grid" --Vasha (cazadora de tildes) 16:47, 18 July 2018 (EDT)

And we will forget to change it when there is a different one now. Plus it will make the grid inconsistent between magazines. :) Annie 16:50, 18 July 2018 (EDT)

Well, they changed format from bedsheet to digest in the middle of the run. Which you cannot see anywhere unless you go through the issues one by one and just notice it - I did not know until I actually got an old issue and it was the wrong format and went researching it. :) Annie 16:50, 18 July 2018 (EDT)

(unindent) I can think of a couple of viable options:

• Display the format only if there are multiple formats, e.g. Astounding's format change from pulp to digest to bedsheet to digest
• Display the format only there is a mix of paper and non-paper format: the Astounding grid won't display formats, but Clarksworld's grid will.

Something to discuss on the Community Portal, perhaps. Ahasuerus 17:46, 18 July 2018 (EDT)

Agree - we probably should open a discussion somewhere that is not your page :) Annie 18:02, 18 July 2018 (EDT)

Done. Annie 20:05, 18 July 2018 (EDT)

Essay-fiction mismatch cleanup report

JLatondre tried to show me something in this report and found that it was moderator-only, although I don't see any reason it should be. Are there any other reports set to moderators-only by default that other people might want to see sometimes? --Vasha (cazadora de tildes) 20:54, 23 July 2018 (EDT)

Originally, all cleanup reports were moderator-only. A few years ago many of them were made available to all editors. The reason that some remain moderator-only is that they support the ability to "ignore" records, which is limited to moderators. It is possible to make an "ignore-enabled" report suppress the display of the "ignore" option when viewed by a non-moderator, but it requires special coding. Since we have well over 200 cleanup reports, I've only done it for active or explicitly requested reports.

Requests accepted! Get your very own shiny cleanup report for a low low price of just two quatloos! :-) Ahasuerus 22:40, 23 July 2018 (EDT)

OK, well, I am still working on Cuentos breves y extraordinarios and trying to figure out which contents should be essays, so this problem might come up again. But no need to recode the report just for this one project. --Vasha (cazadora de tildes) 23:19, 23 July 2018 (EDT)

Vasha77, it's the "Variant Title Type Mismatches" report. It reports any mismatch, not just essay / fiction. It doesn't have an ignore option (since they should always match).

Ahasuerus, your post implies you have opened it up, but it doesn't have an asterisk?. -- JLaTondre (talk) 07:52, 24 July 2018 (EDT)

Actually, "Variant Title Type Mismatches" supports the ability to ignore mismatches, but it's rarely used. At the moment there are only 27 "ignored" mismatches.

As I recall, the "ignore" functionality was added primarily due to a request to allow exceptions for certain types of translations, e.g.:

• L'homme venu du futur and L'échiquier fabuleux, NOVEL translations of the novella The Fairy Chessmen
• Un monde magique, a NOVEL translation of the collection The Dying Earth

We have only 9 exceptions of this type, so it hasn't really caught on. The rest of the exceptions are more questionable. Some appear to be data entry errors like this INTERIORART variant of an ESSAY. Others are intentional, but may not be optimal like this record which says:

• The two versions of this title are the same, but the 1997 version is presented as a "true encounter with the paranormal", hence listed as non-fiction, while its later appearance is presented as a fictional story.

It may be worth reopening the issue of type mismatches on the Community Portal. I can create a list of the 27 mismatches if needed. Ahasuerus 11:11, 24 July 2018 (EDT)

I think you are right - sometimes things get ignored by mistake (I tend to ignore on one screen while having the same report open in another tab so that if I click on the wrong thing, I can actually see it again and deal with it) and sometimes things get ignored just because someone wants the report dealt with. Annie 14:17, 24 July 2018 (EDT)

Done! Ahasuerus 15:29, 25 July 2018 (EDT)

Unicode Character Issue

This title is showing up on both the Titles with Invalid Unicode Characters and Publication Title-Reference Title Mismatches reports. I copied the title to a on-line Unicode checker, but it didn't have an issue with it. The publication & title records both have the same content. So not exactly sure how to solve either one... Thanks. -- JLaTondre (talk) 16:48, 26 July 2018 (EDT)

Investigating... Ahasuerus 16:57, 26 July 2018 (EDT)

There was an invisible "Left-To-Right character" embedded in these two records (title and pub.) The most recent change to the input filter added this character to the list of Unicode characters which we automatically strip at input time. The two cleanup reports use the list of characters leveraged by the filter to search for anything that may have slipped past it. I have removed the offending character from the two records, so everything should be back to normal :) Ahasuerus 17:10, 26 July 2018 (EDT)

Thanks! -- JLaTondre (talk) 17:13, 26 July 2018 (EDT)

Accented author names and pseudonyms

Hi. Either my brain has melted due to the heat wave here and I've forgotten about this topic, or I've actually never come across an edit like this so far. And I'm not sure if this is a bug or feature. According to Template:PublicationFields:Author: "Accented characters. If you are entering a name such as "Philip José Farmer" that is printed with an accented e, that accented character should be reproduced in your entry of the name. Two versions of an author's name that are printed with and without accents are treated as variants". I understand that "as variants" means "as pseudonyms", and I just tried to create such a variant for this title record by changing it's author to "Michal Hvorecky". However, the submission contains "Michal Hvorecký" again afterwards and nothing has changed. Therefore no variant/pseudonym can be created. Is this a bug? Or is the help out-of-date? Jens Hitspacebar 18:13, 28 July 2018 (EDT)

I see that you have uncovered one of the ISFDB's more shameful secrets :-\ Let me start by quoting something that I wrote a couple of years ago:

The way the ISFDB database works, there are three types of letters derived from the original Latin alphabet:

• English letters.
• Latin-1 (aka ISO 8859-1) letters, which include non-English characters used in many European languages like German, Danish, Swedish, Spanish and French (except for œ) -- see the bottom part of this table for a complete list of covered characters.
• Other Latin-derived letters used by Polish, Czech, Romanian, Hungarian, etc.

The reason this distinction is important is because our database treats these types of characters differently when running searches. Let's compare Philip José Farmer and Stanisław W. Czarnecki. They both have non-English characters, "é" and "ł" respectively, in their names. If you enter "philip jose farmer" (note the use of the English "e" instead of the accented "é") in the regular Search box and click "Go", you will be redirected to Farmer's Summary page. If, on the other hand, you enter "stanislaw w. czarnecki" (note the use of the regular English "l"), your search won't find anything.

The reason for this behavior is that "é" is part of Latin-1, so the database "knows" that it is related to "e". It is similar to the way the database handles uppercase and lowercase characters during searches: even if you enter an all-lowercase or an all-uppercase search string, the database will still find the right records because it "knows" that A/a, B/b, etc are the same for search purposes. The Polish letter "ł", on the other hand, is not a part of Latin-1 and the database doesn't "know" that it is related to "l".

(end of quote)

Hopefully this provides enough background information to explain how we are forced to handle these cases. Here is a 2016 discussion which covers this topic -- the most relevant parts start with MartyD's comment. Hope this explains things! (Sorry the software hasn't been improved yet -- it's a fairly big can of worms.) Ahasuerus 18:55, 28 July 2018 (EDT)

Ah, yes, thanks. Now I vaguely remember having read the discussion you quoted from back then. I wasn't aware that this problem isn't fully resolved yet. Jens Hitspacebar 19:10, 28 July 2018 (EDT)

Ultimately we will want to convert the database to native Unicode. It will help address all kinds of issues, including this one. Easier said than done, though... Ahasuerus 21:33, 28 July 2018 (EDT)

Ben Shecter, Author merge

Hi. Thanks for your note User talk:Pwendt#Ben Shecter. I suppose you do too much of that to be able to watch every user talk page.

Concerning the fix for Shecter and the mis-spelled Shechter (my recent error, not in the publication), my understanding is that I should only now --after manual merge of the author data including Notes-- correct the spelling, in publication The Mummy Market or INTERIORART, equivalently because "Shechter" has only the one publication and one title. And for the same reason, not to be corrected earlier, because author-name and publisher-name records are deleted when the those names appear in zero publication records (I understand from Mhhutchins). Deletion may be human-assisted, rather than automatic, but the procedure is desirable to save assistant time, or assistant's overlooking biographical information.

Now I submit no such correction, but only the MakeVariant request for The Mommy Market by Ben Shecter and Mummy/Shechter submission. --Pwendt|talk 12:48, 1 August 2018 (EDT)

That's right. The following records are automatically deleted when the last title/publication record associated with them is deleted:

• authors
• publishers
• publication series

Publication-less titles and title-less series are not deleted automatically. We have cleanup reports which can be used to delete them manually.

I have approved your Make Variant submission and corrected the typo in the INTERIORART record, so the "Shechter" form of this person's name is now gone. I have also moved Ben Shecter's date of birth to the Note field and gave Lenona's post as the source. It will have to do until we find another source. Her posts are generally reliable, but she is a compiler, not a primary source, so we have to be careful.

Thanks for working on this artist! Ahasuerus 14:35, 1 August 2018 (EDT)

The Pumpkin Man

In response to your question on my talk, page I found the text to The Pumpkin Man and the short introductory paragraph states that the short story is the insperation for the novel of the same name. Question now answered. MLB 21:45, 1 August 2018 (EDT)

Updated, thanks! Ahasuerus 21:49, 1 August 2018 (EDT)

Error

I had a submission error out halfway through a content record; it created a new author record but no title record to go with it. As I understand it, author records with no titles should be automatically deleted, so there's presumably something (a half-completed title record?) you need to clean up there. --Vasha (cazadora de tildes) 16:32, 6 August 2018 (EDT)

Thanks for letting me know! I usually clean up these "naked" author records by adding a dummy title and then deleting it. It's gone now. Ahasuerus 17:02, 6 August 2018 (EDT)

Dragon Award new category

Will you add "Best Media Tie-In Novel" as a category to the Dragon Award? It's a new category this year. Thanks! ···日本穣 · 投稿 · Talk to Nihonjoe 19:33, 7 August 2018 (EDT)

Done! Also, keep in mind that only the ability to create new award types is restricted to bureaucrats. Award categories can be added/edited by all users. Ahasuerus 20:26, 7 August 2018 (EDT)

Hmmm...guess I haven't figured out how to do that yet. ···日本穣 · 投稿 · Talk to Nihonjoe 20:31, 7 August 2018 (EDT)

Aha, I see the link now. I guess I've just never looked over there before on award pages. ···日本穣 · 投稿 · Talk to Nihonjoe 20:32, 7 August 2018 (EDT)

Miss Pickerell and Archway maple leaf logo

Hi. I wonder whether you recently removed a cover image from our record of the Miss Pickerell stories in its Archway / Pocket Books edition. It was problematic, and I think you had questioned something as one verifier of the record.

As of last week ISFDB provided wrong cover images for some Archway books; in particular, maple leaf covers (Canada issue? later printing? both?) for some ordinary (US 1st printing?) records, including at least one Miss Pickerell iirc. ...

Anyway, regardless whether I recall the details correctly, I do have an outstanding mental note to tell you my inference about Archway / Pocket Books covers, maple leaf and otherwise. For instance, concerning Miss Pickerell on the Moon:

Amazon CANADA [2] --with image of US cover probably!

Amazon US [3] --with image of Canada cover probably! [URL corrected 2018-08-15]

(Those two product pages are defined by the same ISBN, at two different Amazon domains; two among several ISFDB provides in the left margin of the relevant publication record P273387.

Cover images provided there now, I infer, are those of the US and Canada issue, 1st ed. indicated by [ISBN]Catalog # in the [0-671-]560 sequence. No maple leaf (at the Canada site!) is the US at price $1.75. Maple leaf (at the US site!) is the Canada at price C$1.95. So I infer, but the maple leaf logo is too small for me to read in this cover image and others. --Pwendt|talk 17:57, 14 August 2018 (EDT)

Thanks for your attention on other matters. Seeing a clerical error so gross as the mistaken-duplicate URL, I can't help myself tinkering with this message :--(

On other matters I can't decide where to start. So I don't, now. I have 10/11-day August vacation staring me down. And I am troubled by some of the too-much-reading I did in ISFDB Wiki community pages this weekend, during the submission backlog. --Pwendt|talk 19:18, 15 August 2018 (EDT)

Charles Gerard Winstanley Bancks

For what it's worth, I composed several versions of the second and less important "paragraph" of the Note for this Author 278679 (one you approved), before cutting it to a bare minimum or beyond. If I were writing complete sentences, I think now I would write at least something like this, but I doubt that I could stop with this:

"Full name may be Charles Gerard Winstanley Bancks. The Library of Congress lists that variant, and cites "OCLC", perhaps from its use by some WorldCat participating libraries as the canonical name for this writer."

(Perhaps no WorldCat library reports a publication credited to Charles Gerard Winstanley Bancks (nor to Gerard Winstanley Bancks). For instance, as point of entry: [4]; [5]. Don't pursue either point of entry. I am only letting you know that I know them, if you know what I mean.)

This note barely touches some matters you have raised explicitly, or have indicated. But I want to get one substantive item on the board now, before vacation. --Pwendt|talk 20:50, 15 August 2018 (EDT)

Thanks for the pointers! I have found his biography in John Venn's "Alumni Cantabrigienses: A Biographical List of All Known Students, Graduates and Holders of Office at the University of Cambridge, from the Earliest Times to 1900, Volume 2", Cambridge University Press, 2011, page 139, and updated the record. Ahasuerus 10:35, 16 August 2018 (EDT)

Russian science fiction mystery

In one of the issues of Kasma there is a translation of a story by Sviatoslav Loginov, there titled Theft, but in Loginov's bibliography there is no title similar to "Theft." Any ideas? --Vasha (cazadora de tildes) 22:37, 15 August 2018 (EDT)

The title of the story is "Кража" -- FantLab record, full Russian text (which is how I was able to identify it.) The reason the bibliography that you used didn't know about it was that the story was first published in 2008 while their list was last updated in 2001. Ahasuerus 22:48, 15 August 2018 (EDT)

Excellent. --Vasha (cazadora de tildes) 22:53, 15 August 2018 (EDT)

Russian title for Japanese variant

Will you take a look at this one? I don't know if this title is completely accurate, but I've noted the source there. Thanks! ···日本穣 · 投稿 · Talk to Nihonjoe 18:27, 16 August 2018 (EDT)

Fixed -- they messed up only one letter, which isn't bad, all things considered :) Ahasuerus 19:51, 16 August 2018 (EDT)

Thanks! I was more worried that it was already listed and I just wasn't finding it. Cyrillic is Greek to me. ;)

If you can find any instances of the story being published, it would be good to add it. ···日本穣 · 投稿 · Talk to Nihonjoe 20:17, 16 August 2018 (EDT)

Done! Ahasuerus 22:07, 16 August 2018 (EDT)

Awesome! Thanks for your help. ···日本穣 · 投稿 · Talk to Nihonjoe 00:07, 17 August 2018 (EDT)

Service with a smile! :) Ahasuerus 00:28, 17 August 2018 (EDT)

language: chiShona

We have a story in the database in Shona/chiShona. --Vasha (cazadora de tildes) 10:07, 5 September 2018 (EDT)

Done! Ahasuerus 11:04, 5 September 2018 (EDT)

Authorless Title

This title doesn't have an author. It is probably from this failed submission (at least it matches one of the essay titles). Wasn't sure if you wanted to look at it before deleting. -- JLaTondre (talk) 14:10, 9 September 2018 (EDT)

Thanks for reporting the record! Its title number (2429734) follows the title number of Combining Perspectives on "Frankenstein", which confirms that they were created as part of the same submission. After restoring the backup file on the development server, I was able to re-approve the problematic submission; I suspect that it was a timing issue. I don't think there is much else we can do about the problem at this point, so please feel free to zap the title. Ahasuerus 14:26, 9 September 2018 (EDT)

Zapped. Thanks. -- JLaTondre (talk) 14:32, 9 September 2018 (EDT)

FredBotting

I feel like I've asked this before, but... I'm trying to resolve FredBotting as an author without an titles. I've searched Titles on Author's Name, Reviewed Author, & Interviewed Author as well as Publications on Author's Name. No results are found. What am I missing? Thanks. -- JLaTondre (talk) 18:48, 12 September 2018 (EDT)

This author was apparently added when the unlucky Frankenstein pub was created -- see the discussion immediately above. I suspect that the failure of the original ClonePub submission caused the author record to become an orphan.

The easiest way to fix a "stuck" author is to create a new title for the author and then delete the title. It should result in the author record getting automatically deleted if there are no other records that are linked to it. I tried it a minute ago and it worked :) Ahasuerus 19:36, 12 September 2018 (EDT)

Awesome. I'll try to remember that... ;-) -- JLaTondre (talk) 20:32, 12 September 2018 (EDT)

Content of "Old rule" and "New rule" cells in "Rules and standards changelog" was mixed-up

Hi, I've just seen that you've mixed-up the contents of "Old rule" and "New rule" cells in the Rules and standards changelog in your latest edits, so that the new rule was stored in the "Old rule" column and vice versa. I've swapped the contents of the affected cells now. I think I caught everything, but you maybe want to double-check if there's still something in the wrong cell. Jens Hitspacebar 13:42, 14 September 2018 (EDT)

Oops! Clearly, I need new optic receptor enhancing devices. Thanks! Ahasuerus 13:56, 14 September 2018 (EDT)

Automated database schema diagram generation

Hi, I just used the nice SchemaSpy tool to create a static documentation of the ISFDB database schema from my local database copy. The tool uses Java and can easily create an up-to-date database schema documentation on an automated, regular basis (e.g. nightly). It could replace the old schema diagram (linked from the Database_Schema page). Are you interested? I could upload a zipped version of the generated docs (approx 12MB) here for you to download and have a look at. Jens Hitspacebar 14:46, 22 September 2018 (EDT)

Well, anything would be better than the schema that we currently have, so I appreciate the offer. It would probably be best if you could e-mail it to my email address (ahasuerus at email.com) directly. I am curious to see if the tool has been able to figure out table relationships since the ISFDB doesn't use foreign keys.

Ideally, we would find a free Python-based tool which we could run on the main server, but I am not sure if such a beast exists. Ahasuerus 18:00, 22 September 2018 (EDT)

The email has been sent, including some background information about the tool and its features and usage. Jens Hitspacebar 06:21, 23 September 2018 (EDT)

Got it, thanks!

A few things come to mind:

Moving table and field comments from the Wiki to the database is definitely a good idea. I have created SR 139.

The "Relationship" page is very nice and vastly better than what we currently have. A cursory review finds only two obvious problems:

• For some reason "pub_id", the primary key in the "pubs" table, wasn't used.
• The "note_id" field in the "authors" table is linked to the "notes" table. This field was disabled a couple of years ago. I kept it in the table definition because the order of fields is meaningful and difficult to change without messing something up. Of course, the "implied" algorithm had no way of telling that the field was dead.

As SR 1, "Enforce Foreign Key consistency", points out:

• [ISFDB] tables are all still using MyISAM, which of course supports FK *declaratively* and *syntatically* but doesn't actually do anything about that information at runtime.

What this means is that we should be able to add foreign key declarations to our table definitions without affecting database functionality. Once in place, they should make SchemaSpy's life much easier.

Given the above, I think the best way to approach this issue would be to implement SRs 1 and 139, then re-run SchemaSpy and see what it comes up with. Would you be interested in giving it a shot? The two SRs are quite straightforward and non-destructive, although they will require a fair amount of typing, copy-pasting and double checking the results.

If everything pans out and we end up with a comprehensive and accurate representation of the database schema, we can look for a way to upload the HTML files to the Wiki without breaking internal links. Does SchemaSpy support setting relative paths? Ahasuerus 13:29, 23 September 2018 (EDT)

Oh, I hadn't seen that it didn't detect the "pub_id" relationship. Weird. Maybe a bug in their guessing algorithm. Let's see if I can debug it and find the cause...

As for SR 1 I've just tried and added a foreign key constraint, but sadly it doesn't help here. The reason is that "creating" a foreign key constraint in a MyISAM actually does nothing (though running the command needs a few seconds and outputs something like Query OK, 566379 rows affected). The MySQL doc says: "For other storage engines [than InnoDB], the clauses are parsed but ignored" (https://dev.mysql.com/doc/refman/5.5/en/alter-table.html#alter-table-foreign-key). That means there's still nothing for SchemaSpy to analyze regarding foreign keys. Jens Hitspacebar 15:07, 23 September 2018 (EDT)

Oh. I thought that adding a foreign key to a MyISAM table would update its definition, but apparently not. Running "alter table votes ADD CONSTRAINT title_id FOREIGN KEY(title_id) references titles(title_id)" didn't change the "votes" table in any way :-( Ahasuerus 15:24, 23 September 2018 (EDT)

The only way seems to really implement SR 1, i.e. add the foreign keys and switch to InnoDB, then probably affecting database functionality. Which, though actually a good thing to have, may be over the top at the moment. If you're interested in SchemaSpy nevertheless (or want to try the switch to InnoDB) I can indeed help out and write the SQLs for the COMMENTs and the FOREIGN KEYs. It'd take some time, though. Jens Hitspacebar 15:07, 23 September 2018 (EDT)

I agree that migrating the main ISFDB server from MyISAM to InnoDB would be too much work for too little gain at the moment. Since MyISAM doesn't preserve foreign key definitions, I can think of only one way to make SchemaSpy generate an accurate schema:

1. Restore a recent backup file on a development server
2. Convert all tables from MyISAM to InnoDB
3. Create a SQL script with commands to add appropriate foreign key constraints
4. Apply the SQL script on the development server
5. Run SchemaSpy
6. Whenever the table layout changes, update the SQL script and rerun the process

It would involve a fair amount of upfront work for the developer who decides to undertake the work and it would require tweaking the SQL script whenever the table layout changes. I don't think I have the bandwidth for it at the moment, but if you are interested, more power to you :-)

That said, I think we should be able to move all table- and column-specific comments from the Wiki to the database without changing the engine, right? Ahasuerus 15:40, 23 September 2018 (EDT)

I'm not sure what you mean with your last question. The generated docs are completely self-contained and already use relative paths. Jens Hitspacebar 15:07, 23 September 2018 (EDT)

Right, I should have realized that the docs were using relative paths since they worked fine when unzipped in a test directory. Never mind, a senior moment :-) Ahasuerus 15:42, 23 September 2018 (EDT)

I found the cause for the missing "pub_id" relationship: SchemaSpy only creates an implied relationship if a column name is exactly used once as a primary key across all tables (i.e. if SchemaSpy can unambiguously deduce the parent table of a "foreign key" column). However, "pub_id" is also the primary key in "bad_images", and therefore SchemaSpy skips the relationship generation for all "pub_id" columns. If I tell SchemaSpy to filter out the "bad_images" table (which is possible), the "pub_id" relations are detected correctly and shown. But then, "bad_images" is missing from the docs.

As for your suggestion to use a database copy and temporarily change it to InnoDb with real foreign key constraints: that would work indeed, but is a bit too much work for me as well. I think if we want to use the document generation it should be part of the regular development process.

So, we can either have a complete documentation of the tables but with missing relationships, or incomplete table docs with a (likely) complete relationship diagram. I.e. we should probably not use it at the moment (unless you think otherwise). But we know now what's possible with this tool and can revisit it if something relevant (like SR 1) changes :) Jens Hitspacebar 16:28, 23 September 2018 (EDT)

I see. It's not too bad then: it so happens that "bad_images" is an experimental table. It was created in 2014 in support of Fixer's effort to find "dead" images. It took Fixer approximately a month to populate the table and I haven't run the process since. It needs to be redesigned (multi-threading), rewritten in Python, deployed on the main server and added to the cleanup process.

I'd say that "bad_images" is more dead than alive at the moment, so it's probably better to rerun SchemaSpy without it. Ahasuerus 17:34, 23 September 2018 (EDT)

I just checked if it catches all relationships if "bad_images" if filtered out. There's only one kind of relationship missing now: the ones going from a "pub_series_id" foreign key to the "pub_series" table. The reason is that SchemaSpy's algorithm thinks that "pub_series.pub_series_id" and "series.series_id" are the same primary key, therefore skipping it. It would work if the "series" table and its key were called "title_series" and "title_series_id". So, it's still not fully usable. But it looks like there's room for improvement of the algorithm. I have to think it through, maybe I can send them a patch. Jens Hitspacebar 13:27, 24 September 2018 (EDT)

Thanks for the update. If you get the software patched and working, please let me know and I'll upload the results to the main server. Ahasuerus 16:46, 24 September 2018 (EDT)

As for the database comments: you mean moving them there regardless of SchemaSpy, and replace their wiki pages with something like "See table and column comments in the database for a description"? That should be possible, yes. I could create an SQL file from the wiki page. Jens Hitspacebar 16:28, 23 September 2018 (EDT)

Yes, that was the idea. The advantage would be that comments would be much more likely to stay in sync going forward. I am not sure it justifies the amount of work that it would take, especially considering the need to test that the copy-and-paste parts of the SQL commands worked correctly. I guess we could start with ALTER TABLE commands, which are non-destructive. Ahasuerus 16:54, 23 September 2018 (EDT)

The Alternate Asimovs

Please take a look at The Alternate Asimovs. The cover is missing as it was deleted last year. It looks like it is still available to undelete. However, since I wasn't sure if there might be an issue with it being the correct cover or something, thought I'd ask you to double check as the verifier. Thanks. -- JLaTondre (talk) 17:10, 15 October 2018 (EDT)

Luckily, most of my "A"s are readily available, so I can confirm that the cover matches my verification copy. The image has been restored -- thanks for the heads-up! Ahasuerus 17:24, 15 October 2018 (EDT)

BNF numbers last items

Good afternoon,

I think I got all the BNF links and IDs that I could think of (the report and all text only ones I could find) last week so I think this is finally done. Can you run the checks through notes of all kinds of elements to see if we still have some links somewhere (as we did for DNB) so I can cross that one off my list? I just cleared the remaining authors and titles as well but short of some heavy custom queries I cannot search other notes (series and so on).

On a somewhat separate note - except for non-reported links (LTF for example), we are down to OCLC only (should be under 30K this week). I am clearing the small ones one more time so you will get similar requests as above in the next weeks so we can try to close this cleanup. :) Annie 16:42, 23 October 2018 (EDT)

Sure, I'll run a custom query once I finish restoring the latest backup on the development server. I should probably implement FR 1140 ("Create Advanced Note Search") next. Ahasuerus 19:54, 23 October 2018 (EDT)

Thanks! If there is a custom query that can find these that I can run via the advanced search, I will be more than happy to do that on my own. I just do not feel like figuring out what tables to look at (especially because I expect I will miss the one that actually has it) :) And yeah - 1140 will be very handy later this year :) Annie 20:12, 23 October 2018 (EDT)

I am afraid the 5 types of Advanced Search that we currently have wouldn't help. Unfortunately, I no longer have the custom query that I ran for DNB and my brain can't handle any non-trivial tasks after 6pm these days. Instead of recreating the custom query, I will work on FR 1140 tomorrow, which will hopefully address this issue once and for all. Ahasuerus 21:33, 23 October 2018 (EDT)

Not urgent at all so no worries. :) Annie 21:50, 23 October 2018 (EDT)

Done -- see the Community Portal announcement. Ahasuerus 19:53, 24 October 2018 (EDT)

You are a rock star! Thanks a lot. I will stop bugging you and go do my own searches now :) Annie 19:58, 24 October 2018 (EDT)

Sure thing. I have tweaked the UI a bit and changed the default value in the drop-down list, so you may want to do a full page reload. Ahasuerus 20:19, 24 October 2018 (EDT)

Wool-gathering

Hello,

As I am ploughing (U.S. : plowing) through all sorts of pub series at the moment, a few ideas crossed my mind, which, no doubt, other editors have had before. There is nothing urgent about all this — more wool-gathering than anything else — but if anything can come out of it, all the better.

• Would there be a way to make a title (typically, e.g. Frankenstein Meets Dracula) belong to distinct series ? There are more and more stories of this type, mixing up characters from different horizons. Linguist 12:30, 26 October 2018 (EDT).

There is a Feature Request (FR 406) to "Allow multi-series titles" and a related ongoing discussion of overlapping and hierarchical publication series here. Since crossover series are becoming more common, we may want to make this FR a higher priority. It's not that hard to do from a purely technical perspective, but first we need to decide how the software should behave in unusual cases, e.g. what we should with a title that belongs to 7 series -- see the discussion linked above. Ahasuerus 18:52, 26 October 2018 (EDT)

• Could the series concept be applied to art titles ? I tried it once, and the result was that the concerned titles appeared together, but without a common title name. This would mainly concern covers with an identical art element, as part of a book series (as in this one for instance). Linguist 12:30, 26 October 2018 (EDT).

Checking the software, I see that the following title types let you enter series information, but are not organized as series on Summary pages: COVERART, INTERIORART, REVIEW, INTERVIEW. I can't think of a good reason to prevent editors from organizing these types of titles as proper series, so I would support changing the software behavior. Would you like to post a proposal on the Community Portal? Ahasuerus 19:02, 26 October 2018 (EDT)

• Would there be a way to view all the cover illustrations of an existing series ? Linguist 12:30, 26 October 2018 (EDT).

At this time you can view all cover images for a publication series -- e.g. Nouveaux Millénaires -- but not for a regular/title series. Since cover images are associated with publication (as opposed to title) records, it didn't seem desirable. However, it wouldn't be that hard to do from the technical perspective and I can certainly implement the functionality if there is support on the Community Portal. Ahasuerus 19:07, 26 October 2018 (EDT)

• Would there be a way to view all the cover illustrations by the same artist ? That would be a great help in finding variants. TIA, Linguist 12:30, 26 October 2018 (EDT).

It's certainly doable. The only concern that I have is that some artists are prolific; loading all of their images may take a long time. If we implement this functionality, we may want to display a warning.

Other than that, it seems like it would be a useful feature. We'll need to think of ways of organizing images by COVERART title, its variants, publication year, etc. Ahasuerus 19:14, 26 October 2018 (EDT)

If it’s just impossible or too time-consuming, or been tried before and dropped, please don’t bother going into the fine details to explain why. But if one of these vagaries seems technically possible and easy to do, well, why not ? TIA, Linguist 12:30, 26 October 2018 (EDT).

Excellent questions! Ahasuerus 19:14, 26 October 2018 (EDT)

Thanks for your responses ! I'll move this discussion onto the Community Portal, and see what happens… Linguist 04:52, 27 October 2018 (EDT).

Talking about server loads

Can you check at what parts of the cleanup reports are running at around 1:17-1:19? The server is generally slow from 1:00 to 1:22-24 but does not generally error out - it is just slow. However between 1:17 and 1:19, it is practically dead for almost any incoming attempts to post or get anything. Do we have a report that is really so intensive? I am usually trying to slow down or just stay away around that time but when I am around, I get response 500 for simple gets more often than not; let alone posts. Some do make it through but not too often. Not that 3 minutes will kill us but if we have a bad report, pulling it may be worth a thought. Thanks! Annie 15:59, 1 November 2018 (EDT)

It would appear that "English Publications with non-Latin characters and without Transliterated Titles" is the guilty party. It took 67 seconds to complete this morning. Looking at the query, I suspect that it worked OK early on, when we had few transliterated publication titles. Let me poke around... Ahasuerus 17:10, 1 November 2018 (EDT)

Why am I not surprised it is one of those :) We rarely have anything in that report anyway - maybe time to move it on a weekly/monthly schedule? We get more and more and more English pubs and more and more transliterations all over the place so that will get worse and worse. Annie 17:50, 1 November 2018 (EDT)

It looks like it may be possible to rewrite these reports not to affect regular processing. I am currently in the middle of fixing Bug 713, but then I should be able to give it a try. Ahasuerus 18:25, 1 November 2018 (EDT)

OK, thanks! Annie 12:35, 2 November 2018 (EDT)

The offending reports have been rewritten and deployed on the live server. On the development server, the elapsed time of the main offender has dropped from 67 seconds to under 2 seconds. I'll poke around to see if I can make similar changes to some other cleanup reports which look for missing transliterated values. Ahasuerus 18:44, 2 November 2018 (EDT)

Tell Fixer to give you an extra cookie today :) Thanks a lot. I will keep an eye for other windows like that when the server errors out - slow does not bother me as much - we have work to do and it passes quickly - that one was just.. ugly - if you look at the errored transmissions, a few of those happened at these times as well. :) Annie 21:28, 2 November 2018 (EDT)

That cookie was delicious! In fact, I liked it so much that I found another problematic transliteration report and fixed it. On to the last batch... Ahasuerus 18:44, 3 November 2018 (EDT)

(unindent) Just FYI - same problem at 1:17-19 this morning. I almost thought it is an hour later but then remembered that USA changed timezones last night (Arizona does not) so it is still 1:17. Annie 01:21, 5 November 2018 (EST)

Thanks! I will need to add time stamps to the nightly log file to help identify the culprit. Ahasuerus 07:46, 5 November 2018 (EST)

OK, time stamps have been added. We'll see what happens overnight. Ahasuerus 15:13, 5 November 2018 (EST)

Any early contenders for the honor of being the server killer? :) Annie 11:03, 6 November 2018 (EST)

I think I see what's going on now. The culprits are "Anthologies and Collections without Fiction Titles" and "Magazines without Fiction Titles". They worked fine when they were deployed, but once we started "ignoring" records, compilation times skyrocketed. I'll investigate and see if the problem can be solved by adding an index or whether the queries need to be rewritten. Thanks for catching the problem! Ahasuerus 12:23, 6 November 2018 (EST)

Great. Let me know when you have something deployed - I will try to just stay away at the times when it happens for now. At least while we were chasing these, we found a few more that needed work. :) Annie 12:27, 6 November 2018 (EST)

The new query logic has been deployed. It's not as elegant as the original version, but at least it doesn't bring the server to its knees (hopefully.) Ahasuerus 20:44, 6 November 2018 (EST)

I will be landing around the time the reports run so it will need to wait a night for me to try my usual thing around that time. Thanks for working on it. Annie 20:51, 6 November 2018 (EST)

(unindent) The log file looked pretty good this morning:

01:00:28: Rpt awardTitles: 18.90 seconds
01:00:55: Rpt submissionsByYear: 26.62 seconds
01:01:04: Rpt titlesByYear: 9.11 seconds
01:01:10: Rpt publicationsByYear: 6.18 seconds
01:01:54: Rpt titlesByAuthorAge: 44.37 seconds
01:01:58: Rpt summaryStatistics: 3.63 seconds
01:03:08: Rpt contributorStatistics: 69.60 seconds
01:03:21: Rpt topVerifiers: 13.26 seconds
01:03:26: Rpt topModerators: 5.71 seconds
01:03:30: Rpt novelsInSeries: 3.97 seconds
01:03:59: Rpt authorsByDebutDate: 26.19 seconds
01:04:06: Rpt mostViewedShorts: 3.14 seconds
01:04:11: Rpt oldestNonLivingAuthors: 2.98 seconds
01:04:23: Rpt mostReviewed: 7.54 seconds
01:04:50: Rpt 1: 26.81 seconds
01:05:13: Rpt 2: 23.40 seconds
01:05:55: Rpt 3: 41.29 seconds
01:05:59: Rpt 4: 4.26 seconds
01:06:04: Rpt 7: 2.27 seconds
01:06:14: Rpt 8: 9.47 seconds
01:06:29: Rpt 14: 2.87 seconds
01:06:33: Rpt 15: 3.65 seconds
01:06:41: Rpt 20: 4.01 seconds
01:06:43: Rpt 21: 2.53 seconds
01:06:59: Rpt 32: 12.34 seconds
01:07:31: Rpt 33: 31.47 seconds
01:07:46: Rpt 34: 14.86 seconds
01:07:51: Rpt 38: 2.36 seconds
01:07:54: Rpt 40: 2.55 seconds
01:07:59: Rpt 42: 3.72 seconds
01:08:06: Rpt 45: 3.84 seconds
01:08:37: Rpt 47: 29.90 seconds
01:08:41: Rpt 48: 4.26 seconds
01:08:43: Rpt 49: 2.11 seconds
01:08:45: Rpt 50: 2.36 seconds
01:09:34: Rpt 52: 34.82 seconds
01:09:42: Rpt 54: 8.21 seconds
01:09:45: Rpt 58: 2.34 seconds
01:09:47: Rpt 59: 2.38 seconds
01:09:50: Rpt 60: 2.36 seconds
01:09:52: Rpt 61: 2.37 seconds
01:09:58: Rpt 63: 3.81 seconds
01:10:04: Rpt 69: 4.12 seconds
01:10:08: Rpt 71: 2.82 seconds
01:10:12: Rpt 74: 3.12 seconds
01:10:29: Rpt 80: 15.54 seconds
01:10:48: Rpt 87: 16.38 seconds
01:10:53: Rpt 88: 5.07 seconds
01:10:57: Rpt 92: 2.59 seconds
01:11:13: Rpt 93: 16.23 seconds
01:11:18: Rpt 94: 4.39 seconds
01:11:21: Rpt 95: 2.94 seconds
01:11:29: Rpt 100: 7.92 seconds
01:11:43: Rpt 191: 12.82 seconds
01:12:09: Rpt 193: 25.95 seconds
01:12:15: Rpt 196: 3.96 seconds
01:12:19: Rpt 197: 3.83 seconds
01:12:22: Rpt 218: 3.50 seconds
01:12:26: Rpt 219: 3.69 seconds
01:12:30: Rpt 220: 3.46 seconds
01:12:41: Rpt 221: 11.07 seconds
01:12:44: Rpt 222: 3.50 seconds
01:12:48: Rpt 223: 3.62 seconds
01:12:51: Rpt 224: 3.40 seconds
01:12:55: Rpt 225: 3.46 seconds
01:13:13: Rpt 226: 18.05 seconds
01:13:16: Rpt 227: 3.08 seconds
01:13:38: Rpt 229: 21.05 seconds
01:14:31: Rpt 233: 20.90 seconds
01:14:35: Rpt 234: 3.81 seconds
01:14:42: Rpt 237: 6.64 seconds
01:14:46: Rpt 238: 3.94 seconds
01:14:57: Rpt 239: 11.88 seconds
01:15:06: Rpt 240: 8.41 seconds
01:15:11: Rpt 241: 4.72 seconds
01:15:14: Rpt 242: 3.29 seconds
01:15:41: Rpt 111: 18.81 seconds
01:15:50: Rpt 97: 2.56 seconds
01:15:53: Rpt 99: 2.51 seconds
01:16:02: Rpt 127: 7.19 seconds
01:16:11: Rpt 137: 8.75 seconds
01:16:18: Rpt 167: 2.30 seconds
01:16:26: Rpt 168: 7.60 seconds
01:16:40: Rpt 175: 2.01 seconds
01:16:53: Rpt 188: 2.24 seconds
01:17:06: Rpt 55: 13.16 seconds
01:17:11: Rpt 56: 4.16 seconds

Of course, there is always room for improvement. Ahasuerus 15:43, 7 November 2018 (EST)

Two nights in a row, no server errors - actually the delays at 17-19 are less pronounced than these earlier in the hour. So I'd say that whatever was causing it is indeed fixed. Annie 01:26, 9 November 2018 (EST)

The sweet sweet taste of victory! (Or at least narrowly escaping disaster :-) Ahasuerus 11:35, 9 November 2018 (EST)

Deleted Images

In the past, we've made a point of deleting older versions of images under the impression this saves space. I don't believe this is the case (unless there is some manual clean-up process that hasn't run in quite awhile?). There is this case of when the whole image is deleted. I also just did a trial with this image for deleting versions. The logs show Bluesman deleting the oldest version six years ago, me deleting another version today, and then me restoring all versions. If you look at the current image page, all three images are present, including the oldest which Bluesman deleted six years ago (if you click on the date-time stamp, it will show the image). As far as I can tell, any clean-up people have done on old image versions has been pointless. -- JLaTondre (talk) 10:21, 3 November 2018 (EDT)

Regular "deletion" of a Wiki-based page or file/image merely archives it. There is a MediaWiki maintenance script, DeleteArchivedFiles, which can be run to delete the archived files, but we haven't run it in many years (probably ever.) In other words, manual deletion of old image revisions is still helpful in case we ever find ourselves very low on disk space and need to do something drastic in a hurry.

That said, I wasn't aware of the fact that the reason why images were being deleted was to save disk space. Either I missed the original discussion or I no longer remember it (Fixer tells me that my memory isn't what it used to be.)

Our overall disk space situation improved considerably a few months ago when I identified and zapped a certain infinitely growing log file. Due to this fact, we have a number of other, less drastic, mitigation options that we could exercise if disk space were to become a problem again.

Going forward, my take on it is that we should probably continue "deleting" old image revisions, but it's not something to lose sleep over. Ahasuerus 11:06, 3 November 2018 (EDT)

That's the rationale I remember being given, but then I don't always trust my memory either! ;-) I re-deleted the files I had restored. -- JLaTondre (talk) 11:17, 3 November 2018 (EDT)

Pub Images

Hello,

We have almost 26K publications that have an image format of http://images.amazon.com/images/P/0452287626.01.LZZZZZZZ.jpg which are the same as http://images.amazon.com/images/P/0452287626.jpg. Do you think it will make sense to clean these up with a backend script? Or am I overthinking the whole "clean your links" thingie? We also have at least ~1700 (search - it can use a 4th term to remove the few non-amazon ones but it is close enough to show what I mean) which are showing an images/I link that is not cleaned (such as https://images-na.ssl-images-amazon.com/images/I/41iletI-qeL._SX398_BO1,204,203,200_.jpg. Same question here? Although I can clean these manually if needed. Thoughts? :) Annie 22:50, 4 November 2018 (EST)

I agree re: LZZZZZZZ; SR 143 has been created. The rest of the "odd" image URLs vary and my require additional TLC. For example, this pub uses a "png" image which doesn't follow the standard Amazon pattern. It's probably safer to create a cleanup report for uncommon Amazon URLs and let moderators ignore false positives. Ahasuerus 11:29, 5 November 2018 (EST)

Sounds like a plan. I will add that to my cleanup tasks in the meantime to get some off the list Annie 15:55, 5 November 2018 (EST)

A new cleanup report and post-submission warnings have been added -- see the Community Portal announcement. Ahasuerus 18:49, 5 November 2018 (EST)

Thanks!Annie 11:01, 6 November 2018 (EST)

Encoding problem

I noticed some oddities when trying to cut-and-paste data that was submitted by an unknown person last week -- oddities having to do with accented characters. Observe this submission where I tried replacing the existing data with standard-encoded accented characters and found that they were different. I think most of the weird characters are in the various titles and publications on Sergio Gaut vel Hartman's page-- I hope you can figure out what is going on there. --Vasha (cazadora de tildes) 04:20, 7 November 2018 (EST)

I'll take a look, thanks! Ahasuerus 08:05, 7 November 2018 (EST)

It looks like it will have to wait until tomorrow morning when the new backup file becomes available on the development server. Ahasuerus 08:57, 7 November 2018 (EST)

It looks like the difference is the proposed change alters the name from "Vázquez" to "Vásquez" (the first "z" becomes an "s"). That's why it said it was a new author. Which spelling is correct? ···日本穣 · 投稿 · Talk to Nihonjoe 13:13, 7 November 2018 (EST)

Uh ... You are right. Now I have done my big blunder for the day. (Nonetheless, I think there might be an encoding oddity in those submissions, and would appreciate checking if it's not too much trouble). --Vasha (cazadora de tildes) 13:19, 7 November 2018 (EST)

Sure, I'll take a look once the development server has been updated. Basically, we have a filter which normalizes all entered characters according to certain rules, e.g. the 3 types of the yen sign are replaced by the one that we support. However, there are a lot of different Unicode characters out there, so it's possible that we may be missing something. Ahasuerus 13:48, 7 November 2018 (EST)

I have updated the development server and examined submission 4031428. The two non-English characters, "á" and "í", are encoded correctly, i.e. as Latin-1 values 225 and 237 respectively. Were there any other problematic submissions? Ahasuerus 11:26, 9 November 2018 (EST)

Thanks. I think things are OK. But in general, sometimes there's that letter-plus-combining-accent (what's the technical term for it?) which can be a problem if cutting and pasting from Worldcat. Do you have a way of catching those? --Vasha (cazadora de tildes) 11:36, 9 November 2018 (EST)

Our input filter includes a section which identified and converts a whole bunch of "combining diacritics". However, it's very likely that the filter doesn't catch everything because the number of possible permutations is very high. If you find new combinations, please post the OCLC number (or another third party URL) here and I will add it to the filter. Ahasuerus 11:44, 9 November 2018 (EST)

External Identifier search

Is there any way to search for an identifier from a specific type NOT starting with something? I am trying to cleanup wrongly assigned types (a lot of GR and OCLC ones in the wrong place) and there are a few easy way to spot possible problematic ones in some categories(BNF not starting with cb for example). But I cannot figure out a way to do the search. Going number by number and letter by letter is the only option I have and I will do it this way but just thought to ask :) Thanks! Annie 15:31, 7 November 2018 (EST)

There is no way to do it using Advanced Search as it currently exists, but we can try to enhance the interface. I'll see what I can do tonight once I finish entering the remaining J-Novel Club pubs. Ahasuerus 15:35, 7 November 2018 (EST)

That's what I thought but wanted to verify. It is not urgent at all - and I have a workaround with multiple searches anyway :) Annie 15:44, 7 November 2018 (EST)

It's always a balance between "quick wins", usually of the usability variety, and long-term development... Ahasuerus 16:09, 7 November 2018 (EST)

Oh, I understand that :) If there is a quick way that does not kill the server, I would love to have it. If not - I can work with what we have. I am getting very good with using multiple searches (based on what we have available) to get me what I need :) Annie 16:16, 7 November 2018 (EST)

Done -- see the Community Portal announcement. Ahasuerus 20:18, 7 November 2018 (EST)

Audio Titles and page numbers

We have a huge amount of audio title with pages set to 0 (most likely because the old bibliographic warning that was not accounting for the type of material - you fixed that at some point). Do we need a FR to get these cleaned out? e-books are trickier (due to PDFs...although the rules do not make the distinction) but all the audio materials (all 6 types) should not have their page numbers set. What do you think? Annie 18:39, 7 November 2018 (EST)

Hm... It looks like there may be a bug in the publication display logic. When I run a search on Page Count is exactly 0 AND Format is Exactly audio CD, I get a long list of pubs. However, the standard publication table doesn't display anything in the "Pages" column. Moreover, the Publication page doesn't display anything either. It's only when I pull it up in Edit Publication that I can see that the page count is set to 0.

I guess we'll need to fix the display bug first and then we can decide what to do with the 2914 audio pubs with a "0" page count. Ahasuerus 20:29, 7 November 2018 (EST)

P.S. Now that I am thinking about it, we may have looked into this issue a few years ago. I don't recall what (if anything) we decided, so I'll need to check SourceForge history. Ahasuerus 20:35, 7 November 2018 (EST)

Null vs empty string in the DB maybe? Are the 3K audio CD ones or across all 6 audio types? Annie 20:37, 7 November 2018 (EST)

They are stored as "0"s in the database. Some/most of them may have been submitted by Fixer since Amazon frequently returns "0" page counts for certain "binding" formats.

Grouping them by format, we get:

+------------------------+----------+
| audio cassette         |      537 |
| audio CD               |     1338 |
| audio LP               |       13 |
| audio MP3 CD           |      230 |
| digital audio download |      281 |
| digital audio player   |       44 |
| ebook                  |     1431 |
| other                  |       10 |
| ph                     |        1 |
| tp                     |        4 |
| webzine                |      119 |
+------------------------+----------+

Ahasuerus 20:48, 7 November 2018 (EST)

I see. I will check the ph and the tp ones; what do we want to do with the rest? Fixer cannot really submit the non-ISBN ones and a lot of those do not have an ISBN. I do remember the very annoying bibliographic warning though so I think more came from people fixing that... Annie 20:54, 7 November 2018 (EST)

So... the ph one is this one. I do not see a 0 even in editPub. Any idea what is going on? Annie 21:02, 7 November 2018 (EST)

Never mind - I am blind - it is there. Cleaned now. All the printed media from above is now cleaned.Annie 21:05, 7 November 2018 (EST)

So... it was not Fixer, it was the other robot - Dissembler. I pulled a 0 from this one while fixing the image - and I saw a few more like that today. Not that it matters much but it got me interested on how non-ISBN titles may be robot-based. Annie 22:56, 7 November 2018 (EST)

Ah, Dissembler. It's been quite a few years since it fell dormant. Unlike the early ISFDB robots, which had crawled the Web (mostly library catalogs) and dumped data directly into the database, Dissembler was a well-behaved robot. The original plan was that Al would use it to add forthcoming books. As Al's ISFDB time dwindled, it became less and less feasible. At that point Fixer, which had been created as a data maintenance robot (hence his name), had to take over as the primary data acquisition agent. Eventually Dissembler ran into a brick wall when Amazon implemented a new authentication mechanism and Al didn't have the time to upgrade the code.

Yeah, I had been finding a lot of cleanup in his old submissions though - a lot of the uncleaned Amazon links from the early titles are these submissions - they are not a complete line but a partial one (just a single _something_) so I think there was a missed parser there. I rarely saw these on the identifiers or languages cleanup because of the kinds of books it was adding - and any mention of it had been removed when someone verified later. Annie 17:41, 8 November 2018 (EST)

Re: Fixer's handling of ASIN-only pubs, it was an issue for many years. In October I finally upgraded the code to leverage the new Web API functionality. Now we have submissions like this one, which was crafted by Fixer and submitted on my behalf. As I wrote at the time, "it's not as robust as the rest of Fixer's logic, e.g. it doesn't support internal prioritization, but it lets us handle ASIN-only ebooks in a semi-automated fashion."

I know but these are mainly old records from the pre-Fixer can deal with those days. :) That's the only reason I mentioned it - I know it can do it now :) Annie 17:41, 8 November 2018 (EST)

Re: "0" page counts, I couldn't find anything in our SourceForge archive. I'll ask on the Community Portal to see if anyone may remember if it's something that we used to do. Ahasuerus 11:53, 8 November 2018 (EST)

Makes sense. Thanks! Annie 17:41, 8 November 2018 (EST)

(unindent) Can you pull a report of the number of audio titles with number of pages (higher than 0)? I just cleaned one (clonePub from a text one I suspect) so we may have more of those that need cleaning. Thanks!Annie 21:38, 8 November 2018 (EST)

Here is what I see on the development server:

+--------+---------------------------------------------------------------+
| pub_id | pub_title                                                     |
+--------+---------------------------------------------------------------+
|  43123 | The Lion, the Witch and the Wardrobe                          |
|  81470 | Woken Furies                                                  |
| 187809 | These Dreams That Sleep Disturbs                              |
| 279499 | Misery                                                        |
| 288798 | Animal Farm                                                   |
| 519459 | I Am Spock                                                    |
| 543297 | The Boy Who Could Fly                                         |
| 552029 | Die Sterntagebⁿcher: 7. & 8. Reise                            |
| 554637 | The Enormous Room                                             |
| 603167 | Jump Point, Issue 04.10                                       |
| 629079 | Savage Scrolls, Volume One: Scholarship from the Hyborian Age |
| 680481 | The Fear Trilogy CD Collection                                |
| 690772 | The Labyrinth Index                                           |
+--------+---------------------------------------------------------------+

Ahasuerus 22:38, 8 November 2018 (EST)

Thanks! I will check them here and clean them up if needed. Annie 23:23, 8 November 2018 (EST)

Mismatched HTML report update needed

It is looking only for mismatched pairs - what it misses are lonely <li> items (search) - it found a bit under 100 publications that have li but do not have either ul or ol. Any chance to add these to the report so we can catch them when they happen? Thanks! Annie 22:32, 7 November 2018 (EST)

Done -- see the Community Portal announcement. Also, it occurs to me that we should also display yellow post-submission warnings for notes with invalid HTML. We could even add pop-up validation, although my JavaScript skills are particularly weak. Ahasuerus 11:07, 8 November 2018 (EST)

The yellow warning will be awesome - I tend to notice the weird look of the approval screen more often than not when the html was messed up but sometimes I miss it - and considering how often we have something in this report, so do others. Popups will make it even hard for me to work from my phone... so yellow warnings please... Annie 13:20, 8 November 2018 (EST)

OK, FR 1220 has been created. Ahasuerus 13:32, 8 November 2018 (EST)

Translation Nice to Have Feature

We are having a lot of translations being entered now. When the translated title doesn't already exist, it is a two step process to add the new book and then variant the translated title to the original. At the title level, it would be nice to have an "Add Translation to This Title" button which would simply be the "Add New Novel" (or whatever one is the equivalent type as the parent) screen, but it would automatically variant the new container title to the original title. Hopefully that description makes sense. -- JLaTondre (talk) 09:19, 8 November 2018 (EST)

I see. I guess it needn't be limited to translations. We could add a general purpose "Parent Title #" field to the "Title Data" section of the "New Publication" form. It wouldn't be hard to do.

There is one thing that we need to sort out, though. What happens if the editor enters an invalid parent title number or if the entered parent title record is deleted while the submission is in the queue? At this time, "New Publication" submissions are self-contained, but adding a "Parent Title #" field would change that. We could modify the software to make NewPub submissions with references to an invalid parent title unapprovable, but hard-rejecting a submission with potentially dozens of Contents titles would be a huge waste of editor time.

My current thinking is that non-existing parent title records should result in a yellow warning on the submission review page. The warning would say something like "This title record doesn't exist. If the submission is approved, the newly created title record will not be turned into a variant." It would be up to the approving moderator to contact the submitter and explain that another submission is required in order to finalize the process. Does this approach make sense? Ahasuerus 13:09, 8 November 2018 (EST)

Yes, could be made generic, but usual case is with translations. We have a couple of authors in the process of expanding our translations and I was envisioning this as a help to them. Either way, I would make it an option off a parent title record only and have it use the parent title id. That would avoid cases where the user mistypes the id. It would still need to protect against a deleted title, but that would happen less often than a typo. If the parent is deleted, creating the record without a parent and showing a warning notice would be a good approach. -- JLaTondre (talk) 16:13, 8 November 2018 (EST)

Makes sense. I'll have to check the code, but it looks doable. Ahasuerus 18:42, 8 November 2018 (EST)

I would love that. I still need to get the Bulgarian books in (I keep getting distracted with side project) and anything that allows me to that more efficiently (both for the translations and the transliterations) will be great :) And please no hard rejects - just do not make the variant if the ID does not exist or is the wrong type... I would not be that worried about mistyped IDs - the process is not different from someone mistyping on a makeVariant - it is easy to break if it is an obvious mistake and if it is not, even on a created one.

How about adding the ability to add parent ID and a transliteration for contents titles during the initial submission? Annie 17:36, 8 November 2018 (EST)

Any changes that affect the Contents section tend to be time-consuming. I cleaned up the JavaScript code earlier this year, but it's still a bear and my JavaScript skills are painfully weak.

As far as transliterations go, I find it more efficient to wait for the missing ones to pop up on one of the cleanup reports and zap them all at the same time. Ahasuerus 18:42, 8 November 2018 (EST)

Well, depends on how many there are... 20 are manageable - 400 - not so much. I know what you mean - but if I am going to variant anyway, adding the transliteration while adding the translator is not too bad (yeah, I know - if I am going to open for notes, no need to do it on the content level). It was just my unicorn request... pink unicorn at that. I did not expect it to happen but had to ask. :) Annie 19:02, 8 November 2018 (EST)

I like this idea. As mentioned before, it would be nice to have a way to enter all transliterations for a submission at the same time (such as for shorts in an anthology), but I understand that would complicate the process quite a bit. I'll settle (for now) for a way to enter a translation from a single link. ···日本穣 · 投稿 · Talk to Nihonjoe 19:22, 8 November 2018 (EST)

(unindent) I have poked around a bit and here is what I think. Suppose we have a title like this one and we want to add a translation. Normally, when you click Add Publication to This Title, the following fields are grayed out:

• Title
• Transliterated Title(s)
• Author(s)
• Pub Type

The proposed option, which we will tentatively call "Add Translated Publication to This Title", will look the same with the following exceptions:

• title, transliterated title(s), and author(s) will not be grayed out
• a new drop-down list, "Language", will be added
• a new field, "title note", will be added. It will be used to capture the name of the translator(s) and/or any other translation-specific information.

The new data entry form will create a submission which, once approved, will create a new title record using the entered title/transliterated title(s)/author(s)/language/title note. It will also turn the newly created title into a variant of the original title. The new title will inherent the values of the "title flags" from the parent title.

Does this sound about right? If it does, I will copy this proposal to the Community Portal for further discussion. (It would be more work than I originally expected, but certainly doable.) Ahasuerus 12:46, 10 November 2018 (EST)

Sounds really good. It's more than I was even hoping. ;-) -- JLaTondre (talk) 13:43, 10 November 2018 (EST)

OK, I have posted the proposed solution on the Community Portal. Ahasuerus 15:28, 10 November 2018 (EST)

Variant Approval Nice to Have Feature

When displaying variants for moderator approval, it would be nice if it displayed them similar to merges. In other words:

1. If the values in both columns match, show green on both sides and only show red when there is a delta
2. Display authors in alphabetical order on both sides
3. Display the notes field

This would make it easier to compare the records. Thanks. -- JLaTondre (talk) 10:27, 8 November 2018 (EST)

Good points; FR 1219 has been created. Ahasuerus 13:14, 8 November 2018 (EST)

I like this idea, too. ···日本穣 · 投稿 · Talk to Nihonjoe 19:23, 8 November 2018 (EST)

New language

Hello, is possible to submit a new language to ISFDB? I'd like to submit the Mirandese (Mirandés) because I'd like to submit the book L Princepico (Le Pettit Prince). Thanks, ErickSoares3 08:46, 11 November 2018 (EST)

The ISFDB follows the ISO 639-2 standard for languages. Since Mirandese is an ISO 639-2-recognized language, we should be able to add it. Give me a few hours... Ahasuerus 09:44, 11 November 2018 (EST)

Done -- see the Community Portal announcement. Ahasuerus 10:37, 11 November 2018 (EST)

I just saw! Thanks! ErickSoares3 11:05, 11 November 2018 (EST)

Submission Raw XML Links

For pending submissions in the moderator queue, there is a View Raw XML link. This would be a nice feature on completed merges or force rejected edits (both edits where the current history view is not able to be shown). That would at least allow you to see what was submitted. See this discussion for a relevant use case. Thanks. -- JLaTondre (talk) 09:30, 14 November 2018 (EST)

Good idea; FR 1223 has been created. Thanks! Ahasuerus 09:59, 14 November 2018 (EST)

Turns out the current link is limited to moderators. If there is not a reason for that, it would be nice to have it viewable by the submitter as well. If there is a reason, it would still be good to have the link for moderators. Thanks. -- JLaTondre (talk) 10:21, 14 November 2018 (EST)

I believe that it is limited to moderators for historical reasons -- the script was originally put in the "mod" directory, which is only accessible by moderators. We should be able to move to it to the "edit" directory where it will be available to all editors. Ahasuerus 10:27, 14 November 2018 (EST)

That'd definitely be useful. --Vasha (cazadora de tildes) 10:30, 14 November 2018 (EST)

problematic title

While looking for unusual Transliterated Titles, I found this, where something in the title field is making it not display properly -- it should be "</3</3". How should this title be entered so that software doesn't mistake it for HTML? --Vasha (cazadora de tildes) 13:38, 14 November 2018 (EST)

Also I am not sure how this is being done (combining diacritics)? It is displaying correctly in my browser, but I wonder if it might not work in others. --Vasha (cazadora de tildes) 14:25, 14 November 2018 (EST)

This is Bug 673, which is currently outstanding. It will require a significant amount of work to fix. It's also related to security issues and to the fact that we allow HTML in notes fields. Unfortunately, I am unaware of a straightforward way to get angle brackets to display correctly and consistently at this time. It's something that I have been meaning to concentrate on for the last few months, but other issues keep interfering. Ahasuerus 14:57, 14 November 2018 (EST)

Vasha, this is similar to the other one I fixed - you need to trick the software not to resolve the. Fixed that one as well. Another edit will break it again but... not much we can do about that. Annie 00:31, 16 November 2018 (EST)

Wiki cleanup - how does it work?

Hi Ahasuerus, I can't seem to find info on what is expected when I come across an Author wiki page that is eligible for cleanup. Could you point me in the right direction? I have, for example, been looking into Mary Hoffman recently, and am cleaning up her Bibliography in the ISFDB. And while doing so I noticed that there's a reference to the Author:Mary Hoffman wiki page. In this particular instance the only info that's there is a redirect to an old page of yours :-). Might be a bit atypical as an example, but nevertheless, what should I do when encountering Author (or other) wiki pages in scope of Wiki cleanup? Thanks! MagicUnk 08:21, 15 November 2018 (EST)

I have moved the discussion to ISFDB:Help_desk#Wiki_cleanup_-_how_does_it_work.3F. Ahasuerus 14:18, 15 November 2018 (EST)

Sweet Dreams, Sweet Princes

Can you check this one? Does it really say "Mack Renyolds" in the copyright statement or does it say "Mack Reynolds"? Thanks! Annie 00:25, 16 November 2018 (EST)

Corrected and expanded. Thanks! Ahasuerus 08:55, 16 November 2018 (EST)

See, that's why I like all that cleanup - we fix all kinds of other issues along the way :) Thanks! Annie 13:52, 16 November 2018 (EST)

Publications with Duplicate Titles

Can you look at this and see if you can figure out what it is complaining about exactly? I had been staring at the publication for the last 20 minutes and it just makes no sense. This report can be a bit... interesting to understand but I just cannot see what is wrong now. Its sister publication under the same title is not getting flagged and they do look the same so... what's going on here? What am I missing? Thanks! Annie 15:20, 26 November 2018 (EST)

I am not sure what's going on here. Let me take a closer look... Ahasuerus 15:46, 26 November 2018 (EST)

OK, I think I see what's going on. If you pull the pub up in Removal Editor, you'll see that title 2459918, Das Kargad-Reich / Der Ostbereich (map), appears 3 times, on pages 27, 234 and 525. The way the software currently works, a publication can't include the same title multiple times. You can force duplicates in using various tricks, but only one will appear in the Contents section, which is why you couldn't see anything wrong in the Contents section.

We have FR 609, "Allow multiple occurrences of the same title record per publication record", but I haven't scoped it out yet. Ahasuerus 16:01, 26 November 2018 (EST)

Ah, I thought I checked ever single title and somehow managed to miss that one. Thanks for finding it. Who is going to discuss with Christian (2 new titles, variants and so on)? Annie 17:04, 26 November 2018 (EST)

Actually, 2459918 is only one of the duplicate titles. 2459920 and 2459921, both of them maps, are also duplicates. That's one of the nicer thing about having direct access to the underlying SQL tables -- you can craft all kinds of ad hoc queries :)

I'll go ahead and leave a note on Christian's Talk page. Ahasuerus 17:10, 26 November 2018 (EST)

You may want to track down whatever changed that work in the last days - I expect an import (or 3) which did not detect the duplicates (because I cannot see Christian merging them despite the very clear warning "these are in the same work, what do you think you are doing?" if you try the merge once they are in the same work. Maybe it is an import problem on top of the rest? At least the logs of updates should tell you that. Especially because this one does not have the same issue (571767 is the broken one, 572467 is the ok one). Annie 17:15, 26 November 2018 (EST)

I'll scan the submission table to see if I can find what happened. Unfortunately, it's not always a straightforward process since submissions only store the changed data. Or, to be more nearly precise, what was considered "the changed data" at the time of the submission. Ahasuerus 18:14, 26 November 2018 (EST)

Yeah, I realize that. At least we should be able to find out if it was a clone or an import that created that (or a later merge)... Annie 18:24, 26 November 2018 (EST)

Identifiers and numbers

When you have a chance, can you get me some numbers on how many identifiers we have per type (already housed on their proper fields)? Just curious - so not urgent at all :) Annie 18:47, 27 November 2018 (EST)

Come to think of it, it would be a good addition to the ISFDB Statistics page. I'll see what I can do, although it may take some time since I am in the middle of migrating Fixer and Co to the new development server. Ahasuerus 19:28, 27 November 2018 (EST)

That's what I was thinking as well but decided not to bug you for a change on a page (for now) :) And no worries - as I said - I am just curious. Annie 19:35, 27 November 2018 (EST)

FR 1228 has been created to make sure that I don't forget it. Ahasuerus 13:05, 30 November 2018 (EST)

tag edit?

Pwendt pointed out to me that I have been using the tag "fairy tale retelling" where everyone else uses "retold fairy tales," and that except for one instance, I'm the only person who's used this tag. I would like to change to the same as everyone else; but since I have used this tag over 100 times, can you just edit it/merge it with "retold fairy tales" so I don't have to change them individually? --Vasha (cazadora de tildes) 10:33, 30 November 2018 (EST)

There is a Feature Request (FR 911) to "Allow moderators to edit and merge tags", but it's not a high priority.

In the meantime it would be possible to create an ad hoc script to change one tag. However, manual database modifications are error-prone, so they require a lot of testing (a couple of time flawed scripts almost resulted in corrupting the data.) It's typically more efficient to perform any changes that affect fewer than 500-1,000 records manually. Ahasuerus 12:33, 30 November 2018 (EST)

On a related note, shouldn't "title tag" in the advanced search list just be the more intuitive "tag"? There are no tags for anything but titles, and no plans to add any, right? --Vasha (cazadora de tildes) 17:11, 30 November 2018 (EST)

At one point we used "publication tags" instead of publication IDs. They are no longer visible, but, unfortunately, we can't completely get rid of them yet. They are needed by the image upload process because the Wiki software doesn't play nice with purely numeric IDs. One of these days... Ahasuerus 17:53, 30 November 2018 (EST)

OK, if publication tags are only used by Wiki software, they are irrelevant and unknown to most people. You could still put just "tag" in the advanced search list without confusing anyone; indeed, if you put "title tag" people will start wondering what other kinds of tags there are, an unneeded distraction. --Vasha (cazadora de tildes) 18:07, 30 November 2018 (EST)

Publication tags are one of the selection criteria in the Advanced Publication Search, where they are called "Tags". That's why the tags in the Advanced Title Search were originally called "Title tags" -- to help distinguish between them.

Now that hardly anyone uses publication tags, we could probably remove them from the drop-down list in the Advanced Publication Search. We could them change "Title Tag" to "Tag" to make the Advanced Search terminology consistent with the regular search. We should probably ask on the Community Portal first in case some editors are still using publication tags in Advanced Publication searches. Ahasuerus 18:26, 30 November 2018 (EST)

I posted about this on the Community Portal a week ago and no one has commented. That may indicate that the concept "publication tags" means nothing to anyone, what do you think? --Vasha (cazadora de tildes) 15:21, 9 December 2018 (EST)

Sounds reasonable. I'll go ahead and create an FR. Ahasuerus 16:08, 9 December 2018 (EST)

Help Pages cleanup again

Can we zap this page (and the link to it from How to?

The first section here also need to be removed. Annie 19:14, 30 November 2018 (EST)

Thanks, I'll take a look tomorrow morning. Ahasuerus 21:50, 30 November 2018 (EST)

Case question

In the note about the translator here do you think we should change the case of the Russian or the parenthetical translation/transliteration, or just leave it as-is? It caught my eye because a submission changed it from straight "Translated by" text to using the template. Since the Polish publication presumably didn't have a credit in Russian that would have used literally "Ирены Левандовской", I'm thinking it's more appropriate to use "Ирена Левандовская" in that note. --MartyD 09:23, 1 December 2018 (EST)

Now that I look her up, I see she's Polish. See https://librusec.pro/a/88579 . So I don't even know why we have the Russian/Cyrillic there.... --MartyD 09:26, 1 December 2018 (EST)

The parent title, Очень странный мир, is a Russian novelette. The Polish translation originally appeared in Kroki w nieznane tom 1, which was apparently imported from this FantLab record. The source of the Russian spelling of Lewandowska's name is this FantLab record. I think we can safely zap the Cyrillic version. Ahasuerus 11:24, 1 December 2018 (EST)

P.S. Cf. "Dlaczego?", which appears in the same Polish anthology: "translation credit T. Gosk [Тадеуш Госк/Tadeusz Gosk (1936 – 1993)]". Ahasuerus 11:27, 1 December 2018 (EST)

The OCLC report

The report either needs an ignore to get these 2 out of it (they are oclc links but are there for a subpage so no easy way to move them cleanly and I would rather leave them as links). Alternatively, we can retire that report :) Someone randomly adding a link is easily caught with a search once a month or so so no reason to check the Notes field every night :) Annie 02:02, 19 December 2018 (EST)

Nicely done! Once I refresh the development server, I'll try to tweak the logic to ignore these patterns. I'd rather keep the report around because there is no way of telling if future editors will be familiar with certain things that we currently take for granted. I have seen it happen a number of times over the last couple of decades. (Fixer tells me that it's not surprising considering how fleeting human existence is, but then he is only 10, so what does he know?) Ahasuerus 07:55, 19 December 2018 (EST)

If you are going to tweak the report, add also the ability for it to detect the usual patterns in a subdomain - there were a handful of publications that the report did not catch and all of them had the same pattern: https://somethings.worldcat.org/and so on where the something could be a double subdomain as well (nova.something for example). Or just go the other way, look for worldcat.org and add ignore. Annie 10:15, 19 December 2018 (EST)

Numbers for Translations (again)

Hello Ahasuerus,

Can you get me the number of remaining records in both translator reports? Unlike the identifier where I finally figured out how to count without bugging you, these one cannot be counted via Advanced Search...:) And having numbers to work against kinda helps keeping me focused (plus a lot of people were doing work on these lately so it is a good checkpoint):) Thanks! Annie 14:44, 20 December 2018 (EST)

Sure thing. As of 2 days ago:

• Translations without Notes: 35,559
• Translations without the Tr Template in Notes: 25,998

Ahasuerus 15:02, 20 December 2018 (EST)

Thanks! No surprises then - ~3.5K less in the second group and almost no movement in the first since June. Unlike the external numbers, people keep adding to those so not too bad... Time to go and kick these around a bit :) Annie 17:33, 20 December 2018 (EST)

(unindent) When you have a chance, how are the numbers looking like? I expect the second number to be significantly lower... Annie 20:27, 27 December 2018 (EST)

Live server: 34,363 and 17,772. Impressive! Ahasuerus 20:47, 27 December 2018 (EST)

Well, that's even more significantly lower than I expected :) Zapp had been clearing a hundred (or couple of hundred) daily out of the report after the OCLC report was closed for good, a few more people chimed in here and there and I was a bit bored and could not sleep :)

We may need to do something about the first report - because it requires research and not everyone can do that in all languages, it can get a bit stuck... And when we are stuck to mostly stories, it gets less likely that someone wants to work on them. Two options I can see: restrict it by type, then keep adding types with time (novels are easier to research than short stories) or split the big DB languages on their own... I am more inclined to go for a type restriction quite honestly - it will get it moving... Annie 20:59, 27 December 2018 (EST)

I guess it's up to the editors who work on this report to determine what kind of fine-tuning it needs. Perhaps we could ask on the Community Portal? Ahasuerus 15:12, 28 December 2018 (EST)

I will although you are talking to the only person that had been working on it lately :) Annie 15:20, 28 December 2018 (EST)

Founders House Publishing LLC

If you could let Fixer know that "Founders House Publishing LLC" should be "Founders House Publishing", it would be appreciated. Thanks. -- JLaTondre (talk) 19:26, 22 December 2018 (EST)

Done -- thanks! Ahasuerus 19:45, 22 December 2018 (EST)

Data base design

Hello Ahasuerus,

I'm a new user and have spend some day watching the site and i found interesting this project. Actually I work as a big data developer and o would like to help in the developing of the site, mainly in the data base an data quality issues, as some parts in the relational model i think i could help.

Ator97 16:42, 23 December 2018 (CST, UTC-6)

Welcome to the ISFDB! Any and all help is appreciated -- pull up a chair and make yourself comfortable :-) The following development-related pages and page sections are up to date:

• Database Schema
• Development#Software_Development_Process
• Development/Recent Patches
• the list of outstanding Feature Requests
• Development#Development_Process
• ISFDB Build Process
• ISFDB_Downloads#Installation_and_Setup

ISFDB:Current_events#Roadmap_2017 reflects the priority of requested major features as of February 2017. It's still mostly valid although some recent requests have been given higher priority for various reasons.

Once you have had a chance to review the listed Wiki and SourceForge pages, please let me know and we can discuss which areas of the system you would like to concentrate on. You can also e-mail me at ahasuerus at email.com if you need help with setting up a development environment or have any other questions. Again, welcome to the ISFDB! Ahasuerus 18:44, 23 December 2018 (EST)

Thanks i'll check the documentation ;)

Ator97 22:03, 23 December 2018 (CST, UTC-6)

A minor bug

Hello,

If a chapbook ends up in a series information somehow and a series number of "0", the series name can be cleaned up easily but the series number is greyed out and you cannot save the chapbook in any way or form to fix it. The only way I discovered was a temporary change of type on a first edit and then a second save to chapbook with all values cleared out. I am not sure if it happens for all chapbooks or only when the series number is "0" but thought I should mention it. Annie 16:41, 25 December 2018 (EST)

Bug 718 has been created -- thanks! Ahasuerus 16:50, 25 December 2018 (EST)

Should be fixed now. User:Santa Claus 17:17, 25 December 2018 (EST)

Working overtime on Christmas? :) Cannot test (fixed the one that had the problem last night) but thanks for the fast reaction :) Annie 17:19, 25 December 2018 (EST)

Santa Claus is a notorious slacker -- he works just 24 hours a year! Ahasuerus 17:25, 25 December 2018 (EST)

Fixer: Publisher request

Hello,

When you have a chance, can you ask Fixer what he has in the queue from the publisher "The Wild Rose Press"? Caught an ISBN in one of his lists and we are missing a lot of later books in series and even complete series and authors - might as well work through them and clear them - and working with his list as a start makes it easier :) Annie 16:55, 27 December 2018 (EST)

They are a very active romance publisher. Many (but not all) of their authors have not been published by other publishers, so Fixer auto-assigns them to Queue 3. The ISBNs whose authors are in the database get manually assigned to Queue 1 (if they have been published by more prominent publishers) or to Queue 2.

At the moment there are 65 Wild Prose Press ISBNs in Queue 2 and 271 ISBNs in Queue 3. If you would like to work on them, I could ask Fixer to submit them directly to the main queue or to create a separate "Fixer Public" sub-page. I could also separate Queue 2 and Queue 3 ISBNs if desired. How would you like to approach it? Ahasuerus 17:09, 27 December 2018 (EST)

Let's get the Queue 2 ones into the main queue for now and add the Queue-3 ones into a Fixer Public page - way too many for the main queue... Annie 17:21, 27 December 2018 (EST)

Why don't I submit the Queue 2 ones first. Once you process them, you may be able to request a subset based on some pattern that we are not aware of yet. Ahasuerus 17:27, 27 December 2018 (EST)

Sounds like a plan. Annie 17:42, 27 December 2018 (EST)

The good news is that the e-books have ISBNs so Fixer probably catches them; the audiobooks are a different story - I tend to add all formats when I am around anyway. Annie 17:21, 27 December 2018 (EST)

Oops! I forgot to check the table where Fixer stores ASIN-only pubs. It contains additional 295 Wild Rose Press ASINs. Next I'll check the ISBNs that Fixer has auto-suspended based on his algorithms. (This is a great example of why Fixer's database needs to be reorganized...) Ahasuerus 17:31, 27 December 2018 (EST)

599 additional ISBNs! Ahasuerus 17:37, 27 December 2018 (EST)

Guess I am working on the Wild Rose books for awhile then. At least these do not add to my "I really want to read this" pile. :) Annie 17:42, 27 December 2018 (EST)

Any chance to automatically hold them for me? (if not, I will put them on hold manually) :) Annie 17:21, 27 December 2018 (EST)

It's not something that the software currently supports, but I think it's a great idea. I'll go ahead and create an FR. Ahasuerus 17:34, 27 December 2018 (EST)

Or post from the name of a moderator (that is not you - I know it can post from your name) - which kinda automatically puts it on hold for that moderator as noone else can touch it. Annie 17:42, 27 December 2018 (EST)

It's possible, but would they want their manual submissions conflated with Fixer's submissions created on their behalf? Also, a Fixer submission held by a moderator can be "unheld", which you can't do with moderator-originated submissions. Ahasuerus 17:49, 27 December 2018 (EST)

I would not mind when working on Fixer's queues - but then my submission queue is very weird anyway. Cannot say about anyone else. Automatic hold is probably better but either works.Annie 18:06, 27 December 2018 (EST)

(unindent) Fixer has submitted 65 Queue 2 ISBNs. Ahasuerus 17:56, 27 December 2018 (EST)

Thanks! Annie 18:06, 27 December 2018 (EST)

(unindent) These are done. No useful patterns except that it is much easier to work author by author (I kept chasing series and editions for each book Fixer had in that queue). How hard will it be to combine the output of the 3 remaining groups (Q3, the ASIN only and the suspended ones into a single list and order by author name? If not feasible, then order by author inside of each group and just post all 3 lists? I am not sure what will work better here - Fixer posting in batches of 50 or so or just dumping the list on the Public page. Annie 18:22, 28 December 2018 (EST)

None of it would be easy, I am afraid. The three "queues" are based on different pseudo-tables with different table layout. What's worse, they are accessed using different programming languages. I'd have to write a custom script to extract authors for this publisher and then group ISBNs/ASINs by author. Let me think about the best way to approach this issue... Ahasuerus 18:39, 28 December 2018 (EST)

Just dump the 3 lists into a Public List (or 3) - I can split and group on my side and work based off that. Annie 18:55, 28 December 2018 (EST)

OK, let me see what I can do. At this time there is no way to dump "suspended" or ASIN-only pubs to Wiki-compatible files, so I'll need to do it in steps. First I will create a script to move the "suspended" ISBNs for this publisher to Queue 3. At that point I should be able to create a Wiki dump. Then I will implement the ability to create Wiki dumps for ASIN-only pubs. It may take some time.

P.S. See what I meant when I said that Fixer's internals needed to be rewritten? :-\ Ahasuerus 19:08, 28 December 2018 (EST)

Ah, I see... How about a flat list that lists the ASINs/ISBNs only? I do not need it much fancier than that technically speaking... I can grab the rest from Amazon). If not feasible, let's start with Q3 for now then? Annie 19:51, 28 December 2018 (EST)

Let me do a quick scan of Queue 3 ISBNs. Some of them were assigned to Queue 3 a while back. Perhaps their authors are already in the ISFDB. Give me a few minutes... Ahasuerus 20:02, 28 December 2018 (EST)

OK, a few dozen ISBNs have been promoted to Queue 2 and then submitted on Fixer's behalf. 3 or 4 are no longer listed by Amazon, so their links will be dead. Unfortunately, it's not always clear whether an ISBN was never released or whether it was expunged after publication. Ahasuerus 20:14, 28 December 2018 (EST)

Oh, see, Fixer is offering end of year promotions to some poor ISBNs :) Got them all on hold - will work through them :) If something looks like vaporware and I cannot find it anywhere at all, I usually consider it not-published - especially with anything from the last decade or so. It is not that easy for things to disappear anymore... Thanks! Annie 20:29, 28 December 2018 (EST)

I find that it depends on each site's policies. Amazon used to keep ISBNs and ASINs forever, they just displayed any cancelled or superseded ones as "unavailable", "out of print", etc. Lately, however, they started removing unavailable ISBN/ASIN records. For example, consider Young Again in Another World: Volume 3 and its brethren, which were unpublished shortly after the publication of Volume 2 -- see the Note field for links. Their Amazon pages are now blank, but Goodreads still has the series on file, including the ASINs.

The old way was problematic because they displayed cancelled ISBNs which were never published. (It was a problem for bibliographers, but it was a bigger problem for authors.) Unfortunately, the new way is even more problematic from our POV because the pages of superseded ISBNs can disappear. You just can't win... Ahasuerus 21:05, 28 December 2018 (EST)

True - but between archived pages, GR and blogs, chances are that a really published book from the last decade will still be mentioned somewhere. It's not perfect but then... if we miss a few books, we are still better off than missing a thousand or so from the publisher. So... we do what we can. Annie 21:12, 28 December 2018 (EST)

(unindent) After running more in-depth queries against the "suspended" pseudo-table, I realized that there was a flaw in my original search logic. Suspended, rejected and submitted ISBNs share they same table and my selection criteria were off. The "599 additions" that I mentioned earlier were actually 4 rejected and 595 submitted ISBNs, so they are off the table. (Yes, Fixer, I know what you are going to say about carbon-based lifeforms...)

This leaves us with 342 ISBNs in Queue 3 and 295 ISBN-less ASINs. Tomorrow I will create a custom query to find all ISBNs and ASINs for this publisher and group them by author. It will be handy to have around. Ahasuerus 23:14, 28 December 2018 (EST)

Not so bad then :) Plus by the time we run them, some of those will already be in the DB - there will be matches to what I am adding manually. And the query will be handy because there will be more publishers that need their titles added. Thanks for looking into it (and tell Fixer he is not getting a bonus this year if he keeps being so cranky with carbon-based lifeforms). :) Annie 23:21, 28 December 2018 (EST)

The prices formats again

Hello,

I thought that we have a report that catches things like 23 € (the space), 0.99€, 18,50€ and €18,50 (note the comma in the last two)? I remember you enhancing it to check for €? Annie 19:01, 27 December 2018 (EST)

SVN check-in 122 added the following check:

• p.pub_price like concat('%',CHAR(0x80),' ','%')

so the change was limited to "spaces following the Euro sign". I see that we also have 33 pubs with the patterns that you list. I am going to add them to the selection logic. Thanks! Ahasuerus 19:27, 27 December 2018 (EST)

Done. The data will become available in about 5 hours. Ahasuerus 20:15, 27 December 2018 (EST)

I am manually clearing the ones I found so nothing should ping overnight (unless I missed one or two) but good to keep an eye for those going forward :) We do strip trailing spaces when the price is saved, right? Thanks! Annie 20:18, 27 December 2018 (EST)

That's right. All leading and trailing spaces (as well as double spaces and lots of other things) are stripped when submissions are created. Ahasuerus 20:21, 27 December 2018 (EST)

Thought so but wanted to make sure - realized I was leaving a trailing one in some cases and did not want to go reedit if not needed (and recheck where it happened). Thanks! Annie 20:26, 27 December 2018 (EST)