ISFDB:Community Portal/Archive/Archive31

This is an archive page for the Community Portal. Please do not edit the contents. To start a new discussion, please click here.
This archive includes discussions from

Archive Quick Links
Archives of old discussions from the Community Portal.

1 · 2 · 3 · 4 · 5 · 6 · 7 · 8 · 9 · 10 · 11 · 12 · 13 · 14 · 15 · 16 · 17 · 18 · 19 · 20 · 21 · 22 · 23 · 24 · 25 · 26 · 27 · 28 · 29 · 30 · 31 · 32 · 33 · 34 · 35 · 36 · 37 · 38 · 39 · 40 · 41 · 42 · 43 · 44 · 45 · 46 · 47 · 48 · 49 · 50 · 51 · 52 · 53 · 54 · 55

Expanded archive listing

Archive of messages from September - December 2013

Double submissions

About a week ago I noticed that submissions I'd made had been doubled in the queue. I thought my mouse was going bad, and was making two submissions with one click. So I bought a new mouse. But it didn't solve the problem. I went to the Windows Control Panel to change the mouse click speed, and it still didn't affect anything. There were still double submissions being made occasionally. So tonight I looked at the Recent Rejects list and notice that there are more than the usual number of submissions being rejected by moderators of their own submissions. And there is a higher than normal number of submissions by moderators that are remaining in the queue, submissions that don't make any change to the record. (Some moderators must not regularly check the queue, especially those that only moderator their own submissions.) Does this mean other people are having the same problem? Is the system "hiccuping" and creating duplicate submissions? Mhhutchins 03:10, 4 September 2013 (UTC)

Hm, I don't think I have seen any duplicate submissions, but if it's happening to even one user, it's bad news. I'll take a closer look tomorrow to see if there are duplicate date/time stamps or any other indications of what may be going on. Thanks for reporting it! Ahasuerus 04:18, 4 September 2013 (UTC)

A submission to update a record was accepted:

2013-09-03 18:35:04 / 2218726 - PubUpdate / Bluesman / Bluesman / The Guardsman

A submission that is still sitting in the queue which shows no change to the record (the same publication record that was updated 5 seconds before):

2218727 / N / PubUpdate / 2013-09-03 18:35:09 / Bluesman (Talk) / The Guardsman

Mhhutchins 04:29, 4 September 2013 (UTC)

I've ran into some performance problems at my end (the server seeming slow to respond) which caused "double" submissions due to bad manipulations from me. Hauck 05:00, 4 September 2013 (UTC)

Checking the contents of the full "submissions" table, I see a number of empty submissions created a few seconds after a related submission, e.g. Bluesman's previously referenced submission 2218727, which was created 5 seconds after submission 2218726. There is nothing in the database explaining why or how this may be happening, so I am rather mystified. The good news is that these empty submissions do not appear to be very common and they seem to be harmless, but I don't have the exact numbers yet. I'll need to run more complex queries to get the stats and determine if it's been happening more frequently lately. Ahasuerus 03:37, 5 September 2013 (UTC)

OK, here are the counts of empty submissions over the last 4 years:

2010: 11 per month
2011: 13 per month
2012: 22 per month
2013-01: 12
2013-02: 23
2013-03: 19
2013-04: 15
2013-05: 18
2013-06: 18
2013-07: 18
2013-08: 47
2013-09: 9 (first 4 days of the month)

So the spike in August wasn't dramatic, but it was nonetheless noticeable, and September seems to be in line with the "new normal" so far. Also, I see that all of these empty submissions are Pub Updates, which may be relevant. 04:13, 5 September 2013 (UTC)

On the surface they may "seem to be harmless", but yesterday I accepted a submission which added a dozen titles to a record (from a new editor), and then without noticing it (assuming it was an update to another issue of the same periodical) accepted the subsequent submission, only to learn it was a duplicate. This forced me to remove the duplicate titles and then merge the publess titles with the ones already in the database. This same scenario happened last week as well. As someone who does a lot of moderating of other editors' submissions, I suppose I would notice this more than other moderators. But I still don't see how other moderators aren't seeing this with their own submissions (or they haven't read this posting.) It makes me nervous every time I hit the Submit button for my own submissions, and the Accept button when moderating others. Knowing that it only happens on Pub Updates relieves some of the anxiety, so at least I only have to keep an eye on those types of submissions. And who do I see about reimbursing me $10 for a new mouse I didn't need? :) (BTW, I would call a jump of more than 160% a dramatic one, and 9 times in 4 days becomes 68 for the entire month, making it a 375% jump, from an average of 18 in the three previous months. Seems pretty dramatic!) Mhhutchins 04:41, 5 September 2013 (UTC)

Wait, that's a different and potentially more damaging scenario! I was only counting submissions that were literally empty, i.e. there was no data in the submission. What you are describing is a duplicate submission, a beast of a different color. Ahasuerus 05:04, 5 September 2013 (UTC)

I used the phrase "duplicate submission" in my original post, not knowing another way to describe it. :) Mhhutchins 05:45, 5 September 2013 (UTC)

Do you happen to remember the user name of the editor who created the submission? I could look up his/her submissions in the database and see if anything stands out. Ahasuerus 05:04, 5 September 2013 (UTC)

It was a submission by ArtZub, who is updating issues of the SFRA Review. The duplicate happened around 2013-09-03 22:16:57 (server time), because that was the time I noticed a second submission was a duplicate and rejected it. I can't know when it was actually submitted. Look around that time for submissions by ArtZub. If you look at the Recent Rejects list, you'll find many moderators rejecting their own submissions, and I think that is strongly connected to the problem. I've never rejected as many of my own submissions as I have in the past couple of weeks, probably more than in my last two or three years combined (or it seems like it.) There would be twice as many if I didn't go ahead and accept those duplicate submissions that don't do harm to the record. Mhhutchins 05:36, 5 September 2013 (UTC)

Thanks, I'll take a look. Ahasuerus 05:42, 5 September 2013 (UTC)

Also, I wonder if what we are seeing (in at least some cases) may be related to the sporadic server performance over the last few weeks. If an editor clicks the "Submit" button and the server appears to hang, then s/he may be tempted to go back to the previous page and click "Submit" again, thus creating a duplicate submission. Ahasuerus 05:04, 5 September 2013 (UTC)

That's possible, but I don't think so, at least not in my case. I never go back and do a second "Submit" if there's a delay in the server. I wait until it's accepted, no matter how long the wait. Mhhutchins 05:36, 5 September 2013 (UTC)

There is definitely something wrong as I am getting duplicated submissions and there seems to be nothing wrong with the server hanging my end. --Chris J 05:23, 5 September 2013 (UTC)

For about a year I have had frequent doubling of submissions, about one every couple of weeks on average. I've always been puzzled as to how it happens but I've always put it down to a server glitch. If I later see it in the queue I reject it, marked "Duplicate". PeteYoung 05:38, 5 September 2013 (UTC)

What's puzzling is that I am yet to see a duplicate submission even though I have processed over 7,000 of them in the last two months. Something must be different here, but I can't think of anything that could account for the difference. Ahasuerus 05:42, 5 September 2013 (UTC)

Aren't most of your moderations adding new pub records? I believe you handle much more of that type than practically anyone. How many submissions (of any type) have Chris J and I processed in the same time period? Can you break it down by submission type? Mhhutchins 05:49, 5 September 2013 (UTC)

Well, they usually start out as NewPub submissions, but many of them necessitate PubUpdates, including adding contents element. This year I have done 1,201 AuthorUpdates, 6,541 PubUpdates and 5,295 TitleUpdates, so it's not all NewPubs.

The only type of edits that I don't do very often is when you pull up an existing collection/anthology and change its content. I can't think of any reason why it would be any different than other types of submissions, but it's something to consider. Do other moderators find that their duplicate submissions tend to occur after changing the Contents section of a pub?

Also, we may want to ask other active moderators if they have seen this problem and, if they have, when it started to occur for them. So far we have two data points: Pete's "about a year [ago]" and the increase in the number of empty submissions starting in August. Ahasuerus 06:09, 5 September 2013 (UTC)

How many of those 47 empty submissions in August occurred in the last week of the month? That's when I noticed the duplicate problem. Mhhutchins 06:42, 5 September 2013 (UTC)

Here is the breakdown of the empty submissions for August and the first 3 days of September:

08-1-4: 0
08-5: 1
08-6-9: 0
08-10: 1
08-11: 0
08-12: 2
13-15: 0
08-16: 1
08-17-18: 0
08-19: 1
08-20: 1
08-21: 1
08-22: 3
08-23: 2
08-24-25: 0
08-26: 12
08-27: 5
08-28: 5
08-29: 7
08-30: 5
08-31: 0
09-01: 1
09-02: 3
09-03: 5

So yes, it looks like the pattern changed on August 26, which all but excludes the core ISFDB software as the culprit since we haven't made any software changes since late July. I will poke around various server logs (Web server, database server, etc) and see if I can find anything. Ahasuerus 20:29, 5 September 2013 (UTC)

Can you also check to see if there's been a significant increase in moderators rejecting their own submissions for the past couple of weeks or so? Mhhutchins 06:42, 5 September 2013 (UTC)

Here is how many submissions have been rejected by the submitter, which includes self-rejections by moderators and cancellations by non-moderators: 157 in April, 162 in May, 106 in June, 141 in July, 142 in August, 15 in the first 3 days of September. For moderator self-rejections only the numbers are: April - 96, May - 95, June - 53, July - 80, August - 107, September - 14. Ahasuerus 20:29, 5 September 2013 (UTC)

Most of my doubles have been new submissions and it seemed to me to start about the time when the tick buttons (for merging etc) were enlarged. --Chris J 07:22, 5 September 2013 (UTC)

I've not noticed any enlarged buttons. On what screen do they appear? Mhhutchins 07:33, 5 September 2013 (UTC)

When you merge titles, art etc. the buttons you tick to merge on the left used to be small, now they are larger. I thought someone had made a good improvement. This happened at the same time as I started to get double submissions. Also when I submit an item: go to another window to approve it; go back to the original window and reload the page everything is find, but when I try to back to a previous page nothing happens. It stays on the same page. All of this happened about the same time. I do not think it is my browser. --Chris J 07:55, 5 September 2013 (UTC)

I do merges every day, and have noticed no changes in the size of the button. Just checked again, and there is no difference that I can see in the size than they've always been. But I did notice the oddity you describe when a new window is opened from a link in a publication record, but in my case, it's only when new windows are opened from links to Amazon. I thought it was something Amazon was doing to prevent you from going back to the previous page. Strange. Mhhutchins 17:51, 5 September 2013 (UTC)

Out of a total of 16 I've marked as "duplicates" since mid-July 2012 (and there may be several more amongst those I didn't bother to annotate), I have had 4 doubles that appeared when creating a new publication, the most recent being 29 June. My most recent duplicated submission was 28 August. PeteYoung 08:29, 5 September 2013 (UTC)

I have stopped "fixing" the duplicate submissions and am leaving them in the queue to be investigated. Mhhutchins 21:47, 5 September 2013 (UTC)

Depending on how the site's hosting is set up internally, it's possible some component along the way retries a request if the handling for it (our combination of python processing and database access) doesn't complete quickly enough. You wouldn't notice on a simple page access or even a search submission, since you'd get the requested results in the end, but an update would either submit the same modification twice (making the second one look "empty" -- I know I have seen a few of those lately; I normally accept them instead of rejecting) or, in the case where something new is being created (a pub or contents), you get a doubled one. Note even the submission I have on hold currently.... From an Editor's point of view, there would be a longer-than-usual pause, and that's about it. --MartyD 03:00, 6 September 2013 (UTC)

(back in business after an internet outage) Yes, that would seem to be the most likely explanation. I don't see anything in the logs that may suggest other scenarios. There are some logging settings that I can tweak to collect more data about what's going on internally, but it may take some time to get them right. Unfortunately, this is not my area of expertise, so I am trying to learn new tricks here... Ahasuerus 16:55, 6 September 2013 (UTC)

(unindent) Any progress on this front? It's still happening, today even more than normal. Just take a look at the recent rejects list. Mhhutchins 20:52, 18 September 2013 (UTC)

I am afraid not :-( I still can't recreate the problem, in part because it never (and I mean never) happens to me for some reason. However, in the process of looking into it I think I have found a way to make the Recent Rejects list to come up faster. It's not perfect because we are using a very old version of the MySQL database, but it should be faster -- please let me know if you see the difference.

P.S. Not that it should matter, at least in theory, but do you, by chance, use multiple windows when editing ISFDB? Ahasuerus 23:18, 18 September 2013 (UTC)

I have multiple tabs open but I don't change from one to the other during submissions. After I've made a submission, I click on the "Moderate Submission" link, which takes me directly to the submission's moderation screen, I click on the "Approve" link, and get the SQL update report. Sometime later I'll check the queue (some us of moderators still do check the queue every now and then!) and see that there is a submission identical to the one I just approved, but there are no changes to be made in the record because it already matches the data that I just approved. Looking at the Reject list I see Chavey has a lot of self-cancellations. Also, look at Swfritter's submission that's currently in the queue:

2227305 N PubUpdate 2013-09-18 12:12:22 Swfritter (Talk) Double or Nothing

It's a pub update, but shows no change to the record. Now look at this submission he accepted:

2013-09-18 12:12:18 2227304 - PubUpdate Swfritter Swfritter Double or Nothing

Four seconds apart! I'm telling you it's happening to other moderators, just not as frequently as me because I'm making more submissions, and I monitor the submission queue more than others. Mhhutchins 23:36, 18 September 2013 (UTC)

Oh, I realize that it's happening to multiple editors, I am just trying to figure out what accounts for the difference.

Let me try another angle: when you click the Submit button and the very next Web page seems to hang half way through its displayed SQL statements (as it sometimes does due to server performance issues), do you hit the Refresh button or F5? That could cause the form to resubmit and a duplicate submission to be created. (Seems unlikely, but hey, it doesn't hurt to ask :-) Ahasuerus 00:32, 19 September 2013 (UTC)

Nope, I never refresh, or go back and resubmit. It's always one click and then I wait...even if it takes a minute to get a response. That's why I first I thought my mouse was double-clicking and making two submissions, but the new mouse made no difference. Sometimes when there are multiple changes in a record (like adding 40 reviews to an issue of Locus), the SQL response stops before it's completely loaded. (Same as a very large wiki page which loads in chunks when the server is slow.) But eventually it finishes, and I never try to stop it before it's loaded or try to refresh the screen. I've been around long enough to know that refreshing the screen doesn't speed things up. Mhhutchins 00:52, 19 September 2013 (UTC)

Oh well, it was worth a shot :-) Ahasuerus 02:14, 19 September 2013 (UTC)

Isn't there a way to look at all submissions to see if a second one is made within seconds of a previous one, from the same editor, and is identical? Are there certain submission types where it happens more often? Mhhutchins 00:52, 19 September 2013 (UTC)

It's possible, but it's not a straightforward process. The body of each submission is stored in a single field in XML format, e.g.:

<?xml version="1.0" encoding="iso-8859-1" ?>
<IsfdbSubmission>
 <TitleMerge>
   <KeepId>8044</KeepId>
   <DropId>170676</DropId>
   <Submitter>Alvonruff</Submitter>
   <Subject>Foreigner</Subject>
   <Title>8044</Title>
   <Year>8044</Year>
   <Series>8044</Series>
   <Seriesnum>8044</Seriesnum>
 </TitleMerge>
</IsfdbSubmission>

so it has to be parsed before we compare it with other submissions. And since we have a few dozen submission types, the parsing can get tricky.

In addition, my original review of known duplicates about a week+ ago suggested that there wasn't a great deal to be learned from looking at the submissions. All duplicate submissions (and in one case a triplicate submission) were identical and covered a variety of submission types, e.g. Remove Title and Pub Update, so it seemed to be a random occurrence unrelated to the submission type. I suspect that Marty's suggestion re: examining certain log files is likely to be more fruitful, but it's a bit outside of my area of expertise and I still need to learn more about them before I can make any progress. Oh well... Ahasuerus 02:14, 19 September 2013 (UTC)

Forthcoming publication

So now we're creating records for a book one year in advance of publication? Mhhutchins 04:22, 4 September 2013 (UTC)

That would seem way way too early given how many times things can change between now and 2014-09. Ahasuerus 05:09, 4 September 2013 (UTC)

Colons in titles

If I understand the ISFDB standards for titles, colons in titles should not be preceded by a space. Hence titles such as Elvene : The Kiri Myth of Ocean Woman, Necroscope : Wampyri, Tomadachi : The Edge of the World, Job : Une comédie de justice, etc. are violating those standards. Am I wrong about those standards? Even if we normally use that standard, do we ignore that (as we do capitalization "standards") for non-English languages? For example, I see an awful lot of French publications that use the extra space; is that a French "standard" that we use for French titles, or is it just that a French-entering editor is used to seeing titles that way and hasn't "converted" them to the English-language norm? Chavey 01:29, 5 September 2013 (UTC)

It is the french typographic code (see here) which demands to have an unsecable space between the last word and the colon, idem between a word and an interrogation mark), as in this pub where the space and the colon are integral to the title. When entering subtitles (even in French) I try to use the ISFDB de-facto standard (no space) but in this case it would be wrong to modify the data as I've thought that the non-english titles were to be entered with their diverse idiosyncrasies and most importantly (but I'm probably deluded) as they are printed.Hauck 14:54, 5 September 2013 (UTC)

I agree with Hauck in this case. If a work's title includes a colon and that colon is intrinsically part of the title, then the ISFDB title should reflect the form as chosen by the author (and by extension, the publisher), whether that form uses spaces before, after, both or not at all. But when a colon is not present, and only used to separate title from subtitle, there should be no space before it per the ISFDB documented standard. Mhhutchins 18:51, 5 September 2013 (UTC)

I might add that my opinion here is not based on the French typographic code for colon usage, and would apply to all publications regardless of their language. Mhhutchins 18:57, 5 September 2013 (UTC)

I'm not sure if the use of a space before and after a colon is a non-English standard. But I do know it's the standard for ISBD ("International Standard Bibliographic Description") used by librarians in creating MARC (MAchine Readable Cataloging) records. But it's not the ISFDB standard. The colon is used here to offset a subtitle from a title, or to indicate a change in font in a title, even when the colon isn't present on the publication's title page. (This appears to be the same usage as ISBD, but without the extra space.) I can not say if that standard is documented in our help. Mhhutchins 02:08, 5 September 2013 (UTC)

(after edit conflict) I was just about to say that libraries separate titles and subtitles with " : " and some of our older (ca. 2000-2004) data comes from library catalogs. It's also possible that some of our editors started using the same standard after getting exposed to these records. Ahasuerus 02:14, 5 September 2013 (UTC)

(After edit conflict with Ahasuerus) The Help page for Titles, a template portion of "NewPub" Help and various other help pages, says:

"Subtitles. If the title has a subtitle, enter it, with a colon and a space used to separate the title from the subtitle. For example, the 1986 edition of George MacDonald's "Lilith" has "Lilith" on the title page, and below that, in a smaller font, "A Romance". This should be entered as "Lilith: A Romance". It is sometimes a judgement call as to whether a change of font or a colon indicates a subtitle or just some creative license on the part of the typesetter. If in doubt, take your best guess and document the guess on the publication's wiki page."

And that seems to me to be fairly precise about this as a standard. But I can certainly imagine the possibility that I am misreading it. Chavey 02:20, 5 September 2013 (UTC)

You're not. It's pretty clear. So now we know the standard is documented. Someone might say it doesn't specifically ban the pre-space, but that would be nitpicking. Mhhutchins 02:30, 5 September 2013 (UTC)

Then unless there is an objection here in the next 4 days, I announce my intent to correct those 320 titles & 419 publications to meet this standard. I will not be able to post notes to the verifier pages of all of those publications, and this will need to act in place of those notes. Chavey 04:35, 5 September 2013 (UTC)

Be prepared to be called anglocentric. Mhhutchins 04:48, 5 September 2013 (UTC)

Well, we are not using a US- or UK-based standard in this case, so we are just being ISFDB-centric :-) Ahasuerus 05:05, 5 September 2013 (UTC)

That fact never stopped the accusations before. Mhhutchins 05:51, 5 September 2013 (UTC)

Hope my objections were heard. If not, I'll probably reverse such "corrections" for french titles when I'll encounter them.Hauck 05:55, 15 September 2013 (UTC)

Even if they're not French, as long as the colon is part of the title, (and not just a user-generated separator), feel free to reverse the correction. I'll correct any that I find. Mhhutchins 06:21, 15 September 2013 (UTC)

Thanks, that was my thinking : titles as printed, subtitles without colon (bonus : it can allow, for french publications, to differentiate the two). I don't know if the intented "corrections" were made, but if it was the case, the method would not be very pleasing. Hauck 06:40, 15 September 2013 (UTC)

Brief editing outage

I will be restarting the Web server and the database in the next few minutes. I don't know if it may help with the reported "duplicate submissions" problem, but there is no harm in doing it. There will be a brief outage starting around 12:45am server time. Ahasuerus 05:44, 5 September 2013 (UTC)

Done. Ahasuerus 05:46, 5 September 2013 (UTC)

Can moderators do what they want?

The more I contribute to ISFDB the more I ask myself if actually the moderators of ISFDB follow any rules at all:

Anytime I fix obvious errors my changes are rejected as I did not contact somebody to ask before, but my own entries get modified without even telling me. This includes even removal of sub-titles for no good reason. Moderators know everything better?
When I ask why e.g. SF Anthology does not follow the guideline, that the book name is what is written in the book I get told that the information is instead stored in the series part SF Anthologie. When I do the same method for e.g. this Ullstein series the series entry gets removed by the same moderator.
Assignment of publication series is more or less random. What's written on the book seems to be totally unimportant: While e.g. this book clearly has the same logo telling it to be part of Ullstein 2000 series it's moved to another series for no good reason. No notes in the series description indicate any special insight, so here only the magic wisdom of the moderator counts?

I needed to learn a lot to get proper entries, but for some fields it is rather random whether they are right or not. For one book its right, for the other not.

Do I misunderstand something here - I thought the idea is to get an objective database of stories and books and not to follow dictatorship of a few individuals. --Stoecker 21:14, 8 September 2013 (UTC)

To answer the question "Can moderators do what they want?": Yes, they have the ability to do pretty much anything they want. A better question would be: "Should moderators do what they want?" No, they have to follow the documented standards, some of which may be ambiguously documented. We try to keep ambiguity out of the standards, but being humans, and with different minds, there's a possibility that one person's interpretation differs from another's. That's why we have a Rules and Standards Discussion page to iron out such differences. The last question: "Do moderators do what they want?" No, there are too many other moderators around to make sure one moderator doesn't start making up his own rules.

About your problems: whenever you have a disagreement with a moderator over the outcome of a submission, bring the issue immediately to the attention of the other moderators on the ISFDB:Moderator_noticeboard. This allows you to present your view and and the moderator to present theirs.

Concerning the situation with series in titles: you have every right to enter the title field of a publication record EXACTLY as it's presented on the pub's title page, including the series data if you wish. BUT, that doesn't mean you can control how the title field of the title record is given. It is redundant to give series data in the title field of a title record when the series data is given in the series field of the title record.

Another conflict appears to be how to determine when a title is in a title series, or a publication is in a publication series. Continental European publications present a situation which many British and US publications don't. Their concept of the publication series goes further than almost all Anglo and American publishers. (I believe DAW Books is the only prominent US publisher that has a publication series similar to those of European publishers.) So US and UK moderators may have trouble when situations like this arise. I usually allow those editors with the book-in-hand make such decisions, except in cases where there is an obvious error.

If you believe assignment of a publication series is "more or less random", please speak out about the specific instance. Blanket statements don't help solve the problem. Most of the time it's just a failure in communication. But if the moderator you're dealing with is inflexible (even reasonably so) or is unable to provide you with the documented standard, don't hesitate to bring the problem to other moderators. Mhhutchins 00:02, 9 September 2013 (UTC)

What is totally strange remains: In my examples the Alpers anthology consists of 6 books and here for unknown reason parts of the title get moved into the series (in contrast to all German libraries, who do otherwise). But for the Ullstein SF-Stories and Titan series the same is untrue and the series gets removed. There is no difference between these. If one is a series, the other must be as well. If one is not, the other also must not be. I can live with both. What I don't understand is that one magically is different from the other for not understandable reason. I own the 2 remaining Alpers books, but wont add them, as the title as I would enter it surely gets changed and the current entry style is total nonsense in my eyes and I wont add that. Result of discussion with Stonecreek gave me no reason to think otherwise. --Stoecker 20:04, 14 September 2013 (UTC)

What Michael said. Also, keep in mind that some of these issues have long and complicated histories going back literally years and the details can be hard to explain in a brief communication. In general, the recurring theme in many Rules discussions is the (probably inevitable) tension between:

our desire to record information as it appears in publications
our attempts to limit the number of author, title, publisher, etc records by standardizing things like capitalization
our software's limitations

Ahasuerus 01:19, 9 September 2013 (UTC)

I have no problems with attempts to standardize. Fine. But when there are rules and you try to follow them and the result then is, that again and again somehow these rules never are valid for whatever you do, it gets strange. --Stoecker 19:35, 14 September 2013 (UTC)

Since I think that I am the moderator in question, let me state again that it would have been better to first ask a moderator about changes and additions accustomed with German (or European) publishing. I have been on vacation for several weeks, but still there are others who would have been readily available. I could't find out who made the changes, sorry for that.

The addition of a title series named Science-Fiction-Stories wasn't a thoughtful step, because of the included titles, discussed before here.

As I explained before in an argument, we don't assign data by 'logos' but by the statement in the publication. On p. 1 of the new numerated series is only 'Science Fiction'. In the argument with editor Deagol linked to above I also layed out the reasons of establishing the standard for the pub. series 'Ullstein Science Fiction' with #31001. I found out after my vacation that somebody had changed this series to 'Ullstein 2000' without asking me, among them for two books primary verified by others (one of the two being me). At this point I changed it back and changed the other publications from that time period also. I should have asked you about the reasons of not asking first but I had no evidence that you did it. If you did it, it would have been necessary to ask the primary verifier first. If you didn't, please take my apologies for not informing you directly. (But, did you do it?) There also was the addition of the pub. series 'Ullstein 2000' to a book that wasn't published as part of the series (now changed to correct denomination). I do think that this was done by the same person (not necessarily you). I know it takes some time to research or ask, but it is much better than to implement mistakes that possibly endure for a long time. I also can only recommend again to add notes what exactly is stated in a given publication, see here for an example. It not only gives you the information you need but also helps to improve the quality of our database. Stonecreek 09:24, 9 September 2013 (UTC)

If the common design of all the books doesn't qualify as a series identification, then stop using publications series at all. E.g. "Ullstein 2000" is nowhere mentioned in my books except the common design and the numbering. The publication series and book series assignment is totally random and not understandable. For me it is pure dictatorship. The series descriptions (usually empty) itself holds no key to the series assignment, the logos shouldn't be used, the texts in the books don't match the series names or mostly don't exist. When there are clear texts and I use them the entries get removed due to some obscure undocumented agreement. When I use the obvious design it gets changed due to some other obscure rules. --Stoecker 19:35, 14 September 2013 (UTC)

I can only confirm what the previous moderators said, mainly that the managing of the publications series (which are endemic in some european countries) is always problematic as publishers are not always consistent (one series can start with a name then take another slightly different but still be essentially the same). So, in order to speak the same language and avoid too many occurences, there is always some standardization to be achieved, keeping in mind that the (unique) result of this process can be "forced" on other contributors. As Christian is a thoughtful moderator there is probably only a misunderstanding between you. Hauck 14:12, 9 September 2013 (UTC)

If there is a common assignment policy, then at least document it in the series description (i.e. the only place where it can be found). If there is superior knowledge stated there, than a short look can remove any doubt. But for now for a new entry you check all the stuff for half an hour only to be overwritten by a moderator with some obscure methodology usually neither explained or when explained being totally non-understandable. --Stoecker 19:35, 14 September 2013 (UTC)

I agree with Hervé (Hauck) that Christian (Stonecreek) is quite fair and thoughtful in his moderating of other editors' submissions. He is not one that will knock you down and roll over you to make you agree with his position. As both Ahasuerus and Hervé said, sometimes we are forced to "standardize" the data fields of some publications in order for them to be found in searches of various fields, including the publisher and publication series field. This bending of the rule to enter data "as stated" can be frustrating to some new editors, but that is one of the underlying faults of the database and its software design, not the moderators'. This can be just as frustrating to moderators as well, but it's one we've learn to live with and together have created a damn fine database. Mhhutchins 16:25, 9 September 2013 (UTC)

While most other moderators I had to deal with seem to adhere to some rules for Stonecreek I can't see that. For me it looks like he is right always and there is always a reason why something is different than for all the other books. It's hard to see common sense. --Stoecker 19:35, 14 September 2013 (UTC)

I also have to add that this is the second time I got carried away by your statement in your Note to others of 'So if you think you know better or want to fix any of these details, feel free to do so.' and that you aren't interested in specific details other than the basic ones. I now have learned that I have to stick to the standard procedure regardless what is stated in an editor's talk page. Again, my apology to you. Stonecreek 16:15, 9 September 2013 (UTC)

I never noticed that statement. You, Stoecker, have effectively given other editors permission to make changes to your verified records without notifying you first. It seems to have been written and posted in a moment of pique, so maybe it isn't your true feelings. I would suggest removing it if you wish to be notified of such changes. Mhhutchins 16:25, 9 September 2013 (UTC)

Well, changing the title of a book by removing a complete subtitle totally isn't something like a little detail. So I can either remove that note and have to care for all or don't be asked for large changes. Hmpf. --Stoecker 19:35, 14 September 2013 (UTC)

The changing of the title of the book in question had to be done with regards to ISFDB standards and is answered here. Let me add that it can be quite helpful to study the help pages for editing the ISFDB before complaining. Stonecreek 13:36, 26 September 2013 (UTC)

First Blood by David Morrell

Our record for First Blood by David Morrell says:

This novel introduced Rambo, the character who became famous in a series of movies beginning with one called, like this book, "First Blood." Although this novel is generally considered an adventure thriller it has clear supernatural or paranormal elements that lead to its classification here as speculative fiction.

As far as I know, there were no supernatural elements in the book and the author says on Facebook that "[m]y writing doesn't have supernatural elements", so I am puzzled by this Note. Is there something here that I am missing or should we change the record to NONGENRE since the book has been reviewed in an SF magazine? Ahasuerus 03:11, 10 September 2013 (UTC)

Never read the thing, but if the author says it's nongenre, we probably should believe him. Maybe the editor who wrote that note was afraid it would be removed from the db otherwise. It's eligible for the db because its author is above the threshold, regardless of where it was reviewed. Mhhutchins 03:27, 10 September 2013 (UTC)

Well, the author claims that all of his books are non-genre. Checking reviews, it would appear that he writes thrillers and psychological horror, which gets him reviewed in SF and horror magazines. Ahasuerus 03:55, 10 September 2013 (UTC)

Having never read any of his work, I assumed he was a genre writer, based on his prominence in the horror field. I always assumed that the ISFDB considered psychological horror to be eligible, thus the inclusion, without question, of the non-supernatural works of Stephen King, Robert Bloch, Joe R. Lansdale, Peter Straub, Dean Koontz, Richard Matheson etc., most of which are not categorized as NONGENRE. Isn't psychological horror a subgenre of horror? Mhhutchins 04:27, 10 September 2013 (UTC)

It's always been my understanding that we only catalog supernatural horror and that psychological horror is excluded because it doesn't have any speculative elements. Whenever I add a work of horror and there is no easy way of telling whether the book contains SF elements, I add a note to that effect so that future editors would know why the book is in ISFDB.

In the meantime, a Usenet poster has added the following commentary:

However Morrell DOES use supernatural themes in his fiction. The short story collection "Black Evening" has a few with a horror tinge as does his novel "Creepers". They're listed accordingly on isfdb.

so it looks like Morrell's blanket statement was a bit too, well, blanket :-) Ahasuerus 04:56, 10 September 2013 (UTC)

Or blankety? Mhhutchins 05:04, 10 September 2013 (UTC)

(unindent) OK, I have changed the title type to NONGENRE and updated the Notes field. Ahasuerus 00:14, 14 September 2013 (UTC)

Minor server patch to support mobile devices (r2013-111)

I have been examining our server logs for the last couple of days, trying to figure out what is causing duplicate submissions. One of the things that I noticed is that we have a lot of errors generated by Apple's mobile (iPad, iPod etc) users. Apparently these devices allow you to "save" Web pages locally and then access them later on like a bookmark. As a side effect, this operation generates an error in our logs because we don't have certain files on our side supporting this process. It's a not a big deal, but it makes the logs harder to use for debugging purposes. I have added the missing files on our side and hopefully it will make debugging a little easier. Also, if you use a mobile device and save ISFDB pages, they will now appear as small ISFDB icons instead of the Apple default. Ahasuerus 04:51, 10 September 2013 (UTC)

Where to discuss use of ISFDB data on other projects?

I'm currently using a local copy of the ISFDB to generate, via Python code, a consolidated Index for the 4 collected volumes of Algis Budrys' book reviews originally published in Galaxy & F&SF.

Glad to hear that the ISFDB data is being used for all kinds of projects! Ahasuerus 21:52, 11 September 2013 (UTC)

I'm almost there but struggling on the last few details, so would like to discuss ISFDB d/b structure, & SQL for retrieval of data. I don't think the Development section of the Wiki is appropriate as that's about the ISFDB's software. IIRC there used to be an offsite online board that discussed such things, but that was a long time ago! Mjcrossuk 17:07, 11 September 2013 (UTC)

Well, we can start the discussion here and take it elsewhere if it gets unwieldy. Shoot! :-) Ahasuerus 21:52, 11 September 2013 (UTC)

OK, I'll post some details later today - it's just after midnight here now. In the meantime, if you want to see the current outputs, they are online at http://michaelcross.me.uk/ajbbm/. Each of the 4 generated pages (Index, Issues, Authors, Titles) has a summary of issues at the foot of the page. Some of those issues are just a SMOP; the main SQL-related one is how to get hold of the type of the reviewed book so I can detect when it's an anthology and thus identify the author(s) as editors in the output pages - I know where the info is, it's how to get it within the already-complex SQL. Mjcrossuk 23:21, 11 September 2013 (UTC)

It took me longer than I expected to put this together so I wasn't ready yesterday.

The titles table can have a number of rows for a particular title that is a book, with different title_type values (eg REVIEW, and NOVEL/ANTHOLOGY/COLLECTION etc).

I'm retrieving reviews by Algis Budrys, so I select rows with title_ttype = 'REVIEW', but I also need the row for a reviewed title that has the other title_ttype value so I can distinguish anthologies from novels, collections etc.

Some titles can have more than one non-REVIEW row, eg those that have been a Novella and then a Novel. You may have multiple, different works with the same title.

The ideal would be to have a query which returned one row per reviewed title, with all the information, with one row per author where there are multiple authors.

If that's not possible, then a rather kludge-y alternative would be to run a query to retrieve all anthology titles and store the results in a Python list, then checking each title as processed from the main query to see if it exists in the anthology list.

Here's my SQL:

-- List of reviews by AJB

CREATE VIEW AJB_Reviews AS
  SELECT title_id, title_title
    FROM titles
   WHERE title_ttype = 'REVIEW'
     AND title_id IN (
               SELECT title_id
       	  FROM canonical_author
		 WHERE author_id = 12
                  AND ca_status = 1);

-- Create 1 view per volume, containing issue details

CREATE VIEW AJB_Galaxy_View AS
  SELECT pub_id, pub_title, pub_tag, pub_year
    FROM pubs
   WHERE pub_tag LIKE 'GAL%'
     AND pub_ctype = 'MAGAZINE'
     AND pub_year BETWEEN '1965-00-00' AND '1971-12-31'
   ORDER BY pub_year;

-- Create one view for each volume, containing review details

CREATE VIEW AJB_Galaxy_Reviews AS
  SELECT *
    FROM AJB_Reviews
   WHERE title_id IN (
    	SELECT title_id
    	  FROM pub_content
    	 WHERE pub_id IN ( SELECT pub_id FROM AJB_Galaxy_View ) );

-- Retrieve reviews for Galaxy, in issue order

SELECT author_canonical, author_lastname, title_title, AR.title_id, AM.pub_id,
       pub_title, pub_tag, pub_year, lpad( pubc_page, 3, '0' ) pubc_page
  FROM AJB_Galaxy_Reviews AR, AJB_Galaxy_View AM, pub_content pc
 WHERE (    pc.title_id = AR.title_id
        AND AM.pub_id = pc.pub_id )
 ORDER BY pub_year, pubc_page, title_id;

Mjcrossuk 23:17, 13 September 2013 (UTC)

I have asked a couple of our SQL heavy hitters to chime in, but in the meantime let me make a comment.

The "publication tag" field (pub_tag) has been deprecated for some time. In addition, it's not always reliable because the field was user-editable for many years. The best way to find all pubs for a given magazine would be to use the series that the magazine is organized as, e.g. here is the issue grid for Galaxy Science Fiction, which shows all Galaxy pubs. If you examine the Python script that creates this page, you will see how the data extraction and sorting process works. By the way, do you want just the reviews published in the US version of Galaxy or do you also want the UK (and French?) reprints? Ahasuerus 01:16, 14 September 2013 (UTC)

I hadn't realised that the pub_tag field had been deprecated, but it does seem to be working for what I'm trying to do; I'm only interested in the issues of Galaxy & F&SF where Budrys' book columns appear rather than the whole span of a magazine. Thus my use of the pub_tag is only looking for 'GAL' and 'FSF' at the start, together with a pub_ctype of 'MAGAZINE' to avoid any anthologies containing stories from the magazines. I will look at the code you mention.

I hadn't thought about whether the non-US editions might contain different reviews, but I suspect that the compilers of the four collected volumes may not have done either! All I'm trying to do is compile a consolidated index for the four collected volumes of Budrys' book reviews columns from Galaxy & F&SF. Mjcrossuk 11:17, 14 September 2013 (UTC)

I don't know if the community at large is interested in all this database-level detail. Feel free to move this elsewhere. My general reaction to the above is (a) creating views is awfully heavyweight, especially for so little data, and (b) using "where in (select ...)" is not usually a good practice. Backing up for a moment, I would try to describe the general problem / data sets in words. Basically, you want all of Budrys' reviews in Galaxy, along with some information about the magazines themselves and the page on which the review appeared. And then you want some information about the reviewed titles (author and title). So I'd start that way.

I'm going to ignore the question of how best to identify the magazines themselves and just use what you were using.

Getting the first part, for example, could be done (I've used ANSI syntax for the joins, as it might make some of the groupings a little easier to visualize) like so:

  --
  -- Reviews by Budrys in Galaxy, 1965 - 1971
  --
  SELECT rt.title_id, p.pub_id, p.pub_title, p.pub_tag, p.pub_year,
         lpad(ifnull(pc.pubc_page,'   '),3,'0') pubc_page
    FROM titles rt
    --
    -- Constrain the review title's author to Budrys
    --
    INNER JOIN canonical_author rca ON
      rca.title_id = t.title_id
      AND rca.author_id = 12 AND rca.ca_status = 1
    --
    -- This next join finds the entry linking the review to
    -- the publication in which it appeared.  NB: This entry
    -- also tells us on which page the content (the review)
    -- appeared.
    --
    INNER JOIN pub_content pc ON
      pc.title_id = rt.title_id
    --
    -- Constrain the publication to Galaxy 1965 - 1971.
    --
    INNER JOIN pubs p ON
      p.pub_id = pc.pub_id
      AND p.pub_tag LIKE 'GAL%'
      AND p.pub_ctype = 'MAGAZINE'
      AND year(p.pub_year) BETWEEN 1965 AND 1971
    --
    -- Constrain the title's type to REVIEW to exclude any other
    -- content by Budrys.
    --
    WHERE rt.title_ttype = 'REVIEW'
    ORDER BY p.pub_year, pc.pubc_page, rt.title_id;

To get information about what was reviewed, you need to find the reviewed title, which is linked to the review via an entry in the title_relationships table. You could make the above a view, but you can also just do some more joining (it's ok to join over the same table multiple times). I came up with this (again ANSI inner join syntax just to help with visualizing):

  SELECT ta.author_canonical, ta.author_lastname, t.title_title,
         rt.title_id, p.pub_id, p.pub_title, p.pub_tag, p.pub_year,
         lpad(ifnull(pc.pubc_page,'   '),3,'0') pubc_page
    FROM authors ta
    INNER JOIN canonical_author ca ON
      ca.author_id = ta.author_id
    INNER JOIN titles t ON
      t.title_id = ca.title_id
    --
    -- Everything above is for the title that was reviewed.
    -- This next join finds the entry linking the title to
    -- the review title.
    --
    INNER JOIN title_relationships tr ON
      tr.title_id = t.title_id
    --
    -- The review title
    --
    INNER JOIN titles rt ON
      rt.title_id = tr.review_id
      -- this probably not necessary, since it's implied.
      AND rt.title_ttype = 'REVIEW' 
    --
    -- Constrain the review title's author to Budrys
    --
    INNER JOIN canonical_author rca ON
      rca.title_id = rt.title_id
      AND rca.author_id = 12 AND rca.ca_status = 1
    --
    -- This next join finds the entry linking the review to
    -- the publication in which it appeared.  NB: This entry
    -- also tells us on which page the content (the review)
    -- appeared.
    --
    INNER JOIN pub_content pc ON
      pc.title_id = rt.title_id
    --
    -- Constrain the publication to Galaxy 1965 - 1971.
    --
    INNER JOIN pubs p ON
      p.pub_id = pc.pub_id
      AND p.pub_tag LIKE 'GAL%'
      AND p.pub_ctype = 'MAGAZINE'
      AND year(p.pub_year) BETWEEN 1965 AND 1971
    ORDER BY p.pub_year, pc.pubc_page, t.title_title, rt.title_id;

I think that does what you said you wanted and works correctly. You'd have to do some testing / manual verification to see. Feel free to follow up on my talk page with questions (or here, if you think they may be of general interest, or start a new page somewhere). --MartyD 00:11, 15 September 2013 (UTC)

Many thanks for your reply. I'll give it a go, and respond on your talk page. i wasn't too concerned about performance, or best practice as the data volumes are small as you say, and it does run in a few seconds (after the first time which is bit slower). Mjcrossuk 12:08, 15 September 2013 (UTC)

MartyD's SQL provided me with the clue I needed to solve my original question about determining if the reviewed title was an Anthology... I extended my views to link to the title_relationships table, & hence to the title table entry for the reviewed title, and thus the title_ttype attribute. I had been using an out-of-date database diagram which didn't have the title_relationships in it. Is there an up-to-date database diagram here somewhere? Mjcrossuk 17:23, 15 September 2013 (UTC)

I'm pretty sure there isn't an up-to-date database diagram here - and I'm not sure where you even found an out-of-date one! The limitations on file-type uploads seem to preclude a full one at readable size now. I'll see what I can do with the limits we have - almost every processing tool for the downloaded data seems able to generate most of the relationships, and those that can't are probably choking on our lack of foreign key constraints etc. We've made big steps in database improvements in the last few years, it would be a shame if only the 3 or 4 developers responsible UNDERSTOOD them! BLongley 10:57, 18 September 2013 (UTC)

The out-of-date diagram is here: http://www.isfdb.org/wiki/index.php/Database_Schema Mjcrossuk 19:25, 18 September 2013 (UTC)

Ah, that. I was assuming something more pictorial. I'm sure we can improve that documentation, although pictures would be a nice addition. Leave it with me for a bit. BLongley 12:18, 21 September 2013 (UTC)

The last sentence of the introduction section on that page says, in part, "An up to date diagram of the ISFDB tables and their relationships can be found here." and links to a picture. Mjcrossuk 12:31, 23 September 2013 (UTC)

Kathleen Duey, below the threshold?

The author Kathleen Duey is currently indexed as if she were above the threshold, and her bibliography includes about 40 books listed as non-genre. I know of no reason why she should be viewed as being above the threshold. She has won the Golden Duck award (2001) for an illustrated children's fantasy, but has never otherwise placed higher than 8th in any regular SF/F award. She has not been on the scene long enough to be "an influence" on others; she has no Wikipedia page; etc. I move that she be re-categorized as "below the threshold". Chavey 17:54, 13 September 2013 (UTC)

Seems alright to me, so I agree to this move. Stonecreek 20:57, 13 September 2013 (UTC)

I think I view the (in)famous "hard to define threshold" differently. If an author has published 40+ SF books, it's to our advantage to list the rest of her output so that a user could quickly determine whether a given book is SF or not.

Moreover, I don't think that influence is a good criterion to use when determining whether an author is above the "threshold". Let's use Robert Louis Stevenson as an example. His Strange Case of Dr. Jekyll and Mr. Hyde has been very influential, yet Stevenson is not above the threshold and we don't list his non-SF works. Ahasuerus 01:02, 14 September 2013 (UTC)

Well, Chavey wrote that the author has about 40 books listed as non-genre in addition to nearly the same amount of genre novels: both groups are almost on a par. I'm not so sure that the author's non-genre work should be of special interest to genre lovers, the bulk of them (if not all) aimed at a young public. Stonecreek 07:53, 14 September 2013 (UTC)

She has about 40 genre and 40 non-genre works. I can sort of understand Ahasuerus's suggestion that we use "non-genre" to help users figure out which books of some authors are spec fic and which are not -- but in most cases the answer should be "If it's here, it's spec fic; otherwise it's not.". (Yes, that's an ideal we don't always meet, but that's what we should aim for.) And I disagree with this use of non-genre. While I don't think it should include every "influential" author (e.g. Stevenson), I think "influential / important" should be a minimal criteria for an author being above the threshold. Chavey 04:33, 15 September 2013 (UTC)

Script request

Can someone with a better knowledge of the software than me write a script that can find records which have a publication date before 2005 (2004 and earlier), AND have an ISBN-13. I tried using the Advance Search publication section, but it won't accept wildcards in the ISBN field. (Can that be fixed?) Mhhutchins 02:48, 15 September 2013 (UTC)

I'll have to check, but I think the Search logic tries to be clever about finding both ISBN-10s and ISBN-13s at the same time, so adding wildcards may mess things up. I'll have to take a closer look, though. Ahasuerus 04:30, 15 September 2013 (UTC)

The quick search uses LIKE, so wildcards work there, but the advanced search uses = instead. Don't know why, off-hand. The "smart" logic only works with valid ISBNs, turning those into four variations: ISBN-10 and ISBN-13, dashed and dashless. Dashed and dashless is still something to keep in mind when searching. --MartyD 12:08, 16 September 2013 (UTC)

Dashed and complete ISBNs can be found in the quick search. But if you're looking for ISBN ranges in the simple search, you must remove the dashes. For example, you want to see all records for DAW Books with ISBNs starting with 0-7564-01, entering the dashes give you no results, but entering "0756401" does. Why is that? I suspect most users wouldn't know this. It took me awhile when I first started looking for ISBN ranges.

BTW, I've never used wildcards in the quick search. Just tried it and it didn't work, at least not with an asterisk. What should I have used? Mhhutchins 15:43, 16 September 2013 (UTC)

"%" should work, e.g. "9780760%3%" finds this pub and this pub. Once again, Advanced Search tries to be more, well, advanced :) , so it uses asterisks rather than %. But yes, we need to synchronize the behavior of the two searches. We also need to make Advanced Search work with Unicode characters -- pages 2+ are currently broken. Ahasuerus 17:15, 16 September 2013 (UTC)

I've wondered why some advance searches won't go past the first 100 returns. But I found a workaround in most cases: change the "record%3D" number in the URL to the next 100 records (e.g. change "100" to "200"), and it brings up the next page. Mhhutchins 18:57, 16 September 2013 (UTC)

I'm finding these records occasionally and want to clean them up if possible. It seems moderators are allowing these through without looking at the ISBN field. There was one earlier today which was published in 1975 but had an ISBN-13. Thanks for your help. Mhhutchins 02:48, 15 September 2013 (UTC)

Yes, that should be a simple script and I will add it shortly. There are 378 pubs like that as of the last backup. Ahasuerus 04:30, 15 September 2013 (UTC)

Patch r2013-116 - Recent Rejects improvement

The list of Recent Rejects should take less time to compile now, although it may still take a few seconds. Ahasuerus 23:20, 18 September 2013 (UTC)

Anyone interested in taking over the ISFDB LiveJournal account?

It's not a huge task - no actual work involved at all really - but I started collecting Authors and Editors with an LJ presence a few years back and found it quite an interesting read. You can see what I mean by visiting http://isfdb.livejournal.com/friends/ and browsing for a bit. There's several hundred authors, major to minor, so quite a lot of traffic BUT there's never any need to actually DO anything - you just might see a glimpse of a preview cover or a title change or a forthcoming novel or a reprint or something.

You can even split the job and make it a shared account, so long as one of you is agreed on who changes the password, and you don't argue about replies. I've posted very little, mostly to explain misconceptions about what ISFDB is and what it isn't. Occasionally how to use the data offline rather than via the web, but that's been rare.

Any takers? BLongley 10:45, 20 September 2013 (UTC)

I guess that's a silent but resounding "No". I wonder if it would have got more replies if it was about Facebook or Twitter accounts? BLongley 11:07, 22 October 2013 (UTC)

"The Gernsback Prophecy" exhibition in Karlsruhe

The Gernsback Prophecy (page in German) is an exhibition in Karlsruhe, Germany, that is running for about another month. I will be visiting Karlsruhe and Heidelberg on the weekend of October 20th, and I plan to check out the exhibition on Sunday (the 20th). If anyone is interested in meeting, having a drink, or whatever, please let me know. You can answer here or send a ping via private email. Cheers, Patrick -- Herzbube ^Talk 19:16, 23 September 2013 (UTC)

I'd be interested, but I will have to take the chance of a visit at another weekend, I'm afraid: I'll be on vacation in the South of Holland at that time. Sorry, Stonecreek 13:29, 26 September 2013 (UTC)

Server maintenance

Due to the "duplicate submission" problem that has been plaguing us for the last month+, I will be running some test on the server starting in about 5 minutes. I will need to disable submissions for the duration of some tests -- sorry about the inconvenience! Ahasuerus 21:39, 23 September 2013 (UTC)

Patch r20113-117

The submission approval process has been changed to use a single database update when approving/rejecting/hard-rejecting submissions. This should help eliminate "half-approved" submission which had their status changed to "Integrated" but had no approval date/time stamp or moderator name associated with them, thus making them invisible on the Integrated Submissions page.

I doubt it will completely eliminate the duplicate submission problem, but at the very least it should make our Integrated/Rejected lists more accurate and help diagnose other problems. Ahasuerus 00:58, 24 September 2013 (UTC)

Patch 2013-118

The Recent Verifications page has been improved and should take less than a second to load from now on. In addition, the "ID" field and the "Title" field have been combined into a single "Pub Title" field. Prior to this patch, this was our slowest loading page. Ahasuerus 03:14, 25 September 2013 (UTC)

Server speed

The server has been particularly slow for the past 24 hours, slower than usual. Is there an explanation for it? Mhhutchins 16:14, 29 September 2013 (UTC)

It's been sporadic -- fine for a bit, then slower than molasses in winter, then fine for a bit, etc. So pretty much the same pattern that we have seen for the last few weeks, only more so. A few days ago I enabled logging of "slow queries", i.e. all database updates and searches that take more than a few seconds. The log shows that we have a growing number of very simple database updates and searches (like "find all titles for this pub") that take an abnormally long amount of time to complete.

So we have narrowed it down to the speed of database interactions as opposed to our software or the web server or any number of other things. The next question, of course, is why the database is sporadically slow. It could be caused by the virtual server which hosts ISFDB being misconfigured or the storage system (basically the hard drives and the equipment that supports them) going bad or a few other things. We have no control over these things and no way of telling what's going on at that level -- it's supposed to be the hosting company's area of responsibility -- but we can ask Al to tell the hosting company to (a) reboot the server and (b) check their logs to see if they need to replace a disk going bad or something. I'll shoot him an e-mail and see what he finds. Ahasuerus 21:48, 29 September 2013 (UTC)

I'm not sure if the following problem comes from the server or from a change in Google Chrome, but every time I do an Advanced Search, and get a results page, no matter how many times I go back (using the browser) to the search results page, it has to do a new search instead of just giving me the same page (a cached copy of the same results). This not only takes up my time, it must also slow down the server for other users while it's doing a new search. It's only been happening for the past couple of weeks or so. Mhhutchins 15:50, 30 September 2013 (UTC)

Although it is possible to change this behavior on the server side, the ISFDB software doesn't do it. In addition, the Advanced Search logic hasn't changed in months. I have tried to recreate the problem using Google Chrome and Firefox and I see that only Chrome reruns the search while Firefox uses cached results. Perhaps it's possible to customize Chrome's behavior under Options?.. Ahasuerus 16:58, 30 September 2013 (UTC)

I looked but couldn't find it. I'll try again. It must be from the recent updates to Chrome. Perhaps other Chrome users are doing the same and the combination of many users at once doing advance searches slows down the server? Mhhutchins 18:23, 30 September 2013 (UTC)

The database log shows that some "slow queries" are caused by Advanced Title searches, but they account for a very small portion of all "slow queries". There are some other potential server issues that we are currently looking into... Ahasuerus 00:13, 1 October 2013 (UTC)

P.S. Have you seen an improvement in the last 24 hours, by chance? Ahasuerus 00:17, 1 October 2013 (UTC)

It's not timing out like it had been. But this auto-refreshing is slowing me down. I tried looking at the advanced settings on Chrome and could find nothing about using cached screens or how to stop auto-refreshing of pages. If you have a few minutes would you mind checking to see if you can find out how to set such things? I have Version 29.0.1547.76, which downloaded some time last week, and this was about the time the refresh problems started, so it must be connected. Mhhutchins 00:25, 1 October 2013 (UTC)

I have reviewed the official list of changes, but they are either unrelated to this problem or so low level (what's a "NULL check of shelf in tray_background_view" anyway?) that I can't see a connection. I also played with "chrome://flags", which is where most of the really advanced options are, but couldn't find anything that may be related. Perhaps an unintended or at least undocumented side effect of some other change(s) in this version?.. Ahasuerus 01:08, 1 October 2013 (UTC)

The list you link to is for a slightly older version of Chrome: 29.0.1547.57 (mine is 29.0.1547.76). A little googling for "refresh" and "29.0.1547.76" came up with a couple of hits for people with refresh problems. Mhhutchins 01:39, 1 October 2013 (UTC)

"them", by Joyce Carol Oates

We currently have this novel listed as a genre novel. Reading Wikipedia's plot synopsis, I can see nothing in there that looks even potentially genre. Does anyone know any reason why this title should not be deleted? (She is not "above the threshold" for this genre.) Chavey 16:31, 29 September 2013 (UTC)

If it's not spec-fic, then the title record should be changed to NONGENRE, as should all of her non-spec-fic novels. She's done significant work in the field for her non-spec-fic work to be in the database, IMHO, as long as it's properly categorized. Mhhutchins 17:07, 29 September 2013 (UTC)

Thanks, done. As seems to be standard with non-genre books, I put in the first edition of that book, but not later editions. Chavey 02:17, 1 October 2013 (UTC)

Patch r2013-121 - Statistics page improvements

The performance of the ISFDB Statistics page has been improved. The downside -- although I am not sure it is a downside -- is that it now reports the total number of Title records, i.e. it no longer excludes Variant Titles. Ahasuerus 23:11, 4 October 2013 (UTC)

Editing interruption

I will need to disable editing for a couple of minutes starting at 8pm (9pm US Eastern) server time in order to improve the performance of one of our pages. Ahasuerus 00:55, 5 October 2013 (UTC)

Done. The "Top Moderators" page should take about 5 seconds to load now. Ahasuerus 01:07, 5 October 2013 (UTC)

Perhaps a better way to determine "Top Moderators" is to deduct the moderator's submissions from the total of moderations. This gives a better picture of which persons are actually moderating instead of just accepting their own submissions. Mhhutchins 02:39, 5 October 2013 (UTC)

True, but, unfortunately, it would make the page slower again :( We have 1.5 million submissions on file and any operation that examines that many records is problematic from the performance standpoint. Ahasuerus 03:09, 5 October 2013 (UTC)

Strange. It's simple arithmetic: X (the number of accepted submissions, i.e. the current "moderation" figure) minus Y (the number of contributions) equals Z (the number of moderations of other editors' submissions). There is a problem in that it takes at least a couple thousand submissions before one becomes a moderator. But it also doesn't take into account follow-up submissions to clean up another editor's errors. So it's a rough estimate. In my case 336,919 - 245,225 = 91,694. I have moderated almost 92,000 submissions by other editors. Mhhutchins 03:26, 5 October 2013 (UTC)

The "top contributors/moderators" pages do not scan the whole submission table, which would take a long time because the table contains a gigabyte of data. Instead they use indices, which are much faster. In order to add a new column which would show the (X-Y=Z) number, the "top moderators" query logic would have to scan two indices instead of one. It wouldn't be fatal, but it would make the page slower.

If it's something we'd like to have and if you wanted to try an experiment, you could make a concatenated index and do the counting. In fact, I'd try throwing sub_state into it, too, and get rid of the separate sub_state index:

create index sub_staterevsub_idx on submissions(sub_state, sub_reviewer, sub_submitter);
select sub_reviewer, count(*), sum(case when sub_reviewer <> sub_submitter then 1 else 0 end)
  from submissions where sub_state='I' group by sub_reviewer;

You should be able to determine if it would work without changing the code via explain, as it'll show you if it would use the new index or not. You could explain the query without the index, then add the index, then explain it again. I also think with the concatenated index, you wouldn't need separate indices on sub_submitter or sub_reviewer. Rhetorical aside: why does the current form of that query (and the contribs queries) use "distinct"?

If that idea works, you could probably do something similar for the top contribs query, working the sub_type column into an index. --MartyD 11:58, 5 October 2013 (UTC)

We can certainly do it if there is interest, but keep in mind that these pages are frequently retrieved by robots. Every time a robot accesses a slow loading page, it slows things down a bit for everyone else. Decisions, decisions :-) Ahasuerus 03:39, 5 October 2013 (UTC)

It's not worth it if it's going to slow down the server. But I see the list of moderations as rather useless if it doesn't record the actual moderations that a moderator does, rather than just recording the number of submissions that they've accepted. Moderating your own submissions by definition isn't really moderating. Mhhutchins 17:09, 5 October 2013 (UTC)

As a moderator who does mostly self-moderation, I agree with Mike that it would be useful to be able to credit those editors who do the actual work of moderating newer submissions. Using his formula, it seems like it can't be more time expensive than to generate both the top moderator page + the overall submissions page, then a little arithmetic. Of course that formula has some problem, e.g. in my case in gives me a large negative score: -6,338. (I did a lot of submissions before becoming a mod.) A nearly accurate formula could be constructed by generating a table of the number of overall submissions by a moderator on the day they became a moderator. That, of course, would be a very time-consuming script to run. However, that script would only need to be run on those rare occasions when we welcome new moderators, and then stored in a fixed table (such as with our Computationally Intensive Lists). Chavey 22:21, 5 October 2013 (UTC)

It's a bit more complicated since we also have editors who were moderators at some point in the past only to be demoderatorized later due to inactivity. However, Marty's proposed solution seems to work quite well on the development server. I'll try it on the main server later today. Ahasuerus 16:16, 7 October 2013 (UTC)

"Eripmav", by Damon Knight

Eripmav is a 1-page short story by Damon Knight. When Damon Knight and Kate Wilhelm were GoHs at Noreascon II (1980), the New England Science Fiction Association republished the story in an unusual format: As a t-shirt. (To see the t-shirt, and read the story, see the t-shirt front and back. I suggest expanding the pictures to full size.) Does this qualify as a "publication"? Apparently, it was sold at the con. Chavey 21:55, 5 October 2013 (UTC)

I would think so. But please don't create a new "binding". Leave the Pub Format field blank, and just describe it in the Note field. Mhhutchins 01:13, 6 October 2013 (UTC)

Ok, added it here, as a Chapterbook, since that seemed the "publication type" closest to this. Binding left blank, with notes; "Cover photo" left blank, since the two photos are both of the "contents". Chavey 17:48, 6 October 2013 (UTC)

Patch r2013-123 - default preferences bug fixed

The software should no longer allow accidentally modifying our default language/web site/etc preferences, i.e. the display settings effective for those who do not log in. Ahasuerus 01:37, 6 October 2013 (UTC)

Patches r2013-124 and r2013-125 - NewPub/EditPub field separators fixed

The line separators on the New Publication and Edit Publication pages have been made consistent. The Clone Publication page is still inconsistent because of some other changes that need to be deployed at the same time, but I hope to wrap everything up tomorrow. Ahasuerus 05:22, 6 October 2013 (UTC)

Editing interruption at 5pm server time

Editing will be unavailable for a few minutes starting at 5pm server (Central US) time. Ahasuerus 20:55, 7 October 2013 (UTC)

Done. Ahasuerus 21:06, 7 October 2013 (UTC)

Top Moderators updated

The Top Moderators page has been enhanced to show the breakdown of moderated submission between self-approvals and submissions created by other users/robots. Ahasuerus 22:19, 7 October 2013 (UTC)

Cool! (Even if it does show how self-absorbed I am :-) ) Chavey 19:44, 8 October 2013 (UTC)

Thanks. This is basically what I was asking for. Mhhutchins 22:16, 8 October 2013 (UTC)

BLIC

If anyone updates a publication record which has linked BLIC in the Note field, please remove the link, but leave the record number. These links never worked (well, beyond 24 hours for one user) and should be removed when updating records with them. Thanks. Mhhutchins 23:42, 9 October 2013 (UTC)

Patch r2013-127 - Clone Publication changes

Clone Publication has been modified to look and feel more like New Pub and Add Pub. Specifically, mouseover help and auto-verification have been added.

Unfortunately, although the changes look simple, they turned out to be something of a bear to code and test, which is why it took forever and a day. I am still not 100% sure that everything is working as intended, especially when entering new reviews and interviews during cloning. If you see anything unusual, please report your findings here. Ahasuerus 05:04, 11 October 2013 (UTC)

Editing outage - 2013-10-11

Editing will be disabled between 3:40pm and 3:50pm server time (Central US time) due to maintenance. Ahasuerus 20:34, 11 October 2013 (UTC)

Done. Submission type-specific "top contributor" lists linked from the Major Contributors page should load much faster now. Links to a few recently implemented submission types have been added. Ahasuerus 20:48, 11 October 2013 (UTC)

Patch r2013-129 - Top lists and stats consolidated

All top lists and database stats can now be accessed from a new page, ISFDB Statistics and Top Lists, which is linked from the ISFDB home page. Various "top lists" links which used to be displayed in the moderator navigation bar and on the Major Contributors Wiki page have been removed. In addition, all "Top" lists have been limited to contributors with at least 10 relevant contributions. Ahasuerus 22:26, 12 October 2013 (UTC)

These are some very nice improvements. As well as the other improvments, I like the idea of limiting the lists to folks with 10 or more contributions. Chavey 03:18, 13 October 2013 (UTC)

Thanks! One fix at a time :) Ahasuerus 01:10, 14 October 2013 (UTC)

Patch r2013-130 - Recent Edits and My Recent Edits fixed

"Recent Edits" and "My Recent Edits" have been aligned with "Recent Integrations" and "Recent Rejects". ClonePubs and AddPubs should appear correctly and Author Updates are now linked to author history. Ahasuerus 02:00, 13 October 2013 (UTC)

Standardizing double quotes

The vast majority of the double quotes that we have are regular double quotes found on all keyboards. However, we also have 23 title records with "double low-9 quotes", e.g. Illustrierter Anhang: „Der Overview-Effekt“, 18 records with "right double quotes", e.g. Alistar Waynewright: „Being Inc.” and 7 records with "left double quotes".

I propose that we change the software to convert these types of quotes to standard double quotes at data entry time, which should eliminate the potential for record duplication and also make searching easier. It would be a simple change since the software already changes "curly apostrophes" to regular apostrophes, so we'll just need to modify the code to do the same thing for non-standard double quotes. Ahasuerus 01:10, 14 October 2013 (UTC)

It's a slightly Anglo-centric proposal, but I think that the values of standardization outweigh the problems. I support the suggestion. Chavey 15:38, 17 October 2013 (UTC)

Hearing no objection... the software has been updated and all irregular records have been brought into compliance. Ahasuerus 01:34, 19 October 2013 (UTC)

Patch r2013-132 - Edit Title changes

"Edit Title" has been modified to disallow YYYY-02-29 dates for non-leap years and will display a pop-up message if you try to enter them. Originally, we tried to accommodate both the Gregorian and the Julian calendar, which have different leap year rules: 1700-02-29 or 1900-02-29 would be valid under the Julian system, but not under the Gregorian one. The reason why we wanted to support both calendars was that the transition from one to the other started in the 1580s and wasn't complete until 1923. However, it later turned out that our database didn't support non-Gregorian leap years, so it would have simply errored out at submission approval time. Hence the change to disallow these dates at data entry time.

Please note that this doesn't affect the behavior of all other edit forms which silently change invalid dates to 0000-00-00. Now that Edit Title looks solid, I plan to add pop-up validation of dates and required fields throughout the system in the foreseeable future.

In addition, the Edit Title page no longer complains if you enter a YYYY or a YYYY-MM date and will automatically add "-00-00" or "-00" as required. I believe this was the last place in the software which had a problem with abbreviated dates, but if you find any others, please let me know. Ahasuerus 03:31, 14 October 2013 (UTC)

Patch r2013-133 - Added pop-up validation to Edit Author

The Edit Author page has been changed to generate pop-up errors if the Canonical Name and/or the Last Name field is left blank. It should also generate a pop-up error if the canonical field value contains a double quote.

In addition, a minor graphical glitch, which affected the positioning of author images and cover scans, was fixed. Most browsers handle this type of glitches automatically, so chances are that you won't notice any difference. Ahasuerus 22:20, 14 October 2013 (UTC)

The more common error here is a wrongly formatted date field (birth or death). I still get submissions which have defaulted to "unknown" due to format errors. Can that be programmed to generate an error warning as well? Mhhutchins 00:00, 15 October 2013 (UTC)

Indeed. As a matter of fact, that's what I am working on tonight :) Ahasuerus 01:04, 15 October 2013 (UTC)

OK, I think I got it. I also made the logic a bit smarter so that it would reject future years, 8888 and 9999 for authors. Ahasuerus 02:05, 15 October 2013 (UTC)

Patch r2013-135 - Pop-up validation for Add/Edit Awards

Add Award and Edit Award have been modified to perform immediate validation of missing/invalid dates, categories and award levels and display appropriate pop-up messages. Ahasuerus 22:36, 15 October 2013 (UTC)

Patch r2013-136 - Add/Make Variant

Add Variant and Make Variant have been modified. It is no longer possible to create variant title records without titles, authors or years. In addition, appropriate pop-up messages have been added. Ahasuerus 03:57, 16 October 2013 (UTC)

Wow. Never thought that was possible anyhow. Mhhutchins 04:21, 16 October 2013 (UTC)

FWIW, only a subset of possible permutations was allowed, so in most cases you would have received an error. Ahasuerus 05:09, 16 October 2013 (UTC)

Why would anyone do that in the first place? Mhhutchins 04:21, 16 October 2013 (UTC)

No good reason that I can think of, but you know how confused new editors can be :) Ahasuerus 05:09, 16 October 2013 (UTC)

I could imagine someone thinking they could create a variant with just a title and expecting the other fields to be populated when they did an "Add Publication" to the new title. That would be a bad design, but I can imagine someone thinking it would work. Chavey 15:51, 17 October 2013 (UTC)

Add Gender to Author Edit?

I would like to suggest adding a gender checkbox to the author edit page and add it to the pull-down menu in Advanced Search. This would be helpful to researchers conducting gender studies such as Eric Davin.--Rkihara 17:23, 16 October 2013 (UTC)

This field has been requested a couple of times. It's not hard to add, but first we need to decide what to do about transsexual authors like Jessica Amanda Salmonson (born Jesse Amos Salmonson) and Jean Marie Stine (born Henry Eugene Stine -- see Hank Stine). Do we want to list their "last known gender"? Oh, and a simple checkbox probably won't work, we'll need a dropdown list with "-" (for "unknown"), "male" and "female" as valid options. Ahasuerus 18:52, 16 October 2013 (UTC)

I gave some thought as to whether we needed to identify transgendered authors and since the government and media generally recognizes a person's self-identified gender think it's not necessary. I also think that most transgendered people don't wish to be prominently identified, so if we do make the identification we should downplay this and show only the self-identified gender on the displayed author page, while allowing trangendered individuals to be found only in Advanced Search.--Rkihara 01:59, 17 October 2013 (UTC)

There are a number of ways we could address this issue, e.g. the previously-mentioned drop-down box could include "unknown" (default), "male", "female", "male-to-female transsexual" and "female-to-male transsexual". Or we could have two fields, one for "current gender" and one for "birth gender". However, that sounds like a bit too much work considering the limited number of affected records. I wonder if one drop-down list for "current gender" with "unknown", "male" and "female" as three possible choices may be sufficient for our nefarious purposes. Ahasuerus 03:10, 17 October 2013 (UTC)

For simplicity I would vote for just male, female and unknown, as it would cover 99.99% of the database. A short comment could be added to the biographical notes for the transgendered--Rkihara 04:31, 17 October 2013 (UTC)

Raphael Carter does not fit into the gender binary, and prefers that friends use neither male nor female pronouns. So Raphael would fit none of those 3 categories: Raphael is specifically not unknown. There are some author databases that keep track of gender (e.g. LibraryThing), and at least some of them distinguish between "male" and "presumed male". The question becomes, how much do we need to know about them before we assign them a gender? Do we have to have seen them? (Is that good enough?) Do we have to have a reasonably authoritative reference that uses a gendered pronoun for them? Can we assume someone named "Susan" is female? As a gender researcher, I also would like to see this data in the database. (So far, when I've identified a non-obvious gender identity, I've made sure to add something to the author's wiki bio age that includes a gendered pronoun.) But to add this feature (which I support), we need to decide what rules apply when we specify a gender as "known". Chavey 16:04, 17 October 2013 (UTC)

Self-identification is the simplest way to assign gender, otherwise it becomes too complicated. Not all of the transgendered undergo sexual reassignment surgery or even enter into the role of the opposite sex. Bradley Manning, now Chelsea, for example, who is now identified in the media as "she." Raphael Carter might fall under the category of "asexual," a category that a large number of people self-identify with, although he might object to the label. I would suggest that complex gender histories be marked with an asterisk, so "male*, female*, asexual*," with comments added to the biographical field.--Rkihara 16:39, 17 October 2013 (UTC)

I find nothing about this discussion to be of interest. What other categories are authors awaiting to be placed into? Ridiculous. Mhhutchins 20:52, 17 October 2013 (UTC)

I am not sure about authors, but some people slice and dice gender in all kinds of ways these days. To quote this NPR story:

ADLER: But some students are going further. At one college that Joy Ladin visited, things were so fluid you could make up a different pronoun for a different event.

LADIN: So you can be she/her at one event and then you go to lunch and you say, OK, now I am he/him. And then one charming young woman told me, oh, yes, today, I'm just using made up pronouns.

LYNN WALKER: We encountered high school students who said, I want you to call me Tractor and use pronouns like zee, zim and zer. And, in fact, I reject the gender binary as an oppressive move by the dominant culture.

Which goes to show that if we start adding to the list of choices, it may be hard to stop. And that's why my thinking is that limiting the choices to "unknown" (default), "male" and "female" and relegating everything else to the Bio page may be the easiest way to handle the issue. Ahasuerus 23:35, 17 October 2013 (UTC)

[unindent] This is going to be a hornet's nest that we should stay away from entirely. It would make more sense to divide authors by genre in a literary database than it would by gender, and would be just as difficult. How would you handle Jeff Jones, who did not change his professional name when he became a woman? Should the work he did as a man be considered different than what he did in his last few years as a woman? (The Wikipedia article on Jones says she "lived for a time as male". Really? I guess 50 years would be considered "a time".) If it's necessary to record an author/artist's gender, then when do we start recording their race, ethnicity, religion, and sexual orientation, none of which are important enough (and too fluid) to go through classifying thousands of people. It's the work that comes out from between their ears that matters, not what's between their legs. Continue to link to Wikipedia and the SFE, and let them discuss gender all they want. This is a bibliography. Mhhutchins 00:02, 18 October 2013 (UTC)

If we take your position to the extreme, then all we need to know about an author, other than their works, are their names and pseudonyms. DOB, DOD, Birthplace, photos are unnecessary. The ISFDB then becomes a lot less useful. Gender in the arts is a serious topic of research, not to mention curiosity. I always wonder if an author who identifies themselves by their first initials are male or female. Limiting genders to Male, Female, and unknown are fine with me. It doesn't have to be parsed any finer than that.-- —The preceding unsigned comment was added by Rkihara (talk • contribs) .

Yes, if you take the position to the extreme, but nothing in my response advocates removing such data from the database. DOB and birthplace are necessary in a bibliography. Gender is not. If an author chooses to be identified by initials only, that was their choice. If you're wondering what their sex is, you can easily find that on the internet in 2013. And you can always create an Author bio page on the ISFDB to add that information. Mhhutchins 01:34, 18 October 2013 (UTC)

Sure, the information can be added to the ISFDB Wiki or found on the Web. However, there are advantages to having it available in queryable form within a database.

For example, consider Eric Leif Davin's Partners in Wonder: Women and the Birth of Science Fiction, 1926-1965. He spent a great deal of time identifying the 203 women who published SF in genre magazines between 1926 and 1965 -- think how much easier his life would have been if he had been able to query a database! (Or at least ask a programmer to write a three line script to extract the data.)

Similarly, having this data available within ISFDB would have saved James Nicoll (a well-known Hugo-nominated fan writer) a ton of effort when he was data mining ISFDB to compile gender stats for SF publishers a couple of years ago.

Granted, it's not earth-shattering and it may not get us the Nobel Peace Prize, but it seems like a useful thing to add. Ahasuerus 02:20, 18 October 2013 (UTC)

I can think of much more useful things editors can be doing than determining the sex of thousands of writers. It's understandable that in this db's entire history this subject never arose. Mhhutchins 03:47, 18 October 2013 (UTC)

Well, I recall this question being asked on Usenet a few years ago and it was also independently raised by Darrah right around that time. Certainly not a hot red item that we absolutely must have tomorrow, but I think it won't hurt adding it to the queue. Ahasuerus 03:59, 18 October 2013 (UTC)

Good luck to those who choose to do it. I won't be participating. Mhhutchins 03:47, 18 October 2013 (UTC)

It's possible that some editors may go out of their way to enter this type of data, but I would expect that most of us will simply pick from the drop-down list when editing Author records for other reasons. Sort of similar to how the language field has been handled, although it will take slightly more effort. Ahasuerus 03:59, 18 October 2013 (UTC)

And probably with the same diligence? For instance, an editor is updating a record for an author named "George" and so automatically assumes it must be a man, ticking the box for "author with a penis". So we've changed the sex of George Sand without surgery involved. Mhhutchins 04:57, 18 October 2013 (UTC)

I concur with Michael, even if this kind of data may prove useful to some researchers (I've much admired the quantity of work furnished by Davin), IMHO it means some trouble ahead and may open the way to demands for other type of data whose relevance to bibliography may be even more tenuous. Hauck 13:39, 18 October 2013 (UTC)

It may be useful when editing a name like George Sand and declaring her female to add either a picture of the person in question (preferably) or to add a biographical note to prevent the surgery as mentioned above. Note that Italian names for males can be taken for girls' names too by persons speaking another language (e.g. Andrea).--Dirk P Broer 20:42, 21 October 2013 (UTC)

Exactly, my daughter surname is Andréa, which is usually a female surname in France (it's more Andréas for a boy). Hauck 16:15, 23 October 2013 (UTC)

There are also English names whose usage has changed over time, e.g. Shirley, which can lead to all kinds of confusion. Ahasuerus 01:14, 22 October 2013 (UTC)

[unindent] I've asked Eric Leif Davin if he would like to share his thoughts on this discussion and I've posted his response below with his permission.--Rkihara 19:43, 18 October 2013 (UTC)

"I would certainly support adding author's gender to the database. It would certainly have made my life much easier in writing "Partners in Wonder" if I could have turned to an easily retrievable database with this information. More than that, it would have easily answered the charge, "There were no women in science fiction before the feminist intervention of the 1970s (or before 1965 or 1960 or whenever)". Indeed, with such a database, perhaps the charge never would have been made in the first place, and our entire understanding of science fiction history would have been different from the beginning.

There is a reason the U.S. Census compiles such information as this. It helps us know just who we're talking about when we talk about them and it helps us understand the nature of the groups we're talking about. When we make assertions about a group being composed, or not composed, of certain types of people, it certainly helps to know whether those assertions are based on actual knowledge, or simply selective guesswork. There can be no accurate scholarly or even factual work about any subject with out data, and the more data, the better. For too long, in fields such as the gender of science fiction writers in the past, scholars have been writing entirely in the dark, and making highly opinionated guesses based on mere unsupported assumptions. It would have been better, all around, if we had actually had the facts at hand.

As to the specifics of listing, just "male," "female," and "unknown" would add much to our knowledge & cover much of the territory.

Feel free to repost this comment to your on-going discussion. And thanks for asking my opinion.

-- Eric Leif Davin"

There is no question that this would have made Mr. Davin's research easier. Just as labeling them made it easier for Hitler to find Jews and homosexuals. But unfortunately (for Mr. Davin and his fellow researchers), until publishers identify an author's gender within their publications, it is not the job of bibliographers to attach such labels to books. Mhhutchins 20:21, 18 October 2013 (UTC)

Well, any structured data can be abused. For example, we collect authors' e-mail addresses and enter them into the database. A spammer could download our backup file, extract them and spam the heck out of their owners. Similarly, dates and places of birth could be used for even more nefarious identity spoofing purposes and so on. That's why we generally enter publicly available information form author/publisher/fan/etc sites and books/magazines. I am sure it will be the same way with the gender field. Ahasuerus 20:52, 18 October 2013 (UTC)

Unless I missed the joke, Michael, I think you meant authors not books, since books don't have gender in English<g>. But aside from that, I get the sense that the subtext of your arguments is that you're concerned that we'll "out" someone?--Rkihara 21:30, 18 October 2013 (UTC)

Wow, you are so off the mark. Do you mean we'll "out" a male who is writing as a female or vice versa? That should only concern those who want to label authors. Not me. If Mr. Davin's goal was to create a list of non-white sf authors, would it not be the same as suggesting that the ISFDB should have aided him in this endeavor by indicating the race of authors? Feel free to go ahead with this update to the software, and when the researchers ask us to categorize authors by other criteria, someone will have to explain why it's no different than categorizing by sex.

And no, I meant what I said: books or more precisely publications, because that's what we're dealing with here. I was talking about the creation of publication records. We don't create author records. When publishers start indicating an author's sex in the publications, then we can indicate that in the publication record. Until then, carry on with your attempts to categorize authors. I'll have no part of it. Mhhutchins 22:25, 18 October 2013 (UTC)

Granted, publications are our bread and butter, but we also collect information about other entities: publishers, authors, series, awards, publication series and so on. And the use of author records for statistical analysis goes way back. For example, consider the ISFDB Age Graphs page, which leverages the DOB and DOD values in the database -- it was created by Al many years ago.

Overall I would say that collecting gender information would be very much in line with the way ISFDB has been built and maintained. Ahasuerus 03:05, 19 October 2013 (UTC)

It took you twenty years to figure that out? Also, the collection of "information about other entities: publishers, authors, series, awards, publication series and so on" is based solely on publications. Would any of this information be in the database without an actual publication record? No. Without publications none of that data would be in the database. In my opinion, in this whole discussion no one has actually given a valid reason why we need to collect such information. Why is it important for a user to know if a work was written by a male, a female, or an "unknown" (your term, not mine)? Is there any less reason then to collect data on race and ethnicity of the authors? If a user knowing an author's sex informs his appreciation of the author's work, wouldn't race play an equal part as well for some users? I've had my say and will add no further input into the matter. It seems a few of you have already made that decision, even before it was fully discussed, and the one who rocks the cradle (writes the software) is the final arbiter of the matter. Mhhutchins 03:41, 19 October 2013 (UTC)

(after edit conflicts)

First of all, let me assure you that no decision has been made yet. I haven't even created a Feature Request, much less prioritized it. At this time, three editors support the proposed addition and two oppose it, not exactly a consensus. There is no rush and there is plenty of time for other editors to think about it and speak out. In general, I don't make software changes unless there is a consensus or there is an obvious bug that needs to be fixed. (Of course, it's possible for me to be mistaken about the existence of consensus.)

Re: the idea that the collection of information about other entities is based solely on publications, I mostly agree, although we have some non-title and even non-author based awards on file (which I am not a big fan of, but that's a different story.) However, even though publications are our starting point and determine which records we have -- authors, titles, series, awards, etc -- they don't determine which fields these records have. For example, DOB, DOD and place of birth are not directly related to any publication records and nothing says that we must have them and not some other author-related fields in the database.

When choosing new fields for a record, I consider a few factors. Fist of all, the data, if known, must be unambiguous. One could argue that the place where a person was "raised" is more important than the place where he or she was born -- and many authors list the former rather than the latter in their autobiographies -- but it's too nebulous for us to use because in many cases there is more than one location.

The second factor is relevance, e.g. DOB and DOD are relevant because they show when the author worked, help identify posthumous books, inform the reader that no, there probably won't be a sequel, and so on.

The third factor is how easy it will be to find the information. For example, a few years ago I considered proposing that we add a "Place of Death" field to the Author record, but concluded that it would likely remain largely empty for lack of data.

It would appear that "gender" satisfies the three criteria listed above. First, it's unambiguous in 99%+ of all cases -- much more so than, say, race, where biracial people are increasingly common and the lines are harder to draw. Second, it's also relevant as the topic is a perennial favorite among readers (see discussions and lists of "male urban fantasy authors" and such on Amazon and Goodreads), fans and academics. Third, in most cases it should be fairly easy to determine the author's gender.

Anyway, that's my story and I am sticking to it :-) Ahasuerus 04:57, 19 October 2013 (UTC)

P.S. I see that some of what I wrote above overlaps with Darrah's comments below, but it's 1am on the East Coast, so I will leave it alone for now and take it up again in the morning. Trying to get back to a more normal schedule... Ahasuerus 04:57, 19 October 2013 (UTC)

Michael, there is a saying that "honey attracts more flies than vinegar," in your case I'd have to say it's bile. It's stressful to read your posts, since you come across as an angry ranter, who is dismissive of anyone's opinion except your own. I try to put the best interpretation on what you write and when you cited Adolf Hitler, it did raise my eyebrows, but I interpreted it as a worry that entering the data would "out" people who wished to remain closeted. I've now decided it was a disingenuous expression of hostility. In the end I think you are upset because "you" are not the final arbiter of the matter.--Rkihara 04:28, 19 October 2013 (UTC)

FWIW, I find it very hard to determine the "tone" of some editors in the absence of supporting body language. I made a few mistakes early on (and I suspect that one of them may have contributed to an editor dropping out :-( ), so I try to be <Bugs Bunny>vewy vewy</Bugs Bubby> careful when dealing with written communications. Ahasuerus 05:05, 19 October 2013 (UTC)

Yes, you're right, I will try harder to reserve judgment. Your summary and Darrah's below seem to cover all the talking points. I asked for this feature since I've been working heavily on entering author data for the last two years (~12000 edits), and while doing this I've noticed that there is a large body of speculative fiction written by women in nineteenth and early twentieth centuries. The number is so large that I think a tally might upend our perceptions of how much women writers contributed to the early history of the genre. I've often wanted to analyze this data, and thought that others would too, but since gender is not identified, there is no way to easily do this. Darrah was worried that there are too few of us to do this properly, so I will say that if this change is implemented, I will spend a major portion of my time entering the gender data.--Rkihara 06:19, 19 October 2013 (UTC)

(unindent) Michael certainly has some points worth giving more consideration to. I'm very interested in issues of gender in speculative fiction. But why would we view gender differently than ethnicity/race, religion, or sexuality? We have anthologies devoted to Black authors, to Jewish authors, to LGBT fiction (usually LGBT authors), so these categories are clearly of interest to some SF fans. As with women, there is an (apparent) history of discrimination in SF against gays, authors of color and, some claim, against Christians. And I can certainly imagine some visitors wanting to search for authors from some of these categories. So why would we advantage people like Davin and me who are interested in research in gender, but not collect the data that other researchers and visitors are interested in?
One possible answer to this is that many of these other categories are small enough that it's more natural to document them by giving lists of "known" authors of color, LGBT authors, Muslim authors, etc. In most cases, the natural ownership of such original lists would be with other organizations (e.g. the Carl Brandon Society, Wikipedia's list of LGBT authors, etc.). If we did something along these lines, we should probably try to link to such other lists. But as Davin discovered, this is a surprisingly more difficult task for gender. And it may be that we are better able to deal with this issue than any other group.
Another distinction between gender and these other categories has to do with "outing" someone. Except for James Tiptree, Jr., I know of no other situation where "outing" someone's gender has caused problems. And if some editor decided based on the name that Dale Aycock must be a guy, it's likely that at worst it would result in someone asking us to correct that. Yet claiming that someone was gay, or straight, or Latino, or Black, when that was not true, or not "announced", could cause problems. So that's another reason to leave those other categories to groups that specialize in those issues. There is no other group for gender.
Michael says that he's not interested in participating in this data collection, but I'm sure that there many other aspects of the database that various moderators are uninterested in supporting. I won't enter books that simply publish on demand copies of out-of-copyright books and sell them. Michael, and other editors, are willing to do so. I'm sure that there are editors who wish we would drop the juvenile spec fic (I'm thinking of the "Let's kill off Daisy Meadows" fan club). But others are willing to enter these books, so we include them. In general, I think we should include data when we have enough supporters of that data to do it right. Including non-English publications is working fairly well, because we have several very active editors working on that. But I voted against Ahaseurus' earlier suggestion to include Ph.D. theses on SF partly because I don't think we could do it well. (By the way Mike, that's an example where "the one who rocks the cradle" was not the final arbiter.) At present, we seem to have about 3 editors who would be willing to work on this data (myself, Rkihara, and Ahaseurus). I'm not sure that's enough support to do the job well. So I'd prefer to wait on this unless we get some other editors who are willing to help as well. Chavey 04:50, 19 October 2013 (UTC)

Your position is quite clear and understandable. AFAIC, I can't separate so easily gender from other "personal" data (race, type of upbringing, religion, income of the household, level of education, etc.) when analysing an author's work or stance (cf. works by DeWitt -on race- or Klein -on socio-economic status-). I also think that, as Michael said better than me, the more we stray from the strict bibliographic data based on the publications themselves, the farthest we're from the (IMO) aims of our shared venture and the more increasing is the likeliness of conflict between the contributors. Hauck 09:58, 19 October 2013 (UTC)

As an aside, I'll mention that I've been working on the gender identification of authors in the database. To start with, I limited myself to authors who had published a book prior to 1985. I got through authors with last names A-D before my work stalled, trying to decide if there was a better approach. But my attempts to identify the genders of this subgroup of authors included identifying previously unknown legal names or partial bios for about 200 authors. For the results, see Gender of SF Authors. For the methodology I used, see Gender Identification Methodology. (Which is why I raised the question of how do we decide when we know enough about an author to assign a "known" or "presumed" gender.) Chavey 00:12, 20 October 2013 (UTC)

That's interesting, I've been thinking of taking a go at it myself as a side project if gender is not added to the ISFDB. I'm thinking of sifting through all authors though, starting with readily available pre-compiled lists like the ones you've listed. If I can get a dump of the data in the author directory (canonical names,no pseudonyms), along with web pages, and Wiki added, in comma delimited form, then I can move it directly into a database program and work from there.--Rkihara 06:18, 20 October 2013 (UTC)

It should be possible to download and install the latest ISFDB backup file linked from the ISFDB Downloads page. It has all the information that you list although you may need to write a query to extract just the subset that you are interested in. If MySQL is not working for you, Marty or I could create an extract, zip it up and e-mail it to you. Ahasuerus 13:30, 20 October 2013 (UTC)

If you could extract the data for me it would help a lot. I have limited programming background and installing Linux and learning MySQL would be a whole project in itself. I've experimentally extracted small portions of the Author Directory through cut and paste then cleaned up the data in Excel before putting it into Bento. Having the web page and Wiki info in addition would really speed up verification of gender.--Rkihara 17:39, 20 October 2013 (UTC)

OK, I have extracted all (?) relevant fields from the Author and Webpages tables and uploaded the results to http://isfdb.s3.amazonaws.com/backups/authors-and-webpages.zip . The column structure of the two files is as follows:

Authors: author_id,author_canonical,author_legalname,author_birthplace,author_birthdate,author_deathdate,author_wikipedia,author_imdb,author_image

Webpages: author_id,webpage_id,url

Empty fields contain "\N". Unfortunately, extracting the bodies of all Bio and Author pages from the Wiki would be significantly harder. In addition, embedded Wiki formatting would make them hard to read. I plan to move these pages to the ISFDB proper in the foreseeable future, but we aren't there yet.

Of course, like all snapshots, this one will gradually become obsolete as new data is added to the database, so it will need to be recreated at some point in the future. 22:42, 20 October 2013 (UTC)

Thanks! I was able to load it into Excel without problems and I'm massaging it before putting it into a database.--Rkihara 00:23, 21 October 2013 (UTC)

Great! Ahasuerus 01:14, 22 October 2013 (UTC)

(unindent) As I think more about this issue, my opinion as to how to provide information about the gender of an author is changing. Partly, I am influenced by the arguments Mike and Hauck have given. They argue, for example, that we should not treat gender differently than race, ethnicity, and other characteristics that may (or may not) influence the position of an author and of their works. I argued earlier that some of those characteristics might be more easily handled as lists of relevant authors. So why wouldn't we do this for gender? One reason might be that such lists would be quite long. But my preliminary Gender of SF Authors page would seem to imply that this is do-able. And a list of this form would give us much more flexibility in how we specify how much we know about someone's gender. No one in all of the comments above has addressed my question of "How much do we need to know about someone to claim their gender?" -- except for Michael who noted how easy it would be for an editor to "sex change" George Sands (or my example of Dale Aycock). I think this is a much harder question than most people think. Consider, for example, Diran Ajemian. Male or female? There are exactly six people in Wikipedia with the first name "Diran", all male. Is that enough to be sure that "Diran" is always a male name? Are we ever going to meet him, or find someone who knew him? Should we ignore him because he only wrote one novel? The answer to all those questions is "No". But "Diran" is also a famous historic king, and people named after him are almost certainly male, and this author is almost certainly male. So we should probably list him that way. But any simple field or check box in the author data will be unable to make distinctions between "we know him to be male", "he presents himself as male", and "we presume him to be male because of his name". Yet if we wanted to, a list would be fairly easy to use to make such distinctions (e.g., a tag/footnote for various people in the list). It also then becomes quite easy to deal with the 1% who do not fit easily into the gender binary. As Mike suggests, for the other fields in the author entry, we generally either "know" we have the information, or we "know" we don't have that information ("know" in quotes, since we are sometimes wrong). But with this fields there is rarely a "we think". And when we do, we always exile such speculation to the biographical wiki notes instead. This argues that we should handle gender information differently than we handle "DOB" information (etc.). I suggest some form of lists instead. Chavey 03:14, 22 October 2013 (UTC)

Sorry to chime in so late, but I've been spending a lot of time in hospital recently and not had good enough internet access to work here. I have to say I am rather split on the matter. Yes, there are people who would use the data - James Nicoll did after all - so if we did provide the info in database-readable format it wouldn't be a waste of time. It looks as though there's enough Moderator interest in entering the data that means those of us who don't care wouldn't have to do much (if any) work on it. I actually raised the question on LiveJournal and 'self-declared gender' seemed to satisfy most requesters. If people have already compiled their own lists then we could do a mass update rather than leave us looking foolish about the matter: although I suspect nobody has ever generated a list of male authors from our data, 'female' has probably been treated as exceptional. On the flipside, the simple solution will leave some unsatisfactory data - how far along the change is Poppy Z. Brite for instance? I'm NOT going to support a 'percentage male/percentage female' ratio for a drop down box, I agree that the matter is fairly straight-forward in 99% of cases - but it's the exceptions that are hardest to design/code for. Overall, I think I support the idea but wouldn't actually work on it myself. I remember making assumptions about Andre Norton and Julian May that turned out to be wrong - a definitive answer here would be a help, although HOW definitive it is probably deserves a field of its own for the source. If we can leave it to "someone else's problem" to POPULATE the data, I'm in favour. Forcing the issue to be resolved on Edit Author and I'd be against. And there will of course be almost-immediate requests for sexual orientation as the next problem, but that can be a separate issue. BLongley 04:53, 22 October 2013 (UTC)

A few thoughts:

Gender, if added, will be, just like DOB and DOD, an optional field and the default will be "-" or "unknown". In some cases it will be inapplicable -- e.g., consider Ilona Andrews -- so we couldn't possibly make the field required even if we possessed godlike omniscience :-)
"A list of male authors from our data" may be useful in certain cases. For example, there have been numerous Goodreads, Usenet and Amazon discussions of "urban fantasy by male writers".
Although gender may "influence the position of an author and of their works" -- just like any other number of things can do -- I think a more important question is whether the data is unambiguous and available. Whether an author studied at Oxford or in central China may have influenced him or her, but collecting this type of information is non-trivial and would require maintaining a list of schools and supporting multiple schools per person. It's not impossible to do in a structured manner -- LiveJournal and Wikipedia support the capability -- but it's a great deal of work for a fairly marginal (for our purposes) data element.
Re: the difficulty of establishing who is male and who is female, I agree that there are obscure as well as misleading situations. There is certainly no need to jump to conclusions based on insufficient evidence and the rule "when in doubt, leave it blank" should apply. On the flip side, I frequently encounter new authors (usually found by Fixer) whose Web pages provide next to no ISFDB-useable information but mention the author's gender. Every time this happens, I mutter "If only we had a gender field in the author table, I could at least record something useful about this author".
Re: the feasibility of organizing this information as one or more lists, it's certainly an option. However, there are numerous advantages to having the gender data in the main database. It prevents the two data sets from getting out of sync as we change canonical names, merge authors, etc. It also enables SQL queries which let researchers generate complex lists in seconds. And finally, any kind of comprehensive list is liable to become eventually unwieldy since we have over 113,000 author records in the database.
My main concern is with privacy. People may have any number of reasons to hide their identities or specific information about themselves. Sometimes the reasons are commercial, e.g. a publisher may ask a less-than-successful author to use a pseudonym to fool the distributors' computer algorithms, which may not order a new book by a writer with a history of declining sales. Or the publisher may ask an author to use his initials on an urban fantasy novel to obscure the fact that he is male, which happened to Tim Pratt just recently. Other times it's an experiment like the one that J. K. Rowling did a few months ago. Or it could be a personal thing like with "Christopher Pike", the mega-successful juvenile/YA author who reportedly didn't want his pseudonym disclosed because he had kids in school and was worried that it would make their lives difficult. And sometimes it can be more serious like in the case of "Ivan Kirov". If we were to accidentally disclose the gender of a pseudonymous author, we could inadvertently blow his cover, which could be bad, especially in countries like China or Iran. On the other hand, as long as we religiously adhere to our rule about using only publicly available data, I think we should be fine. Ahasuerus 23:13, 23 October 2013 (UTC)

Patch r2013-138 - Final mouseover help changes

Clone Content has been modified to use the same kind of mouseover help that other data entry forms use. In addition, the mouseover help text for "Interview Page" has been fixed to say "interview" rather than "review" (at no extra charge!). Ahasuerus 02:58, 18 October 2013 (UTC)

Patch r2013-140 -- New Pub -- pop-up validation added

The New Pub (New Novel, New Anthology, etc) data entry forms have been modified. The following types of pop-up validation have been added:

Publication Title - existence
Publication Author(s) - existence
Publication Year - existence and format
Publication URL - format (links to ISFDB Wiki pages rather than to ISFDB-hosted files are disallowed)
Contents dates - format

In addition, the new validation code checks that:

each Contents "title" record has at least one author
each review has at least one reviewee and at least one reviewer
each interview has at least one interviewee and at least one interviewer

The software changes needed to add this functionality were not only extensive, but also rather convoluted, so they are limited to the New Pub forms for now. If everything looks OK this week, I will add the same validation logic to Edit Pub, Clone Pub, Add Pub and Import/Export Contents. Ahasuerus 00:37, 21 October 2013 (UTC)

Patches r2013-141 and r2013-142

A few bugs have been fixed. The only change in the user-experienced behavior is that the data entry validation/massaging logic has been changed to ignore spaces in dates. In the past, "2013 - 11 - 00" was considered an error and resulted in silent conversion to 0000-00-00 at submission creation time. The new logic gets rid of all spaces, e.g. "2013 - 11 - 00" is converted to "2013-11-00", which should help reduce the number of "mystery" 0000-00-00 dates in submissions. Ahasuerus 02:11, 22 October 2013 (UTC)

Fanzines

Just in case it leads to a massive increase in Fanzine activity, I should mention that I've just arranged a link back from efanzines.com. So blame me, not the new users here. BLongley 11:32, 23 October 2013 (UTC)

ISFDB outage 2013-10-26

I will have to take ISFDB down for about 15 minutes starting at 1pm server time. Ahasuerus 17:46, 26 October 2013 (UTC)

Done. Ahasuerus 18:15, 26 October 2013 (UTC)

Ahasuerus - internet outage

FYI, I am internet-less at the moment. Ahasuerus 20:36, 28 October 2013 (UTC)

And so was the database. I'm not sure how long it lasted but wasn't accessible for at least 20 minutes. Or maybe it was just my ISP. Mhhutchins 20:53, 28 October 2013 (UTC)

There was a mess-up with the backups schedule and they ran at 1pm server time rather than at 2am. Sorry about that! Ahasuerus 22:05, 28 October 2013 (UTC)

Patch r2013-143

Edit Pub and Clone Pub have been modified to display pop-up error messages when required data is missing. In addition, it's no longer possible to cause an error by entering all spaces in a field when editing a pub or a title. Ahasuerus 23:57, 29 October 2013 (UTC)

Brief database downtime 2013-10-31 at 3:30pm server time

I will need to take the database offline at 3:30pm server time. It should be back within a few minutes. Ahasuerus 20:26, 31 October 2013 (UTC)

Done. Ahasuerus 20:34, 31 October 2013 (UTC)

Maximum length of publication fields increased

The maximum allowed length of the following publication fields has been increased to 100:

Price
Pages
Catalog ID

Ahasuerus 20:35, 31 October 2013 (UTC)

Proposed change to the Edit pages

What do you think of the following change to all Edit pages:

Ahasuerus 22:12, 31 October 2013 (UTC)

I like. Useful for those those long titles and Image URLs. Will the text in the Note box be in typewriter font, or sans serif as normal? PeteYoung 00:57, 1 November 2013 (UTC)

There will be no change to the fonts used, so the font will be whatever your browser currently uses to display "text area" (multi-line) data entry fields. Most browsers let you control which font they should use for different types of data (proportional, fixed, etc), so you can tweak it on the client side. Ahasuerus 01:20, 1 November 2013 (UTC)

I must be getting old. You'll have to point out the difference as nothing other than the width of the fields jumps out at me. Mhhutchins 03:22, 1 November 2013 (UTC)

That's exactly it! :) Ahasuerus 03:42, 1 November 2013 (UTC)

Looks good. Chavey 04:56, 1 November 2013 (UTC)

It may be cosmetic, but I like it. Make it so! BLongley 07:44, 8 November 2013 (UTC)

I also didn't get the improvement first, but being now in the know I like it. Stonecreek 13:44, 8 November 2013 (UTC)

New editor having trouble editing the wiki

Can someone figure out how to solve the problem this new editor is having? He created this submission in order to let us know of the problem. I have no idea how the registration process works and why an editor isn't able to edit the wiki. I thought anyone can edit a wiki, even if they're not even registered. Thanks. Mhhutchins 22:37, 1 November 2013 (UTC)

The submission says that we "are not sending [him] verification email". I assume he means that although he has created a user name, he hasn't received a verification e-mail. If so, then he will be unable to edit the Wiki until he gets the verification e-mail and clicks on the "confirm" URL embedded in it -- that was the whole point of beefing up our anti-spam safeguards a few months ago. It's possible that his e-mail client either blocked the auto-confirm e-mail or moved it to the spam folder. Let me check his account settings and see if there is anything unusual about them... Ahasuerus 23:01, 1 November 2013 (UTC)

His account had the "do not send me e-mail" flag set, which may help explain the problem. I went ahead and manually set his "account confirmed" flag, so hopefully he should be able to edit his Talk page now. I will leave a note on his Talk page... Ahasuerus 23:15, 1 November 2013 (UTC)

Displaying non-day-dated publication records

Is it possible to have non-day-dated publication records displayed as if they were published on the "32nd" day of the month? That way the second printing in the same month, but not dated, would not be displayed before first printing which is day-dated. For example, a first printing on 2013-10-01 would be displayed before a second undated printing which appeared before November, thus dated 2013-10-00. The current display logic would list the second printing first. See the Del Rey editions of this title and this discussion which prompted the suggestion. Thanks. Mhhutchins 19:50, 2 November 2013 (UTC)

Sure, it's possible. I will create an FR, but I won't be able to implement it immediately because I am currently working on testing/packaging a large volume of submitted changes. Ahasuerus 22:36, 2 November 2013 (UTC)

No rush. I've been meaning to request this for a while, but more pressing issues have been pushed ahead. Thanks. Mhhutchins 23:06, 2 November 2013 (UTC)

Development server problems

It looks like the development server is about to lose its hard drive. If I disappear for a day or three, chances are that I am rebuilding/restoring the server. Ahasuerus 01:11, 3 November 2013 (UTC)

A new drive has been installed and we are back in business. Ahasuerus 02:10, 7 November 2013 (UTC)

Patch r2013-147 - Issue Grid fixed

The "Issue Grid" page, which contained a lot of bad and/or obsolete HTML, has been fixed. The main table should no longer require horizontal scrolling, which was an issue with some browsers. The only functional change is that if a cell contains two issues, e.g. "Astounding" and "Astounding (UK)", the border around the embedded cells is now drawn with a single line rather than a double line. Ahasuerus 02:10, 7 November 2013 (UTC)

House tour - Arthur C. Clarke

This was fun: http://asiaobscura.com/2013/11/sifting-through-arthur-c-clarkes-dvd-collection-in-colombo.html
--Marc Kupper|talk 20:22, 7 November 2013 (UTC)

Well, thank you very much for that link! Truly interesting. Stonecreek 20:30, 7 November 2013 (UTC)

Novacon, anyone?

I'm out of hospital at last, just in time to attend Novacon. Is anyone else from here going there? I'll be the one with a subtle ISFDB T-Shirt. (Design stolen from Al's banner.) BLongley 07:28, 8 November 2013 (UTC)

Damn, wish I was. Alas, I left the country two days ago! Have a good time, don't let Steve Green buy you too many drinks. :\ PeteYoung 07:33, 8 November 2013 (UTC)

It was too many drinks that put me in hospital, I'm not anxious to repeat the experience. :-( Who is Steve Green and why would he buy me even one? Is he a big ISFDB fan? BLongley 08:16, 8 November 2013 (UTC)

Well, I met him, and he's certainly a BIG fan. We got off to a bad start when I mentioned that I'd been warned about him - he rapidly resolved that by not buying me any drinks at all. :-/ BLongley 21:51, 11 November 2013 (UTC)

Given recent history, is that really such a bad thing? :) Ahasuerus 22:58, 11 November 2013 (UTC)

Well, prices were 20p a pint more than I usually pay so I'd gladly have had someone else pay for them! But overall, I drank far less than expected - leaving a large set of books unattended makes you anxious not to spend too long at the bar. Which reminds me - would you be willing to do a mass update from Primary to Primary Transient for me? I sold 8 boxes of stuff, and that was just the "A"s and "B"s. I've acquired a new hobby, dealing books: not so much for the money, but you get to talk to a lot more people that I wouldn't normally have approached. Not sure when I'll deal again, but there's plenty of "C"s and "D"s to go next... BLongley 00:31, 12 November 2013 (UTC)

We got a bit better when discussing you, he thinks you're a nice guy too. He might mention me next time you talk. BLongley 21:51, 11 November 2013 (UTC)

BTW, here's the subtle shirt design.

. It's rather clearer on the shirt itself, but I didn't want to ask permission for using the Mod's Email address, and even omitted my own name/email. Is anyone else doing something to promote ISFDB? If we wanted dozens of new contributors all at once then business cards or suchlike would be helpful - especially if we promoted it in different languages. BLongley 08:29, 8 November 2013 (UTC)

I'm now thinking that a simple CD containing a recent DB backup and MySQL would allow some inventive uses. Too late for me to create one for Novacon, but could do so by Eastercon time. There's several levels of use we could do: 1) data and appropriate mining tools, 2) full read-only application, 3) Test system for checking change's effects, 4) full Development system. At the moment, I think our most pressing needs are for more non-English Editors and Mods and we can then become truly THE Internet Speculative Fiction DataBase. BLongley 08:43, 8 November 2013 (UTC)

First of all, welcome back to the land of the living! Ahasuerus 02:02, 9 November 2013 (UTC)

Thanks. Two new drugs more and two new inhalers later, I can testify that medicine is helping me with everything apart from my memory, my attention span, and my memory. And my attention - ooh, look! A Butterfly! BLongley 06:56, 9 November 2013 (UTC)

As far as CDs go, my first reaction was that although there is no harm in them, it's doubtful that anyone who would find it difficult to configure a local copy of ISFDB (for use cases 1, 3 and 4) using internet downloads would benefit from a CD. However, if we could create a single-click self-installing application for use case #2, it may be quite useful. The question then is how many people want a copy of ISFDB as a "full read-only application"? (We do need to update the installation pages regardless of anything else, especially the "Linux" page, which hasn't been touched in years.)Ahasuerus 02:02, 9 November 2013 (UTC)

Each level comes with its own problems. I use TOAD for data-mining, but some people might prefer a more GUI interface. Use case 2 is becoming less compelling as people get used to 3g or 4g phone services. I'm still semi-Luddite about such, the cutting edge technologies are usually priced too high for my liking. And the requirements for off-line tools are different: I used to use ISFDB offline as a checklist of things I already owned. But I can foresee a future where people take a photo of a book cover for our purposes without having to buy the whole thing: or saving the data for later upload when you can't get a reliable phone service. A 'one-stop shop' would be brilliant for those that like a single App. BLongley 06:56, 9 November 2013 (UTC)

By the way, Dirk Stoecker has submitted a large volume of changes, hundreds of them, and I am currently in the process of testing, packaging and deploying them. Once fully installed, they should make almost all of our Unicode issue go away. I expect that it should help a great deal with attracting new editors. Ahasuerus 02:02, 9 November 2013 (UTC)

We've done very well with our newer non-English editors, but we still have to train some more mods to deal with them. I suspect when a Community portal message is posted in an entirely different alphabet, and gets answered, then it might be time for me to retire and READ some of the books I have. We've lost a few editors that were beyond our help - I recall a Bulgarian editor that retired after a while. BLongley 06:56, 9 November 2013 (UTC)

You mean that books can be READ ? Hauck 12:52, 9 November 2013 (UTC)

Yes, they can. I remember reading a few thousand before I discovered that some CAN'T be read until you learn a foreign language. :-/ BLongley 21:51, 11 November 2013 (UTC)

Steve's a BSFG member, I think he's chaired a few Novacons in the past. Bruce Gillespie asked me to write something about the ISFDB for SF Commentary, so I wrote an article for SF Commentary #85. He edited it down considerably and it ended up as a LoC instead, but no complaints. That should reach a lot of people. Well, those who read the letter columns. PeteYoung 09:14, 8 November 2013 (UTC)

Looking for serious fans of Robert Bloch and James Blish

I just added the entire run (all 1 issue) of the 1946 fanzine "Science*Fiction", edited by Judy Zissman -- better known as Judith Merril. The fanzine includes an otherwise unpublished essay from James Blish and (apparently) unpublished short stories from both Blish and Robert Bloch. (The Bloch story is called "But Doth Suffer", and the Blish story is "Knell"). It is very common, though, for a story to change its name when/if it gets published in a "regular" magazine. If that's the case with either of these stories, then I would want to variant it to the canonical title. If you think you would recognize a Robert Bloch short story, or a James Blish short story, from a plot synopsis, then come over to my page (or invite me to yours) and see if we can identify either of these stories. Thanks, Chavey 09:09, 8 November 2013 (UTC)

2013-11-08 patches

FYI, I have installed a number of patches over the last few hours, but the changes have to do with the underlying HTML and there should be very little impact on the user-experienced functionality. Some dividing lines may have moved up or down a pixel or two, but it should be barely perceptible. The only exception is the change to the "New Pub" and "Clone Pub" approval pages where moderator will now see multiple authors separated by a blue (rather than a regular) plus sign. Ahasuerus 01:45, 9 November 2013 (UTC)

ISFDB and the development

For the last 2 weeks I had intensive discussions with Ahasuerus with lead to nothing. For any request I made I got the answer that the developers don't have time to do anything. So in the last 3 weeks I did not request, but do something. I added lots of improvements to the ISFDB code and did setup an demonstrator at http://isfdb.stoecker.eu/ (logins moderator1 to moderator5 with password pmoderator1 to pmoderator5 and user1 to user9 with passwords user1 to user9).

Some example changes:

proper support for non-latin author names: http://isfdb.stoecker.eu/cgi-bin/ea.cgi?196094
author overview with all titles: http://isfdb.stoecker.eu/cgi-bin/ea.cgi?id=51&lang=ALL
author overview with German titles: http://isfdb.stoecker.eu/cgi-bin/ea.cgi?id=51&lang=German
transliteration of non-latin names: http://isfdb.stoecker.eu/cgi-bin/ea.cgi?196094 (move your mouse over the green "i")
allow i and ï in names: http://isfdb.stoecker.eu/cgi-bin/ea.cgi?196095
better search support: Search for "ï".
function to import individual titles: click on import titles for any publication
style unification for display (see above "Proposed change to the Edit pages")

Changes under the hood:

proper unicode support - E.g. submit a text with russian character in the title and look at the submission comment - no longer cut in the middle of a word (see e.g. the recent changes section)
system to properly import/export XML
cleanup DB access in many cases, removing obsolete code
fixed unit tests
Proper encoding support for HTML output clearly marked, so that security issues are easy spotable and fixable
remove hardcoded design with proper style based approach
replace db access by a security improved version which do not haven hundreds of security holes
joined some duplicate functions (there are many more)
commented CSS usage
replace direct code with proper function calls when one existed
cleanup of many HTML errors
Added some generic functions (like ISFDBLink) to hide implementation details
All new methods are optional, old ones still work, but the new ones improve quality, maintainability and reduce security issues.

Plans:

Add proper support for story translators (i.e. handle translators as additional part of a story like an author, display translations at the translation-authors page)
Add preview of changes and better display of the actual effects of a change submission
Fix all the obvious security holes (it is easy to inject changes into the database without anybody noticing it)
Cleanup database access and allow switching to proper unicode support
Better support for non English users, maybe even including language support for the interface itself

Still I don't believe that any of the important changes I already did or any of my plans will ever become reality. ISFDB's current development model actually means that one already work-overloaded man handles every code submission and takes weeks to check any change. There wont be any progress with that method. Instead of reducing the amount of work, everything I do will increase it.

So this text is to ask whether the whole community believes that it is better to have:

the current extremely slow progress with extensive code review and hundreds of existing security holes or
progress and update of the system and maybe a seldom bug here and there, but obvious security issues fixed.

For two weeks now I'm convinced that all modifications I made will not make it into the actually used system, but I don't want to base my decision only on the word from one person, so I ask here as well. --Stoecker 11:47, 9 November 2013 (UTC)

I'm one of the developers, although not very active nowadays. In defence of Ahasuerus, I would say he's one of the best Bureaucrats we could wish for - he understands the problems with non-English languages and even other alphabets. (Though Arabic and Cantonese may be beyond even his skills.) I would plead for patience - when he was working on paying stuff his time was even more valuable, and things have noticeably improved now he's retired. I recommend giving him lots of time, and encourage him to appoint one or more people he feels capable of doing the job - he's not going to be around forever. BLongley 12:47, 9 November 2013 (UTC)

I don't want to replace Ahasuerus or anybody else. I have enough other things to do. I'm maintainer of software projects, administrator, software developer, packager for openSUSE Linux and so on. I don't want to take over anything which somebody else already does. But that said it is clear, that exactly "giving a lot of time" is not what a can or will do. --Stoecker 13:36, 10 November 2013 (UTC)

Two weeks is a VERY short term in ISFDB matters - even for severe security problems. When I was most active as a developer I could code and test 2 or 3 fixes a day, and they might take MONTHS to be implemented. More developers are needed, but trusted testers are too - and we're short on multi-lingual testers. I'll have a look at your site and see what's coming up - sometimes a core change has ramifications not visible immediately. I wouldn't want to see things broken accidentally so I will support the "develop"/"test"/"retest by someone else"/"implement"/"be capable of reverting a bad change" model still. Ahasuerus has already mentioned you as a valued contributor, please continue your good work. BLongley 12:47, 9 November 2013 (UTC)

I started development at all, because for half a year since I started contributing to ISFDB nothing happend and for me ISFDB does not doat all what I want from it. Even after contribution thousands of changes I'm still in the step of "maybe it gets useful one day". There are so many problems from my point of view that if I spend an reasonable amount of time it would take me probably half a year to fix them. With current development model that would mean altogether something about 5 or 6 years implementation time. This is nothing I'm willing to accept. The current set of changes is my usual test to see if I can and will work with a project or not. I implement something and than evaluate the reaction. Currently I don't see that I can work with the way ISFDB code base is managed. --Stoecker 13:36, 10 November 2013 (UTC)

I am a professor of Computer Science, with a specialty in Software Engineering. I am strongly in support of "extensive code reviews". I have seen a few instances where Ahasuerus has gone to the trouble to list all of the implications and cascading changes that have to occur because of an "apparent" simple change. These examples, along with all those I know from my own study, convince me that it is more important that changes be done correctly than that they be done quickly. I also appreciate all of the work you're doing, but an enterprise like this requires patience. Chavey 20:15, 9 November 2013 (UTC)

Sorry - One of these professors which realised that Open Source actually works only years after major businesses failed to compete with these "unorganized bunch of hobbyists". My current major project is JOSM, an opensource project with no extensive code reviews but a high dynamics in the highly dynamic OpenStreetMap area and a code quality which is hard to match in commercial software development (which I actually know, because I'm commercial software developer for more than a decade as well). --Stoecker 13:36, 10 November 2013 (UTC)

(unindent) A lot of things have happened since this discussion which started the e-mail exchange between Dirk and me. Briefly:

Dirk implemented a significant number of changes and deployed them to the new publicly accessible test server (see above).
I did limited testing on that server (and on my private development server) and reported the bugs that I found.
The bugs were fixed and I have a copy of the modified software installed on my development server.
I have been testing, packaging and deploying the changes for the last week+, although the process was interrupted about a week ago by a hardware failure on my development server. This includes things like the change to the way the New Submissions page looks, although most of the deployed changes have been of the “under the hood” variety.
In the meantime, Dirk continued adding new changes like transliteration support.

The good news is that it quickly became apparent that Dirk is a very experienced and capable developer who produces a high volume of quality code. There are occasional bugs here and there, but nothing major. If anything, he is much more knowledgeable and productive than I will ever be given the technologies that ISFDB uses. And he has certainly made more changes in the last few weeks than I have made in the last few months, if not the last year.

Having said that, we do have a major philosophical difference that Dirk mentioned above. The current development process uses the following model:

All changes affecting user-experienced functionality are discussed and approved on the Community Portal ahead of time.
All changes are carefully tested before they are installed.
To the extent possible, all changes are broken up into small chunks.

In other words, “look before you leap”, “slow and steady” and so on. The process is currently further slowed down by the fact that I am the only active developer/tester and that I spend a significant part of my ISFDB time on supporting Fixer and maintaining the main server.

Dirk, on the other hand, comes from a very different, “he who hesitates is lost”, “fast and furious” open source background. Apparently, the approach works well for certain types of projects and/or for certain types of developers even though traditional developers and managers like me have been historically leery of it. Unfortunately, my understanding of the specifics is limited at best and woefully incomplete at worst -- I have seen a few “fast and furious” projects in action and they were not successful, but that doesn’t tell me much about the ones that have been successful. And I certainly don’t have a gut feeling for what works and what doesn’t work under this model, so I really wouldn’t be the right person to manage a “fast and furious” project. As I told Dirk, it would be similar to putting a retired Army colonel in charge of a rock music concert :-) Ahasuerus 21:01, 9 November 2013 (UTC)

You overestimate the "fast and furious", but in principle you're right. But I don't want that you drop you behavior completely, but it is also clear, that we can't cooperate with your current approach. I'm very experienced at what I do and I can only help you and the project when you accept my experience. --Stoecker 13:36, 10 November 2013 (UTC)

The problem that I am facing with Dirk’s submissions is the sheer volume and variety of changes. They include improving support for non-Latin/ASCII characters, security fixes, HTML/XML fixes, new features, internal cleanup and so on. The result is that almost every page has been affected, sometimes in dozens of places and some scripts had over 70% of the code changed. Ahasuerus 21:01, 9 November 2013 (UTC)

While the amount of "changes allover the software" will rapidly drop, as there aren't any such essential change necessary anymore if current code would be accepted, if I continue it may be the case that in 2 or 3 years probably 90% of the code will have been replaced (step-by-step). Newer software development processes actually encourage such behavior. Not the code itself, but the functionality (i.e. the interface) is important. What is important in my eyes is the fact that there never will be a "break" - i.e. users will never notify these code reworks, only when the results are bug-fixes and improvements. —The preceding unsigned comment was added by Stoecker (talk • contribs) .

This raises two issues. First, we generally discuss all functional changes before we implement them. For example, I posted a screenshot of the New Submissions page on the Moderator Noticeboard before implementing the change that Dirk had submitted. Similarly, I posted a screenshot of Dirk’s changes to the Edit Pub page (see above) and waited for feedback before I started testing it. In Dirk’s case it’s exacerbated by the fact that he is relatively new to the ISFDB, so he is still familiarizing himself with some of the less obvious features of the software. For example, he didn’t discover the ability to see all translated titles via User Preferences until after he had implemented similar functionality in a different way. Ahasuerus 21:01, 9 November 2013 (UTC)

That you did understand wrong. I had not known about that feature before I looked at the code (which tells you something about the quality of the UI when I need to look at the code to find a feature). After a look in the code I also found the user interface. What I did implement is the possibility to use that feature also for user not logged in (which is what I need to link to the correct pages e.g. from Wikipedia). I really can READ source code. It tells me much more than any of the help pages of ISFDB together. --Stoecker 13:36, 10 November 2013 (UTC)

The second issue is that I have two choices when faced with a submission that would change almost every Web page, sometimes massively. I can either apply all of the changes at the same time and hope that nothing gets seriously broken, i.e. the “fast and furious” approach, or I can extricate small groups of related changes, test them separately and deploy them one group at a time. As mentioned earlier, our current SOP is to do the latter, but a major submission like the one that I am dealing with right now has a lot of dependencies, so extricating self-contained changes is time-consuming and somewhat error-prone. It’s doable in the early stages -- and I have been doing it for the last two weeks -- but a highly productive developer like Dirk can quickly get ahead of the testing process and the discrepancy between the submitted code and the production code will eventually become so large that the testing process will slow down to a crawl. That’s why the other day I suggested that Dirk stop for a while and give me time to catch up.

So, we have reached a fork in the road, as it were, and we need to decide which way we go. The options as I see them are:

We keep the testing process unchanged and I continue plugging along until all of the submitted changes have been tested and deployed. I will probably run a few of the most crucial ones by Marty to make sure I didn’t miss anything.
We change the testing process along the lines that Dirk suggested above. Code reviews and change compartmentalization will no longer be done, but we will still be able to test newly coded changes using the publicly accessible test server before they go live. There may be more (hopefully minor) bugs than in the past, but the development process and the bug resolution process will be likely much faster. Ahasuerus 21:01, 9 November 2013 (UTC)

To be more precise. The testing based approach does not omit code reviews. I review each and every of my changes before I submitted them to Ahasuerus, some of the bugs he spotted are because I use my test system for development, so he tested a system "in-development". --Stoecker 13:36, 10 November 2013 (UTC)

In a way, I am almost tempted to go with #2 because it would let me largely get out of the testing business, which is very time-consuming and more than a little tedious, and concentrate on Fixer and other more pleasant activities. Perhaps I could even take a vacation, the first one in 8 years! :) On the other hand I am leery of “great leaps forward”, especially since I don’t fully understand when/why the “fast and furious” approach works and when/why it doesn’t.

So, do we go with the current bird in the hand or do we try for the two in the bush? Do we have anyone here who has experience with successful “fast and furious” projects and who would like to share his insight? Ahasuerus 21:01, 9 November 2013 (UTC)

I would go for staying with the current system, and hope that Dirk and any future developers understand the rationale behind that philosophy, and be able to fully participate in the process. Mhhutchins 02:47, 10 November 2013 (UTC)

No chance with that. I'm pretty aware of the advantages and disadvantages of different development models. Probably more than any of you, because for many years I now develop in both worlds. The current ISFDB model I can accept in commercial development, but not in my free time. So when you keep that model you have what I have done till now, but nothing of what can be done. --Stoecker 13:36, 10 November 2013 (UTC)

BTW, the changes implemented in the last patch (see below) are a good illustration of the value of code reviews. If you go to the public test server -- which is currently running the version of the ISFDB software submitted by Dirk two days ago -- and pull up a random publication in Edit Pub, the Add Author/Add Artist buttons at the top of the page will work fine. However, if you scroll down to the Content section and click on "Add Title" and then on "Add Author", the added Author field will be too long and will mess up the page. If you try to clone the same pub, not only will you encounter the same problem in the Content section, if you click on "Add Author" or "Add Artist" in the top section, the field length will also be different. You will find similar problems with Add Author/Reviewer/Interviewer/etc in Import/Export Contents, Make Variant, and Add Publication.

This isn't to say that Dirk did a poor job with his changes (I made similar mistakes when I began working on the software), but it goes to show that there are so many interdependencies in the software that code reviews can be beneficial. Ahasuerus 04:30, 10 November 2013 (UTC)

P.S. Of course, this isn't a big deal since field length is cosmetic and doesn't affect the underlying data, but a problem with the process of filing data into the database would be a different matter. Ahasuerus 04:52, 10 November 2013 (UTC)

As said I also use this as development server. The "transliteration" is my newest feature and not yet submitted to Ahasuerus. I didn't test my newest stuff much, as I anyway assume it will never be used. This was more a test to see if it actually is possible and what work it takes. I promised to feel responsible for my currently submitted code and fix any bugs detected, but not for any new stuff. --Stoecker 13:36, 10 November 2013 (UTC)

Ah sorry, that has nothing to do with transliteration, but is a style issue. This is to be expected as long as parts of the code use proper style and others old hardcoded mechanisms. I told Ahasuerus by mail that all the HTML based changes are merely a drive-by fixing of display issues I found during testing my major code changes. I did not really spend time on these (except for seriespub page), as they aren't worth the effort when the major stuff does not get accepted. We can outweight that minor offset with the fact that the Note-field of EditPub field now has the same width as the input fields below and above. --Stoecker 13:49, 10 November 2013 (UTC)

And another look. What do you mean? I don't see a difference to www.isfdb.org. There the editable and non-editable fields have different size as well. --Stoecker 14:18, 10 November 2013 (UTC)

If you follow the sequence of events that I outlined above starting with "pull up a random publication in Edit Pub, the Add Author/Add Artist buttons at the top of the page will work fine. However, if you scroll down to the Content section and click on "Add Title" and then on "Add Author", the added Author field will be too long and will mess up the page", do you see the problem? Ahasuerus 17:31, 10 November 2013 (UTC)

What do you mean :-) No, I fixed it now. Introduced CSS style "contentinput" for these fields instead of reverting to hardcoded stuff. Set it to 335px, which matches my browser, may still be small differences on other browsers until all fields use that style. --Stoecker 22:34, 10 November 2013 (UTC)

If you try to edit this interview and click on "Add Interviewee" or "Add Interviewer", the length of the added field is wrong. Ditto if you try to edit this review. In Edit Pub, if you click on "Add Author" in the Content section, the length of the Author2 field doesn't match the length of the Author1 field. Ditto "Add Author"/"Add Reviewer" in the Review section and "Add Interviewee"/"Add Interviewer" in the Interview section. The main "Add Author" and "Add Artist" buttons on the Clone Pub page still have the original problem and so does "Add Author" in Make Variant/Add Variant Title and "Add Artist" in Add Publication. When you download and install the changes that I had made before I deployed to the main server yesterday, you will see what I had to do to make everything work.

Again, this is not to suggest that you write bad code, but there are many dependencies in the software. It's easy to break half a dozen things when making a single change in a seemingly unrelated part of the code. It took me a while to learn where the landmines were and how to avoid them, which is part of the reason why I am in favor of code reviews. Ahasuerus 00:01, 11 November 2013 (UTC)

Actually I had only partly introduced proper CSS. It is to be expected, that pages may have quirks until this is complete. --Stoecker 16:35, 11 November 2013 (UTC)

I don't think display errors are inevitable when introducing new features or replacing the code "under the hood" -- note the current behavior of the software on the live server. As you wrote earlier, "users will never notify [notice] these code reworks, only when the results are bug-fixes and improvements". Ahasuerus 22:56, 11 November 2013 (UTC)

Nobody said "inevitable". Sure - with proper checking you will reduce them. Especially a good idea is not partially implementing CSS changes. :-) But these CSS stuff were "minor" things for me - I'd care for them only after the big ones. They don't break functionality and can be simply detected and reported. Usually I even modify the CSS values in a way, that any missing updates result in bigger display distortions, so users find and report them faster. --Stoecker 17:49, 12 November 2013 (UTC)

I finished only one CSS issue, the "id" to "class" for these multiple used arguments. I told you that these HTML stuff is a minor issues for me, which I don't take serious as long as the important things aren't accepted. Thought you gave me a challenge, here is your result: [1] (again I did not care for other scripts which still use hardcoded sizes!). All elements properly aligned (except for the buttons) and only one place to change the layout when you want to make it totally different. Still has HTML errors due to JavaScript. I've seen your long/short changes regarding the Javascript. Even if using CSS you still think to much in layout terms. Proper name of the argument would have been "meta"/"content", i.e. semantic and not layout based. --Stoecker 16:35, 11 November 2013 (UTC)

Oh no, it wasn't a challenge, just an example of the perils of the proposed approach. As you wrote below, you "start[ed] ... discussion with a bunch of working code and not with words only, so you have something to evaluate" and "you don't need to take my word, evaluate what I did". So that's what we are doing here. Ahasuerus 22:56, 11 November 2013 (UTC)

So please simple ignore smaller style nuisances. I told you, that these are a drive-by, partial and unfinished and I will not care for them except the larger stuff is either accepted or in the process of accepting. No reason to care for details when the whole system is rejected. Most of these issues are easy fixable, as you simply can search and replace most of them with automatic text editor functions. I did not do that, as it would have increased the amount of stuff I sent by another 30% probably (and hard to review intermixed with functional changes). You would not have noticed a single of them when I'd not have choosen 600px, but a value close to 335px, which is nearly the hardcoded value (see above about exposing style issues). --Stoecker 17:49, 12 November 2013 (UTC)

Well, I was at the user end of installing a 'fast and furious' data warehouse project, and it had to be shut down for one or two months because this didn't work. (This wasn't a major problem because the previously installed tools still worked.) Now we work less with numbers and more with texts, but one of the problems was the interdependency of one field to several others, which may be refelected in ISFDB (pseudonyms).

As we don't seem to have a second fail safe system, I would say that the 'fast and furious' approach should be tried out first only in part, where it can do as less harm as possible. Maybe this would be a way to progress in the future? Stonecreek 12:39, 10 November 2013 (UTC)

I already told Ahasuerus the same when he did tell me about failures. You don't need to take my word for it, you actually can test it yourself. I don't plan and also Ahasuerus would not accept any system, where we "develop for ages and then deploy the fancy new system and wait for it to fail". What I do are incremental changes like Ahasuerus also does. But we have a rather different understand about the speed of development and what an "increment" is. --Stoecker 13:36, 10 November 2013 (UTC)

Sorry, this is going to be long-winded, even by my standards.... I am a software engineer by degree and profession. I have implemented small and large commercial applications and systems, architected such systems, run small and large software development projects, and I have also been involved (mostly as a contributor) with various community/open source software projects.

I think there are two primary questions that should be asked and answered. Their answers will guide approaches taken.

How much are "we", which might be ISFDB's parents, the extended pool of caretakers (i.e., moderators), or the entire ISFDB community, willing to tolerate the risk of poorly or incorrectly functioning code? "Poorly or incorrectly functioning" could be many different things. Some examples: Information won't display or is garbled in its display, additions and modifications don't take or are garbled on the way in, deletions (explicit or implicit) don't take or affect the wrong items or leave dangling references, inconsistent User Interface behavior from one function to another, and User Interface organization/labeling leads to confusion or inability to locate desired functionality (making the going more difficult for editors and moderators).
How much do "we", which might be ISFDB's parents, the extended pool of ISFDB developers/code contributors, or the entire ISFDB community, care about the code's future maintainability? By "maintainability", I mean things like: Effort required for a new contributor to learn how the code works, effort required by a random contributor to understand/debug an issue, number of pieces of code that have to be modified to change an existing function's behavior, number of pieces of code/files that have to be modified to add or remove a function, consistent application of a set of design principles from one module to another, and so on.

The ISFDB is not controlling a nuclear reactor, steering a plane, processing a business' customer orders, or dealing with medical records. So #1 is mostly a combination of personal tolerance and assessment of the probability of the combination of problems + intolerant editors (who might fall into two groups: existing editors and new-arrival editors) and the potential to drive people away. If we're willing to run the risk we calculate, then an iterative, rapid-deployment approach is certainly viable. If we're not willing to run the risk, it isn't, and more formal quality assurance outside of the live environment would be needed.

#2 is more difficult to get a feel for. In my experience, the trade-off is earlier gratification today at the cost of delayed or non-gratification tomorrow. That is, I can give you new feature A or modified feature B more quickly today if I do it however it occurs to me. But my doing that is likely to mean that giving you new feature X or modified feature Y will take longer or even be impossible without starting over. Obviously, one little "A" may not prevent "X", but "A" + "B" + "C" + "D" will eventually add up to trouble.

The problem with #2 is that to ensure maintainability, you need a decent amount of non-code effort:

Specification and Design, describing the change, how it will be implemented, how it will fit into the existing system, and trade-offs. This is something coders often do not appreciate and do not like to do.
Architectural oversight, where either a person or a group is the keeper of the "vision" of how all of the system components and code are organized. Similarly, User Experience oversight, keeping the vision of how end users interact with the system.
Code reviews, where someone other than the author reads the code. Code reviews may find bugs, but their main benefits are to make sure someone else can understand the code that is written, to spread knowledge of code beyond the author, and to provide a check that the author is following whatever architectural and stylistic guidelines that have been established. --MartyD 13:53, 10 November 2013 (UTC)

I supplied code to look at, exactly to invalidate such discussions. What I provided surely is better documented and has a much cleaner interface than what I replaced. As said, I do that for a long time now. --Stoecker 14:32, 10 November 2013 (UTC)

It's easy for the imposition of these things to slow down delivery of changes, and if resources supplying the effort are constrained, you can get quite the bottleneck. I'd say we're certainly operating in a constrained-resource environment. :-)

And one final factor is the existing body of ISFDB code, with its limited documentation and various flaws and wrinkles. We have seen on several occasions well-meaning contributors make mistakes or incomplete changes due to unawareness or misunderstanding or disregard for nuances of the existing code. --MartyD 13:53, 10 November 2013 (UTC)

You have an AWFUL codebase. Really. Again - you don't need to take my word, evaluate what I did. The main reason why I never start with discussion, but I always send code first and discuss later. --Stoecker 14:32, 10 November 2013 (UTC)

So, with all of that said, what do I think? Full disclosure: I like robust systems and believe investing in non-code design and review tasks pays off very quickly. I think the ISFDB community is largely non-technical and is not a good testing resource, as people often will not understand (and perhaps will not even recognize) what's happening when things go wrong; I'd hate to see anyone get frustrated and leave because of software issues. Also, the overall moderator pool has been strained in the past, and placing more explanation and clean-up burden on the moderators does not seem like a good idea. So I do not like the thought of rapid deployment here.

I also think the existing body of code is complicated enough that some amount of architectural oversight is necessary, and Ahasuerus is the one who pretty much has to provide that. But I have experienced what I'll call the "bottleneck" myself -- contributions backed up due to Ahasuerus' bandwidth and/or priorities -- so I can understand frustration on the part of others would want to contribute. One way to do that might be to expand to a committee, but the other underlying problem is change conflicts/contention (people wanting to make different changes to the same pieces of code) and the limitations of the existing source control system to assist with such parallel efforts.

What if we change what gets worked on? I'm thinking if we went back to a list of "projects" for all of the proposed ISFDB changes, picked one, and solicited contributors willing to work on it, we could parcel out the work that way. Everyone wanting to contribute can, we get to tackle a top priority, and conflicting changes are minimized. Some oversight is provided, and the contributors can hash out what to do, covering the need for design/documentation. Results can be put up on a test server for people to look at and test. Etc. Ahasuerus can be "project leader" or could delegate to someone else. He's still the final reviewer, since he has to deploy.... --MartyD 13:53, 10 November 2013 (UTC)

I can't match what you say here and above about your experience. Parallel development is a standard nowadays. Modern source code management is designed exactly for this purpose and any good developer knows how to cope with this. Not even a small or middle sized open source project will work reliable without taking into account that others do work at the same piece of code you do. This "look the code for one guy" concept was already outdated in the 90s. ISFDB still has very old cvs, where svn would be better. There isn't even a need for something more fancy like git, because there aren't enough people involved (currently only one who really counts: Ahasuerus). --Stoecker 14:43, 10 November 2013 (UTC)

My bottom line is I think a free-for-all would be bad, so we need to be careful, but I think there's a lot of energy we could tap into that right now is stifled. --MartyD 13:53, 10 November 2013 (UTC)

Well, I had a look at some of the issues ISFDB has. Some of these I fixed as a drive-by. Ahasuerus may or may not include these changes anytime soon. I may not be the nicest guy (because I always make trouble when I don't like something instead being silent) but I'm a real good programmer and also a pretty good software designer. There are only two possibilities: Either you trust me or not. If yes, then I help, if not, then not. For me it is that easy. And to give you all something to trust I spend one week and sent the results. --Stoecker 14:32, 10 November 2013 (UTC)

(unindent) A public test server would not be my server anyway, so that testing and development would be separated. Even for JOSM we have a "tested" version released every month and a "latest" version every day. It would not be a good idea to have a leading edge work on a realtime database. What I propose is approximately that:

Continuos development on one ore more (openly available) development servers.
When the active developer considers code stable deploy code on a (openly available) testing server (bug-fixes immediately).
Test as long as for one or max two week no relevant issue was found.
Deploy that code on production.

The more people try and test, the better that concept will work. Code reviews yes, but not in this extensive and time-consuming for as done at the moment.

And yes, there will be NEW bugs. For sure. But there also will be many old bugs removed (and there are many and editors seems to be content to work around them ATM, instead of actually complaining).

So what I propose is to update ISFDB and make it fit for the future. What I request is to apply certain changes in the way the software is developed. In OpenSource usually the fact is "Who does the work decides how" - Community may decide WHAT, but not the HOW. I can help and change a lot, but not under the current conditions. I already had some conflicts with few of you about the way you treat new contributors. That again is the same issue, but now regarding the code and not the database itself. I did start that discussion with a bunch of working code and not with words only, so you have something to evaluate.

And don't think you can have both (the old way and improvements by me). What I did till now was merely a sort of getting warm. I designed new concepts and the necessary compatibility interfaces and showed how the new stuff can solve several issues. For each of these concepts it takes weeks of work to fully implement them and really improve code quality. I wont even start this work under current conditions. Ahasuerus has not the slightest chance to do the code reviews he currently does in a reasonable amount of time (I'd have no chance to do so for my own projects as well).

You have my offer, you have the conditions, you can see the advantages and also the disadvantages. It is yours (and especially Ahasuerus) decision what to do. --Stoecker 13:36, 10 November 2013 (UTC)

My background and reason for contributing is over 30 years in software support and development, ranging from JFDI ("Just F**king Do It") to military standards where I had to fill in triplicate forms and wait weeks or months for the changes to be approved. So I can see the benefits and risks to most models of development. At the moment we have a major mixture of processes: I've submitted changes that got 'improved' by Ahasuerus without comment or approval on his changes to my changes. I've tried to explain big changes in advance with screenshots and use cases - but those are rare, as I've mostly addressed the 'low-hanging fruit' changes and left the really big risky changes for later. BLongley 21:00, 11 November 2013 (UTC)

I hate the military standards development. Once did a small tool. It took me 2 hours to write it and 1 week to document and test it (additional 3 days of a colleague to write the test plan). My boss did not believe the amount of time when I told it in advance (factor 1:30 for documentation), afterwards he accepted my time planning :-) I NEVER do that in OpenSource. --Stoecker 17:49, 12 November 2013 (UTC)

It varies. One of my most satisfying projects was building a working system from the ground up in less than 3 months. I was the tech lead and we had over 20 people plus the vendor working up to 12 hours a day to get everything done ASAP since it was a high visibility project with the chain of command all the way to SecDef and Congress anxious to see what we could do. Luckily, the end users were very happy with what we had produced. And then the whole thing bogged down in Pentagon politics, red tape, etc, and the end result was unspeakably awful, but that's life :) Ahasuerus 03:24, 13 November 2013 (UTC)

Wow! I thought my achievements in software development were pretty good, but you've beaten me. I was 20 years old when I first saw my code implemented in HMS Ark Royal, at the time the flagship of the British Navy. That code is now obsolete, as is the ship itself. AFAIK, my only lasting achievement is/was an almost complete rewrite of 'PLOD' (Police Liaison Online Database) used by every Police force in the UK: I was invited down to New Scotland Yard and given a round of applause and a free lunch for my efforts. I can't be sure how much of my code is left in there, but at the time it was well-appreciated. I've probably broken a few rules by even mentioning such a project, the Security services used it too and they are mostly exempt from Freedom of Information requests even under the '30 years before we can talk about it' rules. (FWIW, MI5 don't do as good a lunch as the Metropolitan Police!) BLongley 09:32, 13 November 2013 (UTC)

For instance, I started translator support and was pretty confident of what I'd done, but bailed out when I got to pseudonymous translators. There are some changes that really need a bit of team-work, and I don't think we've EVER had that sort of collaboration. I have no problem with a mixture of development styles - we can get some quick changes in in days, but some changes really need a LOT of testing. Ahasuerus is a bit of a bottle-neck, but the lack of testing resource is another major concern. I'm sure there is a happy medium to be found, we just haven't got to it yet. But I certainly don't want to discourage improvements - we need many of them. Security issues are one of the tricky areas - as soon as you try and discuss them in public someone will abuse them, so those are best kept to a small group of developers talking offline. But we have a VERY small group of developers. BLongley 21:00, 11 November 2013 (UTC)

That's an interesting point. When I get a submission which contains bugs or a partial FR implementation, I tend to fix/rewrite the code myself rather than send it back to the developer. I figure that it's easier for me to make the requisite changes on my end than to re-review and re-test everything once I get a new version back from the developer. It also makes it easier to resolve conflicts between multiple changes that affect the same part of the code base. However, in the long run my approach may be counterproductive because it deprives developers of opportunities to learn new things and to avoid similar pitfalls in the future. Ahasuerus 23:10, 11 November 2013 (UTC)

When changing code (except for small typos, ...) you should always report the reasons back to the authors. And bigger stuff you should let the original author fix. Not a good idea for ISFDB ATM, but for JOSM we simply close tickets with patches, when the authors wont do the necessary adaptions even if the idea itself may be good, but be very helpful when they do and try to understand and learn (even it makes more work than rewriting). For us the people are more important than the code. --Stoecker 17:49, 12 November 2013 (UTC)

I am not sure I understand what you mean, but if the idea is that keeping developers happy is more important than the state of the software, then I think our priorities are different. Ahasuerus 01:39, 13 November 2013 (UTC)

I see this everywhere in ISFDB. You value perfection so high, that you kill motivation. For me perfection is the result of motivation and not the other way round. But it is impossible to convince anybody here of that concept. I failed for ISFDB data, now for the code. It was to be expected, but I had to try. --Stoecker 11:14, 13 November 2013 (UTC)

As far as translator support goes, it has occurred to me that it may be relatively easy and painless to add it by expanding the use of canonical_author (emphasis on "relatively".) But that's fodder for another discussion. Ahasuerus 23:10, 11 November 2013 (UTC)

I believe I was thinking along the same lines when I started work on it. Data capture was fairly easy, but I bailed out when I got to displaying new sections of author displays. I think Marty was doing some major changes to that area at the time. And that was so long ago that I've forgotten most of what I HAD developed and tested. In hindsight, I wish I'd submitted the data-capture changes and passed on the presentation problems to someone more expert than me. Collaboration is not one of our strong points, and we should be able to improve that. I think I once proposed a 'skills' section for the moderator list, but you don't need to be a moderator to suggest changes. There is a short list somewhere in the Wiki that lists abilities but it's long out of date and not easily found. Now that we have a major discussion going it might be time to resurrect that page and start pairing off developers and testers. I'm pretty much a monoglot but have spent a few years learning our core system - I even posted my early experiences to encourage other developers and users. I think I've established my development and design skills here - you've approved 50 or more of my changes and I've only needed to fix about 5 things where I introduced new bugs. While we're having a major discussion like this it would be nice to sort out our relative skill sets and work together better. I have a lot of time now, but access from hospital is pretty much limited to Wiki updates, not development. When I'm at home I can do some more productive work but I'm concentrating on getting up to speed on Windows Vista and Windows 7 at the moment. I really should look into Linux development as well, that seems to be long out of date. BLongley 05:55, 12 November 2013 (UTC)

I did not fully verify my concept, but my current concept would be to handle a translator as an co-author (of different type). --Stoecker 17:49, 12 November 2013 (UTC)

Yes, that's what Bill and I were referring to above. Ahasuerus 01:32, 13 November 2013 (UTC)

Seems one new type for the titles entry database and some other small changes. Most work will be to update all the UI classes, so that removal of entries also causes to remove the translation entries at every place and data is displayed right. One of the reasons this is so complicated is the missing central infrastructure to reduce side effects. I started to build these, but don't see these coming live ATM. If I did not overlook something really central here (I doubt it, because there aren't SOO many tables in the DB), I'd probably finish translators support with a 98% security of not leaving stray entries (like the Bill Drussaï one I tried to get rid of) in the database during christmas holidays. Maybe I could also make a database consistency check, which allows to detect dead ends as a debug tool. But ATM I wont start doing it. :-( --Stoecker 17:49, 12 November 2013 (UTC)

As we discussed back in February, we need to limit the number of database access points and put them in centrally controlled libraries/classes. We also need to simplify and document the software that is responsible for the Summary Bibliography and Series display pages. That logic has become so convoluted that adding more functionality to it can cause problems. Of course, there are always other, more immediate, bugs and FRs that compete for developer time. Ahasuerus 01:32, 13 November 2013 (UTC)

(unindent) It's been more than 3 days since the original post and it looks like everyone who wanted to contribute has had an opportunity to do so. In addition, I have reviewed a sufficiently large portion of the submitted code, which Dirk had sent for evaluation, to draw some conclusions about the proposed methodology.

At first I was intrigued by the idea that a "fast and furious" approach can result in "progress and update of the system and maybe a seldom bug here and there". However, after reviewing the submitted code, I see all the problems that traditional, structured development and testing techniques were originally created to address:

Changes to one part of the system that break other parts of the system -- see the "field length" discussion above
Additions of new functionality that break old functionality in a non-trivial and hard-to-detect way -- see this discussion
Minor, "on the fly", changes to different pages which make them inconsistent

This is to be expected (even from capable developers) when making hundreds of changes in an uncontrolled fashion, which is why traditional controls were invented in the first place. There are reportedly projects that do well using the "fast and furious" model, but I don't see how ISFDB could be one of them with the current developer pool.

And so, taking all of this and the feedback from other editors into account, the answer to Dirk's proposal ("You have my offer, you have the conditions, you can see the advantages and also the disadvantages") is "no". Ahasuerus 02:45, 13 November 2013 (UTC)

I said from the very beginning that I do no believe it is possible to change your opinion and I were right. Actually this is what I expected. You still expect that changes are perfect. That's impossible. What I propose is to replace lengthy review processes by testing instead. A bunch of the errors resulted exactly from what I said: "bitrotting", the longer the data is handled outside the version control, the more errors appear by updating the code to CVS changes. Code gets unusable after an amount of time. And all of the issues you found could be found by simply using the interface in test. Till now nobody did such tests - neither I nor you. I not, because for me it is a development test and after the first reaction of you I did not start the tests, because for me they look like wasted time. And you did not test the full system, because you want to break it in pieces.

Till now you did not even made a single try for the stuff the I really cared about, but only about the by-products I declared non-important for me and only partially implemented (and thus with higher error probability). Rather than explicitly caring for the small changes I would remove them in favor of the others. You do exactly opposite. --Stoecker 11:14, 13 November 2013 (UTC)

Having said that, I do see value in implementing many of the changes that Dirk has been working on. The security fixes and improved Unicode support look promising and a number of other changes move the software in the right direction -- and I sincerely appreciate Dirk taking the time to work on them. Ahasuerus 02:45, 13 November 2013 (UTC)

As said: No. I don't want to wait months for simple changes to be approved. That simply makes no fun at all. What I would do in the next half year would exceed the current amount a lot and I don't want to have the frustration on my side which would be involved with each of these changes. One time frustration is enough. --Stoecker 11:14, 13 November 2013 (UTC)

Understood; I was referring to the time you have already spent on it. Again, thank you for contributing dozens of hours of your time to the project and starting this discussion, which has been quite useful! Ahasuerus 17:03, 13 November 2013 (UTC)

Technical Documentation

(moving this discussion to a separate section since the original one is so big now)

In addition, as Marty and Bill mentioned earlier, the development process is slower than it has to be even given the safeguards that we have in place. In part it's due to the fact that the number of active developer has been low over the last few years: Bill has been hanging out at hospitals, Kevin hasn't been seen in a long time, Marty is busy at work and I spend much of my time on Fixer. However, the current development model, which makes me the main bottleneck, is also an issue. Perhaps we could spread the load as suggested by Bill: if Developer A makes a change, it can be reviewed by Developer B instead of by me.

I also like many of Marty's ideas and I think I should start by improving our technical documentation. If I can describe the common pitfalls/dependencies, the direction that we plan to take in various areas and so on, it would kill three birds with one stone:

make it easier for developers to understand how their changes can affect other parts of the software
make it easier for developers to pick bugs/FRs for fixing/implementation without worrying that your changes may have to be redone later
make transition easier in case I become incapacitated or otherwise unavailable. At some point I may need to take a break for a while -- burnout is always something to keep in mind, as Al's experience shows.

Ahasuerus 02:45, 13 November 2013 (UTC)

Well, plain text documentation is manageable from hospital. Give me a few clues as to what we need most and I'll be happy to help with that. Pictures/screen-shots are beyond the usual facilities of hospital IT systems, and coding is right out. I'm already thinking that a map of which modules link to others would be a fairly simple (albeit long-winded) useful start. And our documentation of what each table and field does is well out of date. I know I'm going back to hospital again soon for a six-hour fluid drain and I can save myself from carrying two books in to cover the time and do something PRODUCTIVE instead. BLongley 06:21, 13 November 2013 (UTC)

If you could update the Wiki pages that describe table layout, that would be a really good start. I plan to start by expanding the "hints" section of the Development page to describe the major dependencies that we have. (And we may want to move this discussion some place else since it won't make much sense to most editors.) Ahasuerus 03:21, 14 November 2013 (UTC)

Another good point there - we need "some place else" to leave non-coders less confused but where coders and designers can collaborate. My first attempts at teaching people ISFDB data use is at http://www.isfdb.org/wiki/index.php/Talk:Database_Schema and really could be more usefully placed. I'll break in my latest PCs with some simple work to start with, and call it "Windows Vista and Windows 7 training" on my CV. :-) BLongley 05:25, 14 November 2013 (UTC)

Patch r2013-154 - New/Add/Edit/Clone Pub/Title field length changes

As per the discussion above, the length of the data entry fields in New Pub, Add Pub, Edit Pub, Clone Pub, Import/Export Contents, Edit Title, Make Variant and Add Variant has been changed. If you find anything unusual, please post your findings here. Ahasuerus 04:16, 10 November 2013 (UTC)

Just had a quick look and all the fields are now half there old size. I'm using Windows IE10 if this makes any difference.Kraang 04:33, 10 November 2013 (UTC)

Hm, that's peculiar since everything looks OK when I access New Pub/Edit Pub/etc using IE 10. Could you please hit Ctrl-F5 (which reloads the page and all associated images) to see if it helps? Ahasuerus 04:50, 10 November 2013 (UTC)

One submission had all fields half the size, except for the Note field, which looked normal. Now they all have the longer width. I don't know what I did to make the correction. Mhhutchins 05:01, 10 November 2013 (UTC)

The only thing that comes to mind is that browsers typically keep a copy of certain files -- including images and certain files which control how the data is displayed -- on your hard drive. That way they don't have to reload them every time you navigate from page to page. If the browser doesn't realize that one of the files with the formatting instructions has changed, it will try using the old version of the file, which can cause unexpected behavior. Ctrl-F5 usually helps resolve the problem since it forces the browser to reload everything. Apparently, Mac does it differently, but then again, what else is new? :-) Ahasuerus 05:13, 10 November 2013 (UTC)

The change removed the width from input field and moved that into CSS. If your browser had a cached version of the CSS (which is likely, as CSS changes seldom and gets cached), then the new width in CSS was not active and the old width in the HTML was removed, so the visible field was reduced to the browser minimum. Reloading fixed that. These effects are to be expected when the separate style files of modern webpages are modified, but all such effects are temporary and only affect the display and never the functionality. --Stoecker 13:41, 10 November 2013 (UTC)

The fields look fine now, so all is well.Kraang 15:43, 10 November 2013 (UTC)

Dates Entered as Text

I've just noticed that maybe half of the DOB and DOD dates in the ISFDB are entered as text, rather than numbers. This may also extend to date of publication, though I've not checked. This causes problems in Excel spreadsheets and none of the standard tricks for converting text to numbers seems to work. Any chance of a fix?--Rkihara 22:31, 10 November 2013 (UTC)

Are you referring to the extract that I created a few weeks ago? All ISFDB dates use the same data type, but perhaps the extraction process used a format that Excel didn't like. Could you please provide an example? Ahasuerus 23:38, 10 November 2013 (UTC)

The extracted data was where I first noticed it, as text is left aligned and numbers are right aligned. I copied some data from the author directory as a test and pasting it into Excel gave the same result. The dates which are numbers can be manipulated and formatted as expected in both sets of data, while the dates which are text cannot. The DOB for Isaac Babel for example is text, while his DOD is numeric. I'm using MS Office 2011 for the Mac.--Rkihara 02:35, 11 November 2013 (UTC)

All data displayed by the Author Directory page (and other ISFDB pages) is just HTML-formatted text. Here is what the actual HTML for Babel looks like on that page:

<a href="http://www.isfdb.org/cgi-bin/ea.cgi?20125">Isaac Babel</a> - Babel, Isaak Emmanuilovich Moldavanka, Odessa, Ukraine, Russian Empire 1894-07-13 1940-01-27

When I copied-and-pasted these dates into a Google Drive spreadsheet, GD correctly recognized them as dates and formatted them accordingly. However, when I pasted the same dates into MS Excel, only the DOD was recognized as a date. Further experimenting suggested that MS Excel couldn't handle dates prior to 1900 and a Web search confirmed that "Excel stores recent dates as a date serial number ... Unfortunately, Excel's serial number begins on January 1, 1900; and negative serial numbers aren't recognized."

There are some Excel tricks that you can apparently use to work around the problem, but perhaps Google Drive or another tool may be a better match for what you are trying to do. Ahasuerus 03:23, 11 November 2013 (UTC)

Thanks, it never occurred to me that it was an Excel problem. I do have dates as text running from 1900 into the 21st century, but it's probably another peculiarity of Excel. I'm cleaning up the data in Excel before stuffing it into Bento, which I hope will handle the dates correctly.--Rkihara 07:45, 11 November 2013 (UTC)

FREE fanzines for postage costs alone

I posted this on my user page but suspect that here might get a bit more attention. I'm downsizing again and really can't store 10 boxes of fanzines much longer - I haven't checked them for five years and think they'd be better used elsewhere - preferably going to someone willing to catalogue them here, but if you just want to READ them that's fine with me too. They cost me nothing so that's what I'm charging: ZERO pounds, dollars, Euros. Just cover the postage and remember I'm in England before you claim first dibs on anything. Pete Young has claimed a few already but he hasn't asked for even half a box-worth, so plenty left to go! BLongley 10:56, 12 November 2013 (UTC)

I'm just out of hospital (a different one this time, I was at Novacon) so I may take some time to respond. But I WILL get rid of them somehow - I managed to sell a few items back to the dealer that sold them in the first place, but was surprised that people didn't snap up the BSFA publications. The list of items, long out of date, is still listed on my user page, I don't have the energy to update it. Limited time offer - my life expectancy is 12-18 months so do claim things ASAP, it might take me weeks to find stuff among the other 50 boxes! BLongley 10:56, 12 November 2013 (UTC)

Bill, is your personal email address still at the ntlworld.com domain? I want to see you a private email. Thanks. Mhhutchins 17:46, 12 November 2013 (UTC)

Yes, I've managed to retain that e-dress despite NTL having been bought out by Virgin media. BLongley 02:12, 13 November 2013 (UTC)

Bill, please send me email at (my last name) @ beloit.edu. I'd also like to have an offline conversation about this. Chavey 15:36, 13 November 2013 (UTC)

Linking the British Library record number

In the past, any links to records on the British Library website expired at the end of a user's session. I think I've discovered a fix to the problem.

This link searches for number 1234567890:

http://catalogue.bl.uk/primo_library/libweb/action/search.do?&vl(freeText0)=1234567890&fn=search

Just replace "1234567890" with the BLIC system number and it links properly. For example in this record, I entered "011983806".

http://catalogue.bl.uk/primo_library/libweb/action/search.do?&vl(freeText0)=011983806&fn=search

I've tested this over a period of a week or so and it has remained stable, so the search can't be timed to the session. If anyone is adding BLIC links or updating a record that contains a bad link, please consider using this URL format. If you discover a problem with this format, please let me know here. Thanks. Mhhutchins 18:28, 12 November 2013 (UTC)

Edit change made to Template:PublicationFields:Price

I needed to know the date of the change from old British pence to new pence, i.e. the date of decimalization, and it wasn't in our "Price" documentation. Hence I made that small addition to that template. (See the history there if you want to see the precise change.) Chavey 04:24, 15 November 2013 (UTC)

Look fine to me. Mhhutchins 04:58, 15 November 2013 (UTC)

Server problems - 2013-11-16

FYI, the server has been unstable for the last hour or so. I have tried a few things, but it is still slow and/or erratic. Ahasuerus 19:22, 16 November 2013 (UTC)

Robert Reginald

I've just heard through a friend that Robert Reginald, the author of some of our common references and an editor here, has died. Locus doesn't have it yet, but there is a short obituary on SF Site. --Ron ~ Rtrace^Talk 00:50, 22 November 2013 (UTC)

Ouch! As I recall, he had fairly serious health issues some years ago and tried to take it easy after he recovered. He still did a ton of work at Borgo Press, though, and even found the time to contribute here. He will be missed :( Ahasuerus 00:57, 22 November 2013 (UTC)

Patch r2013-158 - Ability to import individual titles

The "Import Content" page has been enhanced to allow importing individual titles into the selected publication. If you press Cntr-F5, you will also see the new version of the "divider" line. Please let me know if it's too thick and I will adjust it accordingly. Oh, and we'll need to update Help once the new functionality is stable. Ahasuerus 05:12, 22 November 2013 (UTC)

Exceeding cover image file upload standards

To all editors: please do not upload cover files which are larger than 150 kb in size unless there is an exceptional reason. This is the ISFDB standard, and you should be getting a warning that the file exceeds the standard. If you're uploading just the front cover of a book, there is NO reason why it should exceed 150 kb. I have resized hundreds of files uploaded by other editors which have exceeded the standards (both in file size and image dimensions), and there is NO reason why the uploader shouldn't be doing this before uploading the file to the server. Most editors are following the rules, especially new editors once they've been informed of the standards. Why other editors continue to do this is quite baffling. There are many free software programs which can resize files with barely any loss of quality. If there are editors who believe that the standards should be changed, please start a topic on the Rules & Standards discussion page. Or explain here why the standards shouldn't apply to their uploads. Mhhutchins 18:13, 30 November 2013 (UTC)

Tuck's Reference book available at auction

Since it's always nice to have more folks around with the primary reference sources we use, I'll mention that Heritage Auction House has a full set of Tuck's "The Encyclopedia of Science Fiction and Fantasy" (1974) that will be auctioned next Thursday. (I already have a set, so I won't be bidding against you :-). They also have an auction for Tuck's 1st & 2nd edition (1954 & 1959), but that's more of interest to collectors, since from the data viewpoint they are less reliable than the final edition. Chavey 06:49, 9 December 2013 (UTC)

All three volumes can be purchased starting at $60/set from Abebooks.com dealers. Just in case the bidding goes over that amount, don't overbid. Mhhutchins 04:58, 10 December 2013 (UTC)

The in-print Advent books, including these, can still be bought through NESFA Press. They sell the set for $95 if folks would prefer a new copy. --Ron ~ Rtrace^Talk 13:53, 10 December 2013 (UTC)

An even better deal! Mhhutchins 17:12, 10 December 2013 (UTC)

Patch r2013-159 -- Our first graphs!

Now that all major browsers support the "SVG" standard for presenting data graphically, we can start adding graphs to ISFDB. The first small step in this direction was taken earlier today when the old Wiki-based "Stats" graphs from 2012 were replaced with new dynamically generated graphs.

Since the SVG technology is still new-ish and different browsers handle it somewhat differently, I couldn't test every browser/version permutation. I would appreciate it if multiple editors could take a look at one or more of the six "by Year" graphs and let me know if they find anything unusual. If you do, please post your browser version and a description of your findings here. TIA! Ahasuerus 04:39, 10 December 2013 (UTC)

Patches r2013-160 and r2013-161 -- nightly processing redone

The way nightly processing works has been redone. In the past the only thing that this process did was rotate the ISFDB banner, i.e. the composite picture which appears at the top of each page. The way the software that handles nightly processing was set up was awkward and it was difficult to add more features to it. The new process is much more robust and we should be able to turn a number of computationally intensive processes into nightly jobs. For example, the "Summary Database Statistics" page and various "Top Contributor" pages are obvious candidates because they do not need to be regenerated more often than once a day. This should not only make these pages load almost instantly, but it should also make other pages load faster since a slow-loading page affects all users.

For now, "Titles and Publications by Year" and "Title Distribution by Author Age" (available from the ISFDB Statistics and Top Lists page) have been modified to take advantage of improved nightly processing. Ahasuerus 05:28, 16 December 2013 (UTC)

P.S. Forgot to mention that "Title Distribution by Author Age" used to be a static Wiki page last updated in 2010, but now it's a part of ISFDB proper and the data is regenerated every night. It's gratifying to see that the 2013 version has almost three times the number of data points of the 2010 version, in part because we have identified the DOBs of many more authors. Ahasuerus 05:32, 16 December 2013 (UTC)

Patches r2013-162 and r2013-163 - more nightly processing changes

The ISFDB Statistics page has been changed to work with the new nightly processing task. When you access it, the results should be displayed almost instantly.

The main ISFDB Statistics and Top Lists page has been spruced up a bit. I also added links to the 4 "Authors by Age" pages which used to be accessible from the Wiki side. Ahasuerus 22:57, 16 December 2013 (UTC)

Patch r2013-164 - Top Contributors lists moved to nightly processing

All "Top Contributors" lists have been changed so that they are now regenerated every night. In addition, the way graphs are displayed has been changed to eliminate unnecessary scrollbars when using IE and Google Chrome. Ahasuerus 03:32, 17 December 2013 (UTC)

Patch r2013-165 - nightly processing cleanup

Nightly processing has been further cleaned up. Going forward, all pages accessed from the "ISFDB Statistics and Top Lists" page should load near instantaneously. Ahasuerus 06:00, 17 December 2013 (UTC)

Patch 2013-166 - Most-Reviewed Titles page added

The ISFDB Statistics and Top Lists page has been updated to allow navigation to the Most-Reviewed Titles page. The latter lets users generate "most-reviewed titles" lists of various kinds: by decade, by year, "prior to 1900" and "most-reviewed of all time".

Typically it takes anywhere between 1 and 2 seconds to generate a list because the software needs to find and combine reviews of variant titles and translations. Ahasuerus 21:30, 18 December 2013 (UTC)

Patches r2013-167 and r2013168 - Magazine issue grid fixes

Bug 377, which caused magazine issues to be left-justified instead of centered when there were two or more issues per cell, has been fixed. Ahasuerus 22:57, 18 December 2013 (UTC)

P.S. As an added bonus, 0000 years appear as "unknown" now. Ahasuerus 23:16, 18 December 2013 (UTC)

Patch r2013-169 - Pop-up validation of author dates

Pop-validation of author birth/death dates has been fixed to once again allow 0000-MM-00, 0000-00-DD and 0000-MM-DD dates. Ahasuerus 00:59, 19 December 2013 (UTC)

Patch r2013-170 - Author Edit bug fixed

Bug 373, "Entering all spaces for Web Pages and emails causes an error in Edit Author", has been fixed. There were other issues with entering a mix of spaces and semicolons that have been fixed as well. Ahasuerus 17:47, 19 December 2013 (UTC)

Patches r2013-171 and r2013-172 - Most-Viewed authors/novels/short fiction now generated within ISFDB

The ISFDB Statistics and Top Lists page has been updated to link to three new pages: Most-Viewed Authors, Most-Viewed Novels and Most-Viewed Short Fiction. Ahasuerus 05:34, 20 December 2013 (UTC)

P.S. All author lists have been converted to HTML tables. Ahasuerus 18:15, 20 December 2013 (UTC)

Are these "most-viewed" pages based on individual IP address visits? Mhhutchins 22:35, 20 December 2013 (UTC)

I am afraid not. We don't capture our visitors' IP addresses except in transient Web server logs which are not a part of the database. Every time an author page or a title page is viewed, we update an author/title-specific counter, but that's the extent of what we do. Ahasuerus 23:01, 20 December 2013 (UTC)

I wouldn't want the same thing to happen like this list where a Roger Zelazny fan was able to get 10 of his works into the Top 30. I know he's an excellent writer, but having that many titles so high in the list skews it so badly that it's almost unusable. Mhhutchins 22:35, 20 December 2013 (UTC)

Understandable, but I am not sure what we could do about it. Also, keep in mind that all I did in this case was make Al's old Wiki-based lists available on the database side and changed the logic to generate them directly from the current database data rather than as a once-a-year snapshot. Now that these (and other) lists are more readily available, I hope to see additional feedback, which will enable us to make them more useful. Ahasuerus 23:01, 20 December 2013 (UTC)

Changing the binding field to a drop-down list

I am finally ready to start working on changing the binding/format field to a drop-down list. I will add all of the formats that are currently listed, but we also have to decide what we want to do with the following non-standard bindings:

CD-ROM -- 23 pubs
Half Foolscap - 7 pubs
audio DVD - 1 pub
broadside - 6 pubs
codex - 22pubs
email - about 40 pubs
portfolio - 12 pubs

Our choices are:

Do what Help currently says, i.e. "leave the field blank, and describe the publication's format in the note field". It should be fairly easy to modify a hundred pubs.
Add one or more of the "uncommon" bindings to the drop-down list (perhaps CD-ROM and email?)
Add "other" as a choice to the drop-down list, which would make it easier to search for uncommon bindings, and change the Help text accordingly.

What say you? Ahasuerus 21:33, 20 December 2013 (UTC)

I think that adding "Other" as a choice and having the editor describe the publication in the Note field would be the better choice here. There are too many records which are blank for various reasons for choice #1 to be effective. That would still require the 100 or so pubs be edited to comply with the standard. Hopefully this would also allow an intrepid editor (read: me) to search easily to find records which were (unintentionally or not) left blank. Mhhutchins 22:27, 20 December 2013 (UTC)

Well, we could add a cleanup script to display pubs without binding codes, but we have almost 10,000 of them, which may be unwieldy. Perhaps it would be more useful if we were to add further breakdown by year, e.g.:

+---------------------------+
|           1950 |       40 |
|           1951 |       36 |
|           1952 |       53 |
|           1953 |       49 |
|           1954 |       60 |
|           1955 |       64 |
|           1956 |       56 |
|           1957 |       40 |
|           1958 |       71 |
|           1959 |       71 |
|           1960 |       67 |
|           1961 |       76 |
|           1962 |       60 |
|           1963 |       66 |
|           1964 |       63 |
|           1965 |       68 |
|           1966 |       66 |
|           1967 |       71 |
|           1968 |       79 |
|           1969 |       88 |
|           1970 |       66 |
|           1971 |       57 |
|           1972 |       62 |
|           1973 |       62 |
|           1974 |       67 |
|           1975 |       70 |
|           1976 |       58 |
|           1977 |       55 |
|           1978 |       50 |
|           1979 |       76 |
|           1980 |       69 |
|           1981 |       52 |
|           1982 |       94 |
|           1983 |       69 |
|           1984 |       69 |
|           1985 |       82 |
|           1986 |       83 |
|           1987 |       78 |
|           1988 |       73 |
|           1989 |      100 |
|           1990 |      126 |
|           1991 |      117 |
|           1992 |      119 |
|           1993 |      101 |
|           1994 |       80 |
|           1995 |       88 |
|           1996 |      123 |
|           1997 |      120 |
|           1998 |      142 |
|           1999 |      112 |
|           2000 |      104 |
|           2001 |      101 |
|           2002 |       90 |
|           2003 |      108 |
|           2004 |      103 |
|           2005 |      104 |
|           2006 |       98 |
|           2007 |      150 |
|           2008 |      187 |
|           2009 |      173 |
|           2010 |      160 |
|           2011 |      118 |
|           2012 |       48 |
|           2013 |       13 |
|           8888 |       46 |
|           9999 |        1 |
+----------------+----------+

And then each value in the "count" column could be linked to a separate page where a full list of biding-less pubs for that year would be displayed. Ahasuerus 00:07, 21 December 2013 (UTC)

That would work as well. It should be fairly easy to research those in the past few decades just through Amazon. And some of the earlier ones can be found through Tuck and Reginald. Mhhutchins 03:45, 21 December 2013 (UTC)

It's been my experience that many recent pubs without binding codes are either library editions/bindings or large print editions. Much of the time, there is not a whole lot of data about them on the internet and when it's available, sometimes it refers to the trade editions, further muddying the waters. OTOH, I am sure there will be some low hanging fruit to exploit. Ahasuerus 04:27, 21 December 2013 (UTC)

(unindent) So, any objections to adding "other" to the list and moving all non-standard bindings to Notes? (No hurry, I won't be able to get to it for at least a few days since I need to finish certain cleanup tasks first.) Ahasuerus 18:44, 26 December 2013 (UTC)

Non-objection: I like the "Other" idea. What do you think about also "Unknown", which could be used to handle the missing ones as well as newly entered pubs where the information isn't known (so editing one with missing info would set it explicitly to Unknown, a la the language handling + English)? Moderator review screen could highlight "Unknown" in yellow to encourage questioning it. --MartyD 00:35, 27 December 2013 (UTC)

Hm, that's a very good point. "Unknown" is much easier to search for (and implement) than a blank value. I like it! Ahasuerus 04:06, 27 December 2013 (UTC)

Especially with older books, our only information on the books may be from WorldCat, which doesn't give bindings. In these cases "Unknown" is about all we can do easily. It's possible to scour Abebooks and various auction houses to find this info, but that's a lot of work. Although a search for "Unknown" would at least tell us which books need such attention. ("Someday ...."). Chavey 15:49, 27 December 2013 (UTC)

Patch r2013-174 - New Pub bug fixed

Leaving two or more Contents groups blank in New Pub no longer results in the rest of that Contents section not being submitted. In the past, if you entered titles 1, 2 and 5 in the Contents section (leaving 3 and 4 blank), only titles 1 and 2 would have been submitted. This problem affected Titles, Reviews and Interviews. Ahasuerus 00:39, 21 December 2013 (UTC)

Patch r2013-175 - Added pop-up validation to Add Pub and URL validation to all Pubs-related pages

Add Pub has been modified to display an error message if an invalid publication date is entered. In addition, New Pub, Add Pub, Edit Pub and Clone Pub have been modified to prevent editors from entering invalid URLs. I also found and fixed the 6 pubs records which had invalid URLs on file. Author Edit, Title Edit, Publisher Edit and Pub Series Edit still need to be addressed. Ahasuerus 04:32, 21 December 2013 (UTC)

I remember an editor a few years ago created a script to find invalid URLs, and I even helped him clear up many of the records. I am unable to find the project page in a search. Does anyone recall this project or know how to find it here on the wiki? If not, is there a way to write a new script to find these bad URLs? Thanks. Mhhutchins 05:33, 21 December 2013 (UTC)

That was me. I still run a script every couple of weeks that checks the database dump for invalid URLs and for broken Wikimedia links (deleted pages, etc.). Those I fix myself as they are a handful at a time. The more general checking for broken links I put on hold due to the high false positive rate I was getting (improperly configured servers, transient errors, servers which block scripts, etc.) plus the amount of time it takes to check 50k+ URLs. It's still on my list to revist. -- JLaTondre (talk) 13:20, 21 December 2013 (UTC)

Recent flurry of patches

Some of you may have noticed that a lot of patches have been installed over the last couple of days, but no patch notes have been posted. The reason is that the changes have been largely internal and invisible to our users, so there is very little to report. A few error messages have been tweaked and a couple of error conditions have been added, e.g. you can no longer submit a Title Unmerge submission without selecting any pubs to unmerge, but that's about it. There should be more small patches over the next few days as I work my way through the list of what needs to be changed.

Of course, if you notice any unexpected changes or bugs, please report them here. TIA! Ahasuerus 02:32, 24 December 2013 (UTC)

Traffic Stats?

I would like to suggest adding the unique and total daily/monthly visitor stats to the ISFDB stats (excluding moderators/editors/robots which would skew the results).--Rkihara 19:42, 25 December 2013 (UTC)

I think it would be desirable, but it could also present certain challenges. Of the top of my head:

Excluding self-identified bots is easy because they are clearly marked as such in our Web server logs. However, not all bots admit that they are bots and there is no easy way of identifying them short of analyzing their behavior over a period of time, not a trivial proposition.
Excluding editors may not be desirable even if it was easy to do. A person who has edited the database twice but visits the site at least every week is more of a "user" than an "editor" from the usage perspective.
At this time we have two sources of visitor information: Web server logs, which record visitors' IPs and the page(s) that they visited, and author- and title-specific counters which are not IP- or user-specific. There is no easy way of reconciling the two short of changing the database design. Moreover, correlating user names and IP addresses raises various privacy issues, especially if you consider the fact that our backups are publicly available. I guess we could purge this data when creating the public backup files just like we currently do with other user-specific information.

With all of this in mind, let's review our "low hanging fruit" options. First of all, there are companies like Alexa, Quantcast, and Compete.com, which slice and dice traffic stats for a living. For example, Alexa estimates that ISFDB's traffic makes us number 252,237 worldwide, 150,764 in the US and 67,295 in the UK. By way of comparison, Fantasticfiction.co.uk is estimated as number 53,468 worldwide, 33,598 in the US and 20,458 in the UK. Of the other major "competitors", Noosfere is number 272,374 worldwide, Fantascienza is 292,573 and Fantlab.ru is 34,463, which is impressive for a site with no English interface.

Now, keep in mind that these are estimates and we would have to install these companies' specialized software to get more accurate readings. We could also use Google Analytics (the basic version is free) to keep track of visitors, but that can raise various performance and privacy issues. We'd definitely need to review the implications before we go that route.

Alternatively, we could use a relatively simple log parsing program to parse our existing Web server logs. It won't be as fancy as some other options, but it's relatively benign and we will have full control over what it does. I'll just need to remember to run it monthly. Ahasuerus 18:42, 26 December 2013 (UTC)

Maybe just a count of daily/weekly unique visitors rather than a count of total visits would work better?--Rkihara 19:11, 27 December 2013 (UTC)

Monster Porn

Well, "Boffing Bigfoot" is a form of speculative fiction. So if anyone thinks we should be including such titles, here's an introduction to the genre from Business Insider. Chavey 10:57, 28 December 2013 (UTC)

There's too much non-porn sf to enter into the database to waste time creating records for these works. No one's going to stop an editor from entering such, but will probably question their priorities (or at least their reading preferences!) Mhhutchins 17:57, 28 December 2013 (UTC)

I agree completely. And I should have added a smiley face to my post. But I must admit that I find the genre humorous. (You should check out the covers in that article for a good laugh :-) ). Chavey 19:43, 28 December 2013 (UTC)

Not to my taste either, but I think we should lean towards inclusiveness, as more than a few SF writers such as Harlan Ellison, and Robert Silverberg, have written porn novels (genre and non-genre). Some of which are included in the ISFDB.--Rkihara 21:38, 28 December 2013 (UTC)

And Brian Aldiss, and Marion Zimmer Bradley. But all of these people had their porn published as actual books. The "Monster Porn" hit the bottom of our priority list, IMHO, because they have 3 strikes against them: They haven't written other SF (yet); they're in a "less interesting" sub-genre (for most of us); and they're only publishing in ebook format. Drop any of those 3 issues, and they might rise to the level of inclusion. For example, if one of those authors then started publishing mainstream spec fic, we might want to go back and add their monster porn. Chavey 01:34, 29 December 2013 (UTC)

I think the main problem is the sheer volume of self-published SF, erotic or otherwise. Every month Fixer finds roughly 4,000-5,000 new ISBNs and probably twice that number of ASINs -- I don't count them on a monthly basis, but Fixer's database contains 613,512 ASINs vs. 356,615 ISBNs. Until we have enough moderator bandwidth to handle that kind of volume of self-published stuff, Fixer's low priority queues will sleep undisturbed.

Having said that, Fixer does need to improve his handling of e-books published by major publishers. The fact that "Ilona Anrews", a "#1 New York Times Bestselling author", as their Web site proudly proclaims, has e-books that have never appeared on paper, suggests that Fixer needs to stay on top of it. Ahasuerus 01:59, 29 December 2013 (UTC)

(unindent) Sorry, but I can't resist this aside, since now it's running on endless loop through my head:

Stories of tortures

Used by debauchers,

Lurid, licentious, and vile,

Make me smile.

Novels that pander

To my taste for candor

Give me a pleasure sublime.

Let's face it, I love slime.

All books can be indecent books,

Though recent books are bolder,

For filth, I'm glad to say, is in

the mind of the beholder.

When correctly viewed,

Everything is lewd.

I could tell you things about Peter Pan,

And the Wizard of Oz, there's a dirty old man!

From Tom Lehrer's "Smut" (That Was the Year That Was, Reprise Records, 1965). A video of his 1967 performance of it in Copenhagen is here. --MartyD 11:53, 29 December 2013 (UTC)

One of his greats. As a Math professor, I am of course a great fan of Tom Lehrer's. My step-mother was a friend of his at Harvard, and gifted me with the "self-published" 1st editions of his first two albums that she had bought from him. And as a ballroom dance teacher at a college, I've introduced students to "Masochism Tango", and (at their request) we use it fairly regularly at our monthly dances. Chavey 18:51, 29 December 2013 (UTC)

Patch r2013-214 warning

I am about to install a big patch which will touch a few dozen different places in the software. Hopefully, nothing will get broken, but if you see anything unusual, please report your findings here. Ahasuerus 22:12, 29 December 2013 (UTC)

Well, it seems that now only one publishing series is displayed per publisher, see this example. Stonecreek 01:48, 30 December 2013 (UTC)

Excellent catch! It was indeed a side effect of the big change -- fixed now. Thanks! Ahasuerus 03:26, 30 December 2013 (UTC)

Award page update

I have updated the Award page that is linked to from main sidebar so that instead of keeping links to actual award winner year pages (which required manually updating the wiki page), it instead links directly to our Bibliographic award pages (such as this one for the British Science Fiction awards). This reduces the length of the Awards index page noticeably, makes it easier to use an index to these pages, and improves the timeliness of the data linked to on that page. (For comparison, here is the previous version of this page.)" Chavey 09:00, 31 December 2013 (UTC)

I would suggest replacing "Our [award] page" with "The ISFDB [award] page", and moving the link to the top of the list, instead of its relatively low placement. Mhhutchins 17:18, 2 January 2014 (UTC)

I was thinking of that link as the "conclusion" of the paragraph, but I think you're right. I'll do that. Chavey 18:05, 2 January 2014 (UTC)

Patch r2013-236 - Edit Pub

The software behind Edit Pub has been made more secure. There should be no user-experienced changes, but if you see anything unusual, please report it here. Ahasuerus 22:22, 31 December 2013 (UTC)