User talk:Alvonruff/Archive01

Tk?

Al,

Is it a safe bet that everything at http://www.isfdb.org/cgi-bin/ea.cgi?Tk is a "graphic novel" of some sort? Some of them state so outright while others are not so clear. Ahasuerus 15:36, 1 May 2006 (CDT)

Yeah. Tk is bubbling up to the top of my list of "authors" to address (from my list of popular authors without a Wikipedia article). Tk is really the puplisher.

Backups

Sheesh, these marathon design discussions make me feel like we are trapped in Ken Grimwood's _Replay_ and it's 1995 all over again! Which isn't necessarily a bad thing :-) And, um, looking at the date of the backup files posted here, I am afraid to ask when the last real backup was performed?..Ahasuerus 22:00, 1 May 2006 (CDT)

Backups happen nightly. However, that isn't the version that's put online for download. I bring the backup over to the home unit, and remove the wiki tables (need to remove passwords and email addresses). Then a new backup is made from that version and uploaded. Real backups are still around at TAMU and my house. There are those people who build things other than an ISFDB from the database backup, so I'm holding back on putting up a new version to reduce the amount of work they need to do as I redesign tables. Alvonruff 04:39, 2 May 2006 (CDT)

Password/email data removal is certainly very important, but I hope it could be automated in the foreseeable future to eliminate the human factor? As for the redesign issue, do you think we could have two copies of the backups made publicly available, "old but stable" as well as "current but unstable"? My real concern is data preservation in case of (a) catastrophic failure and/or (b) backup software/human error. I have seen businesses where the operations department thought that their backups were fine, but they either never tried restoring and testing them, or they did it so rarely that they didn't catch the point where the backups stopped doing their job -- with predictably disastrous results. Ahasuerus 09:53, 2 May 2006 (CDT)

Okay, here's the backup situation in detail:

1. TAMU does a MySQL backup every night. This backup goes into a standard location. TAMU sysadmins and myself have access to this backup. Access is not available via web browser, and requires a WebDav account and password. Someday I'll share that with other trusted users.
2. Everynight, an automated chron job retrieves that backup and transfers it to my home Linux system.
3. The script loads the backup into my home MySQL system, deletes sensitive tables, and then makes a second sanitized backup. I now have two backups on my system: the original prestine backup from TAMU, and the sanitized version.
4. I then reload the original pristine version into my system, so I have a working copy of the pristine backup.
5. I take automated nightly backups of my database as well, and keep 7 days worth online.

The missing elements are:

1. I don't automatically upload the sanitized version to make it available for download yet. Doing this nightly will be... nice, but it won't prevent a large amount of data loss in case of catastrophic error.
2. The pristine version is not available to anyone other than TAMU sysadmins and myself.
3. I haven't made physical backups of the data and stored them offsite (well, not in a while).

Currently a catastophic failure would require three events in tandem:

1. Loss of physical backups from TAMU (a possibility as you state, although the ISFDB backup file available online is derived from this TAMU backup), and
2. Loss of the disk drive at TAMU where ISFDB data and backup are stored, and
3. Loss of the home system

I have seen systems with triple redundancy go down even though it was "impossible" for them to fail, so yes, I am paranoid and would like to see at least 3+ copies stored in various remote locations -- eventually. For now, an automated nightly backup that would copy the zipped database file to a remote location should be fairly easy to do (one would hope) since we are talking about <100Mb. Once the file is there, it can be backed up to CD/DVD/flash/etc. Ahasuerus 14:09, 2 May 2006 (CDT)

I think the real risk is that trusted elements of the ISFDB community don't have access to the complete set of data. Without all of the data, some aspects of the ISFDB will be nonsense (like change history). To rectify that, we would need to create a third host, available to specified parties where a complete copy of the database would be stored nightly. Alvonruff 13:00, 2 May 2006 (CDT)

I assume that these trusted parties need change history data to be able to do incremental uploads-cum-data mining of ISFDB data into their systems? If the master database file(s) are released to these trusted parties, will the password data be strongly encrypted? After all, trusted parties may have families, friends, unsecured networks, etc etc. Ahasuerus 14:09, 2 May 2006 (CDT)

Lessons learned so far

I have attempted high level cleanup of a few author's data just to see what kinds of problems a typical editor may run into. Here are my thoughts so far:

1. ISFDB's submission/editing tools are limited but reasonably stable for a beta system. They let you recover from snafus fairly gracefully. I am yet to experience major data loss or degradation after a modest amount of system abuse.
2. On the other hand, the editing process is quite time consuming. A drag and drop interface might speed things up substantially, but I doubt we can do it with an all volunteer crew. For now, making Moderators' submissions auto-approved might help things along. A form that would allow you to delete a Work and its associated Publication (only one for now, auto-deleting multiple Publications with one click may be more trouble than it's worth) would be handy.
3. Much of the publication and work level data that we have is quite dirty, in part because much of it comes from Amazon and related places and their data is, well, dirty. It will take a long long time to get all of the data verified, probably man-years.
4. Some form of data verification matrix as described elsewhere will be a must. Ahasuerus 20:25, 2 May 2006 (CDT)

First occurrence of data degradation

Well, it had to happen at some point. Please see the last reported bug. Ahasuerus 00:15, 3 May 2006 (CDT)

P.S. Would it be safe to continue with editing while this record is broken? Or could it cause further database degradation? Ahasuerus 10:16, 3 May 2006 (CDT)

The only thing broken is the reference to that particular title, so you're pretty safe. If you're worried, just avoid that particular author.

Ah, thanks, good to know!

If you need to delete an author from a record, for now make sure that there are no empty lines before other authors (for instance cut the last author and past over the deleted author).

Yes, that's pretty much what I have been doing and haven't run into any problems so far.

I may not be able to repair it from my current location.

Not a problem, there are a few other things in the database that I can work on aside from this record :) Ahasuerus 20:03, 3 May 2006 (CDT)

Sysop discussion page?

Is there a hidden Wiki page where SysOps can discuss policies and operational issues like what kinds of user names should be allowed (is, e.g., "Mutherfooker") and what accounts need to be blocked?

A most peculiar case

http://isfdb.tamu.edu/wiki/index.php/Author:Errick_Rios -- hm...

Yeah. I didn't think we'd have many neutral POV wars here, since we've removed most of the forums for that type of discussion. In this case, not exactly a "bibliographic note" either. On IMDB, there are forums for these kinds of discussions (which are de rigueur elsewhere as well), but my impression is that they are extremely high maintenance. Ultimately, this kind of note shouldn't go on that particular page - the decision to make is whether or not discussion pages should exist. Alvonruff 06:11, 9 May 2006 (CDT)

I was thinking the same thing. Substantive (as opposed to bibliographical) comments are high maintenance, flame-inducive and likely to hinder the kind of methodical work we are trying to do here. I rather like the WP approach, i.e. zap any Talk comments that are not related to editing the parent article. Perhaps we could have something in the Help pages that would point those who want to discuss SF to news:rec.art.sf.written? Or even a URL or two along the lines of Discuss this book/author on the "rec.arts.sf.written" discussion group via Usenet or Google Groups on the Work's/Author's page? Ahasuerus 09:17, 9 May 2006 (CDT)

Author:Rob Gerrand has been created by User:Roge56. Ahasuerus 21:03, 9 May 2006 (CDT)

Saw it. Should we play Police State or Social Experiment? Alvonruff 21:33, 9 May 2006 (CDT)

Well, as long as all of these, er, unorthodox submissions are covered by the same copyright-free license, I suppose we can "let a hundred flowers bloom" (TM by Mao Inc.) and then move them to Wikipedia if/when it becomes necessary/advisable. Ahasuerus 21:38, 9 May 2006 (CDT)

P.S. We probably want a standard greeting that we could cut-and-paste on new users' Talk Pages explaining what the ISFDB is (and is not) and what its Wiki is for. I'll see if I can whip something up shortly. Ahasuerus 08:18, 10 May 2006 (CDT)

Even more visitors at Author:Lori C. Schneider. Ahasuerus 23:20, 18 May 2006 (CDT)

Data verification

A proposed approach to capturing verification data within the database (and some thoughts on Series-related complications) have been posted in Bibliographic Rules. Do you think it will be doable/in the ballpark? Or am I barking up the wrong tree?

This looks like the right tree. Appears pretty easy to implement.

Also, what is the limiting factor when it comes to implementing software fixes/adding new features? Is it that you are the only developer on the project and you have to eat/sleep/work sometimes? If so, I guess I was pretty good with the abacus back in the day, and I am sure these mysterious "MySQL" and "Python" can't possibly be that much more difficult to learn. Admittedly I don't like snakes much... Ahasuerus 11:16, 9 May 2006 (CDT)

I think a Pareto chart will show that the eating and sleeping profiles are inconsequential when compared to the working profile. Being on the road five out of the last six working days was a bit unusual. It'll probably be quicker for me to do bug fixes and complete the current variants / pseudonyms / content stuff, but you could certainly start in on newer features like verification (it will require one new SQL table, which is no biggie). Alvonruff 20:32, 9 May 2006 (CDT)

First I would have to download Python and MySQL and see how long it would take me to get up to speed. I gather things have changed a tad since 1994-1995 when I was maintaining the HTML version of the rasfw (and related) FAQs with a text editor. With luck, I should be able to take a closer look at the reptile in the next few days. Ahasuerus 21:34, 9 May 2006 (CDT)

Proposed blocking policy

ISFDB:Policy updated with a proposed "blocking policy". It is a little tougher on vandals than the current WP policy. Ahasuerus 12:38, 10 May 2006 (CDT)

I have no problem with those. I've been implementing swift justice on spammers for a while now. I've enabled whitelisting, so banning should be done on a user name basis, instead of an IP basis. There are also some regular expression filters that prevent the submission of URL lists that are structured in a specific manner. User:Alvonruff

Well, the filter didn't catch the latest spammer, but Recent changes did and I zapped him manually. Ahasuerus

The filter prevents a spammer from saving articles that follow a common spammer pattern (not described here for obvious reasons). Once confronted with a difficulty, they revert to more primitive listings that are immediately obvious. I think that there are some MediaWiki rules that prevent more than n URLs in a post as well, but I haven't seen the specifics yet. Alvonruff 19:43, 15 May 2006 (CDT)

Multiple Authors bug -- IE-specific

In unrelated news, is there any chance that the submission of Works/Publications with multiple Authors might be fixed in the foreseeable future? The problem makes it impossible to enter complete bibliographies for most Authors and I am hesitant to enter partial Work entries that omit co-authors. Ahasuerus 10:19, 12 May 2006 (CDT)

Okay; took me a while to reproduce. It's a bug only with Internet Explorer, which is why I wasn't seeing it. Obviously related to Javascript handling in IE. I'll look at that this evening. User:Alvonruff

Well, IE is not exactly the cuddliest browser out there, but it's the one that most users use, so that's what I test with. A perfectly designed and implemented user interface may be esthetically pleasing, but it doesn't help the users if 80%+ of them can't use it :) BTW, do you have a breakdown of incoming requests by browser type, by any chance? Ahasuerus 12:56, 12 May 2006 (CDT)

• 69% - Internet Explorer
• 23% - Firefox
• 4% - Safari
• 2% - Opera
• Loose change - Netscape, Mozilla, Konqueror, Camino, Mozilla Compatible, Galeon.
• Alvonruff 13:50, 12 May 2006 (CDT)

Sheesh! Either our users are very advanced (fans are slans, after all) or a lot of them live in Finland :) Ahasuerus 14:09, 12 May 2006 (CDT)

Response time

isfdb.tamu.edu has been intermittently slow lately :( Do we happen to know where the problem might lie? The Web server, their internet connection or the ISFDB innards? Ahasuerus 21:10, 13 May 2006 (CDT)

Too many variables; too hard to say. The innards didn't change recently, but could be the server, DNS server, or Google Analytics. Alvonruff 17:59, 15 May 2006 (CDT)

Grendelkhan

Sure, I remember him from his comments on the forum a little while back. The more the merrier :) Ahasuerus 19:56, 15 May 2006 (CDT)

Thanks! grendel|khan 16:01, 16 May 2006 (CDT)

Project Gutenberg links

I'm sure after the database redesign, this is the very last thing you'd want to think about, but a number of SF works have been cleared recently over at Project Gutenberg. See, for instance, the works of Andre Norton. (Okay, it's just one, but there are like three more books on the way.) Would it be worth it to include a field to link a title to one or more PG etexts (sometimes a text is also published from a different edition, or in a different format)? The author records have a freeform URL field, which can be used to link to a PG author record; it might not be worth the trouble to specifically add a field for Project Gutenberg link, especially as a comparatively tiny proportion of works in the ISFDB have fallen into the public domain. I wanted to run this by you before I started adding PG links under "Web Page 2" or whatnot. grendel|khan 12:50, 16 May 2006 (CDT)

We should be able to support a generalized set of web publication links per title. This would include not just Project Gutenberg etexts, but other *freely* available etexts (such as when Hugo nominees are made available online, or when Cory Doctorow makes his stuff available under a Creative Commons license). Rather than making these author links, I'd prefer to make them title links. I'll move this over to the feature roadmap section later. Alvonruff 14:54, 16 May 2006 (CDT)

Sounds reasonable. BTW, there is a portal that collects links to complete works of SF that are available on line. Ahasuerus 15:09, 16 May 2006 (CDT)

Wikipedia discussion of the ISFDB entry -- notability concerns

See ISFDB:Community Portal and the ISFDB's Talk page on WP. BTW, now that we have 3+ active contributors, we may want to start moving general interest discussions from userpages to more generic forums. Ahasuerus 15:06, 16 May 2006 (CDT)

Something wrong with this title.

Titles 112651 and 112652 are two parts of a novella "To the Stars" by L. Ron Hubbard, title 92605, published in serial form. However, they're also showing up in every other work titled "To the Stars", which I see in works by Robert Silverberg (34767), Harry Harrison (39531), J. T. McIntosh (59342), James Spencer (106566), Arthur C. Clarke (119015), Lee Owens (130669), Robert Heinlein (174800) and, of course, L. Ron Hubbard himself. I don't see any way of removing those two serial titles from being listed in these other titles as serialized versions. I can't tell if this is a bug or just data being entered wrong, so I'm entering it only here for now.

To quote the same ISFDB_Feature_List:

10/29/2006 - Review Serial support. Linkage to titles is still performed lexically.

In other words, all Works with "To the Stars" as their title are currently linked to this serial :( Ahasuerus 19:21, 16 May 2006 (CDT)

Also, I don't see any tools to edit omnibus collections (this originally started when I was going to edit the contents of Heinlein's omnibus "To the Stars" (174800); I take it that the proper way to do this is to mark both the title and the publication as type OMNIBUS and then edit the contents of the publication to point to the titles included in it (as in, for instance, A Heinlein Trio)? And that the content-editing tools are currently unavailable? Ah, I see. It should be online in a week or two. grendel|khan 19:12, 16 May 2006 (CDT)

Alternate titles on publications possibly give incorrect linkage.

I added the Croatian translation of "A Fire Upon the Deep" (publication VTRNDDBNMF2002), but the title link ("Title Reference" on the publication page) leads back to the record for the cover art. Usually the title reference, even if there is cover art, points back to the title record for "A Fire Upon the Deep". (See publication BKTG18990.) Did I do something backwards? grendel|khan 00:04, 17 May 2006 (CDT)

Definitely a thing that makes you go "Hmmmmmm". Looking through the data trail:

• The data submission has the correct parent number, so you didn't do anything wrong.
• The integration worked correctly.
• The pub mapping table has two entries in it: one for the title and one for the cover art, which is the way it should be.
• Oh. The pl.cgi app currently uses the first reference it runs into, which just happens to be the cover art information.

Not the desired result, but there are no data consistancy errors - it's just being displayed wrong. It probably happens elsewhere (like in anthologies and collections), we just haven't noticed that one yet. More than a one-line fix, so I'll put it in the queue. Alvonruff 05:51, 17 May 2006 (CDT)

Pseudonym support

I saw that enhanced pseudonym support was now listed as "DONE", so I tried merging "Steven Swiniarski"'s books into "S. Andrew Swann"'s bibliography as well as adding the books that he wrote as "Steven Krane". I can see that some books are showing up "as by XYZ", but I seem to be unable to figure out how to enter this information :( Ahasuerus 16:06, 20 May 2006 (CDT)

There's currently no method for doing the wholesale merging of one author as a pseudonym of another. That's fairly complex (multiple works, multiplied by finding parent versions of each title) but I'm looking into it. On a per-book basis, there are two methods:

• If the pseudonym work already exists, then go to the title page for that work. Select Make Title Variant - this will make the title a variant title. If the work by the real author already exists, put the title record of the parent in the top part of the form. If the work by the real author does not exist, put the title data in the bottom part of the form.

• If the pseudonym work doesn't exist, but the parent does, then go to the parent title. Then select Add Variant Title. Enter the pseudonym version here. User:Alvonruff

Oh, I see, I missed the new choices in the navbar! However, although making Swiniarski's "The Flesh, the Blood and the Fire" a variant title under Swann did work at the Work level, its related Publication record wasn't moved. It now shows up as a Stray. Did I do something wrong during the submission process or did the process of associating this Publication with the new Work didn't work? Ahasuerus 17:28, 20 May 2006 (CDT)

Yeah. Don't touch anything. It's technically correct (the pub is still connected to the variant) but not the desired effect. The title needs to show the publications of any variant childen as well as pubs connected to the parent canonical title - which I thought I did. I'll get back to you on that... Alvonruff 17:38, 20 May 2006 (CDT)

Everything is correct except for the leftover stray. How did you create the variant? On the home system, I did a make variant on the Swiniarski version, and punched in the data to create the Swann parent. I didn't get a stray that way.

Okay. I know what's going on. Swiniarski doesn't have any titles, but he does have pubs. That's not good - *unless* the author is a pseudonym. But the database doesn't know about the Swiniarski <=> Swann relationship (I'm not seeing it at home, because I didn't link all of the Swiniarski titles). We also want the Swiniarski page to refer to the Swann page (see Robin Hobb) so that when people look up Swiniarski, they'll know to go to Swann. I'll create a pseudonym manager (which we need anyway), so that the relationship can be established. (It only has to be done once per pair, and I don't think it's a good idea to automate the detection of those relationships, or not have a way to delete mistaken pairings). Remember - pseudonyms are hard - we just have to make it seem easy... eventually. Alvonruff 18:07, 20 May 2006 (CDT)

Another mystery solved! :-) As far "pseudonyms are hard" goes, I am currently thinking about it. Perhaps there is a way to simplify them, but I'll have to review the current table layout first. Ahasuerus 18:13, 20 May 2006 (CDT)

Comic books by Bill Willingham?

Before I aim my flamethrower at all of these entries, would you say that all of them deserve to die a fiery death like the filthy rotten comic books that they are? And did they slip past Dissembler's safeguards or did they get imported before the anti-comic algorithms alluded to elsewhere were put in place? Ahasuerus 23:08, 20 May 2006 (CDT)

The current Dissembler algorithm uses publisher names and page counts to find comics. These are offenders that went in before those heuristics, which would have caught all but two:

• Fables, Storybook Love (2004) - General, 192p.
• Robin: Unmasked (2004) - General, 128p.

I'm not tracking General as a publisher (and that's probably bogus, Robin is a DC comic property), and the page counts are pretty large, but they are graphic novels. I'll see if I can't pick up some more subject information. At any rate, they are all toastable. Alvonruff 05:47, 21 May 2006 (CDT)

General seems to be a comic book imprint specializing in reprint editions, which tend to have inflated page counts, e.g., Supergirl - The Archives, Volume 2. Also, Showcase Presents: Superman Family, Vol. 1 (Superman (Graphic Novels)) is by DC Universe and from 2006; shouldn't Dissembler be able ot catch it? Ahasuerus 13:34, 21 May 2006 (CDT)

Since ISFDB covers Neil Gaiman's Sandman graphic novels, why not Bill Willingham's Fables? I'd say they're similarly serious works. (BW has also written for The Dreaming, which is a sequel/spinoff series from Sandman.) Vertigo is DC's mature-reader line, not your childhood comic books. --SAJordan 20:15, 7 Oct 2006 (CDT)

As a general rule, trying to select graphic novels (or most anything else) for inclusion based on whether they are "serious works" or not typically ends up in a never ending discussion of what is and is not serious :( The main reason not to include graphic novels and comic books in the ISFDB is simply our current lack of support for comics-specific information. Unfortunately, although it used to be pretty easy to tell regular novels from comic books, things have changed lately and the line is not so clear cut any more. We are still scratching our collective head over this development... Ahasuerus 22:27, 7 Oct 2006 (CDT)

Editor recruitment

I have been thinking about trawling the History section of SF-related Wikipedia articles for promising editor/moderator material. Do you think Mike Christie, who has been doing a fine job of organizing Ace-related materials, would be a good first candidate? Ahasuerus 20:27, 21 May 2006 (CDT)

Wow. According to his contributions, he doesn't even get distracted by Russian history :)

Hey! Leon Trotsky just called -- he objects to being called a distraction! :) But yes, my databank dedicated to late 19th-mid 20th century Eastern/Central European history was one of the things that I hoped to upload to WP before I transcend. Unfortunately, even mildly controversial topics can become unmanageable timesinks on WP due to the way it's run, so my time is probably better spent here, at least until the place is cleaned up and running smoothly. Ahasuerus 22:27, 21 May 2006 (CDT)

Looks like a good candidate, if interested. Alvonruff 21:24, 21 May 2006 (CDT)

He is busy with other SFnal projects right now, but I'll give it a shot. Ahasuerus 22:27, 21 May 2006 (CDT)

I left a message on WP, we'll see how the target reacts :) By the way, if we are going to have more than a couple of people with sysop privileges and/or if more than one of us gets to play with the code, we may need to have a hierarchy of sorts. Some way of determining which software changes should go in, which bibliographic rules get implemented, etc. Ahasuerus 23:22, 21 May 2006 (CDT)

Hi; just got the message from Ahasuerus. Can you tell me a little more about what you would expect of me, both in the way of time and duties? I'm flattered to be asked, but I do have obligations to the OED SF project, and of course I am still tinkering over at Wikipedia, as well as having a real life and family that occasionally require my time, so I'm just trying to get a picture of what you're looking for in an editor. I'm certainly interested in the ISFDB and use it frequently, and have quite a few bibliographical materials and a fair-sized collection to help with validations. So I'm happy to help, but would just like to understand the role a bit better. Thanks. Mike Christie 06:10, 22 May 2006 (CDT)

The only real duty an editor has will be eventually to help moderate submissions by the general public. At present, only moderators are allowed to edit, as we bring more editing tools online, find bugs in the existing tools, and find the need for new tools. So the "official" obligations at present are:

• Try entering data at your leisure. Most of us generally stumble into some area of focus: Grendelkhan is currently working on Heinlein, Ahasuerus is furiously *deleting* content, and I'm mostly testing new editing apps and webbots (which furiously adds content). We all fix errors as we come across them.
• Document confusion you may have in the wiki, so that we can alter the tools or improve/create the online help.
• Document ideas for improvements that will aid the editing process or make the bibliographies clearer.
• Police the wiki for spam and delete mercilessly.
• Create ideas on formalizing and documenting the bibliographic process, such that people can understand the relative maturity of a particular bibliography.

Other than that, there are no expectations, especially not on time. Any help you can provide will be appreciated. Alvonruff 06:34, 22 May 2006 (CDT)

OK, I can sign up for that. Thanks. I'll take a look around before diving in; I'd guess my areas of interest will be bibliographic templates, magazines, and first edition documentation; plus maybe a few specific authors such as Le Guin, Robinson, Knight and a couple of others I have a particular interest in. Mike Christie 08:10, 22 May 2006 (CDT)

<fanfare>You are hereby knighted into the Order of ISFDB Sysops.</fanfare> Alvonruff 08:46, 22 May 2006 (CDT)

Questions about current status

I have looked around a little bit, and I want to check my understanding of where the project is right now. I tried to log out of the isfdb.org (not the wiki) and got an error (which I assume I should report to you), so I can't currently see those pages as they look to someone not logged in or not an editor.

However, here's what I'm assuming is the current state -- please let me know if I'm missing anything significant.

There are two components, the ISFDB, and a supporting Wiki. The ISFDB will allow a logged-in ordinary user to do editing; but that's not yet been generally released to the public. Any ISFDB login is presumably a login to the Wiki too? Which would explain where there are a hundred or more users with no activity on the Wiki. The Wiki is active and available but since its main function is to support the admins/sysops/moderators there is little value in it for most users, who are just going to post new data on the ISFDB itself. If they disagree with a deletion they might end up kvetching at one of us on our talk pages or elsewhere on the Wiki, but that's all they're likely to do.

How did the comics-related titles get in, by the way, if no other users can enter data yet?

Sorry if these questions seem obvious -- I'm relatively new to Wikipedia so the boundaries between underlying elements are not yet completely clear to me.

Thanks. I'll go grab a book or two and try entering some data now, and report back. I may not be very active for two or three days -- other obligations are looming this week. Mike Christie 23:05, 22 May 2006 (CDT)

ISBN format

It appears that ISBNs are stored without hyphens or spaces; is that universal or just the ones I've looked at? I would suggest that if possible we should not strip hyphens or spaces from the ISBN; they do have structure, after all. I don't actually know whether it's possible to reconstruct an ISBN that has no hyphens; I could see it might not be. Mike Christie 23:58, 22 May 2006 (CDT)

ISFDB1 stored the hyphens. There are two problems with hyphenated ISBNs:

1 - It makes searching via ISBN difficult. For example, before submitting a new publication, the webbot Dissembler first checks with the ISFDB to see if it already has a publication with that ISBN. If one exists, Dissembler skips that publication, reducing duplicates. In order to perform an SQL query match, the hyphens in the two ISBNs must line up precisely, or the database engine won't find a match. When the hyphens are removed, there is no such problem.

2 - There is some structure to the ISBNs, but it is not implemented consistantly. During the transition between ISFDB1 and ISFDB2, when hyphens were still present in the ISFDB, Dissembler had enormous heuristics which attempted to take an unhyphenated ISBN from - say Amazon.com - and reconstruct the hyphenated structure to help in ISBN matching. It became apparent that each publisher had their own standards for how they hyphenated their ISBN space, which could sometimes change even between different imprints that were owned by the same publisher. Here are the different hyphenation schemes used by various publishers and imprints that I knew about:

 X-XX-XX-XXXX-X
 X-XX-XXX-XXXX
 X-XX-XXXXX-X-X
 X-XX-XXXXXX-X
 X-XXX-XX-XXX-X
 X-XXX-XXX-XX-X
 X-XXX-XXX-XXX
 X-XXX-XXXX-X-X
 X-XXX-XXXX-XX
 X-XXX-XXXXX-X
 X-XXX-XXXXXX
 X-XXXX-XXX-X-X
 X-XXXX-XXXX-X
 X-XXXX-XXXXX
 X-XXXXX-XXX-X
 X-XXXXXX-XX-X
 X-XXXXXX-XXX
 X-XXXXXXX-X-X
 X-XXXXXXXX-X
 XX-XX-XXXXX-X
 XX-XXXX-XXX-X
 XX-XXXXX-XX-X
 XXX-XX-XXXX-X
 XXX-XXXX-XXX
 XXX-XXXXX-X-X
 XXX-XXXXXX-X
 XXXX-XXX-XX-X
 XXXX-XXXXX-X
 XXXXX-XXXX-X
 XXXXXX-XXX-X
 XXXXXXXXXX

When dealing with hand-written static pages, the standard seems to be hyphenated ISBNs (see the Locus index for example). For database-driven content, the standard seems to be unhyphenated ISBNs (see amazon.com, isbndb.com, abebooks.com). We're always open to change here, so if there is some compelling advantage to hyphenated ISBNs, then we should look and weigh against the disadvantages of hyphenated ISBNs. In this particular case, however, the choice was not arbitrary - I chose unhyphenated ISBNs because the hyphenated versions were causing a great deal of trouble and time.

Real-world example of ambiguity: ISBNs are handed out to publishers in blocks. For instance, St. Martin's owns two ISBN blocks: 0-312 and 0-812. These are also used by it's imprints like Tor. The 0-312 block is used for paperbacks and trades, while the 0-812 is used for hardcover books. Ace owns the 0-441 block. There is an ambiguous hyphenation case in ISBNs that begin with 0-099. Some examples:

   0-09-977150-0 (Red Fox)
   0-099-54971-9 (Wizards of the Coast)
   0-09968-301-6 (Legend)

But even Legend doesn't stick to its own hyphenation scheme:

   0-09968-301-6 (Legend)
   0-09-944870-X (Legend)
   0-099-44371-6 (Legend)

This leads me to believe that the hyphenated structure can be quite arbitrary, even when taken out to the 0-09944 isbn space (that only leaves 3 digits to differentiate the hyphen style). Alvonruff 05:42, 23 May 2006 (CDT)

I hadn't seen so much variation; I can understand now why the database doesn't store them. Thanks for the clarification. It's good to get this documented too; I'm sure I won't be the last person to ask this question. Mike Christie 08:06, 23 May 2006 (CDT)

While some publishers format their ISBNs inconsistently the majority of the books in people’s hands are correct. Probably the biggest exception among active publishers is Tor who always formats their ISBNs as “0-812-58035-4” when they should be using “0-8125-8035-4.”

I personally would like to see formatted ISBNs displayed as they are much easier on the eyes when trying to verify a book’s data. If hyphenated ISBNs are displayed by isfdb I'd recommend also showing the unformatted version so that someone searching for “0340837489” will find the publication. In the interest of breaking as little code as possible I'd continue to store the unformatted ISBN in the database and format/hyphenate as needed for display.

If there’s interest, I could volunteer the ISBN formatting code I did for another book site I contribute to. It’s table driven and follows the per-country rules defined at [www.isbn-international.org] and its sub-pages. At the moment the code is in C and I believe I did a version in VB. See [www.fantasticfiction.co.uk] where down at the bottom of the page ISBNs are shown as “ISBN: 0340837489 / 0-340-83748-9 (UK edition).” The “UK edition” note comes from the group code part of the ISBN. www.fantasticfiction.co.uk chose to go with “0-8125-8035-4” rather than putting a special case in the tables for Tor's “0-812-5*” but we are free to change the rules. :) Marc Kupper 13:07, 2 Nov 2006 (CST)

ER diagram for database schema?

Do you have an entity relationship diagram for the schema? If not, I think I'll draw one myself just to get it clearer in my head; if you have one I didn't want to duplicate work. Thanks. Mike Christie 17:34, 24 May 2006 (CDT)

The closest thing we have is this diagram.

This is exactly what I was looking for; very informative. Thanks. Mike Christie 22:29, 25 May 2006 (CDT)

The case of two Disappearing Vortex reviews

We have just lost the pointers from two Vertex reviews (April 1974 and June 1975) for [1] as I was reshuffling Publication records :( Ahasuerus 15:47, 12 Jun 2006 (CDT)

Most peculiar. All of the records are here, and the title/author matches. I hate this lexical matching crap. Who designed this monstrosity? Alvonruff 16:19, 12 Jun 2006 (CDT)

Fixed. Not a design issue. Coding error (original title stomped while looking up other data). Will probably fix some serial printing problems as well. Alvonruff 16:40, 12 Jun 2006 (CDT)

Oh, nice! Reviews always seemed to be a bit unstable, design limitations aside, but I could never put my finger on it. I'll keep an eye on them. Ahasuerus 16:48, 12 Jun 2006 (CDT)

Data questions

Al, I have been looking at the impact of merge and delete and trying to understand a couple of situations I've seen. Can you help?

• I've seen a "stray publication" listed. What's the definition of a stray publication?

I am a poor substitute for Al, but if I can save him a few minutes, so much the better :) "Stray Publications" are Publication records that don't have Work records pointing to them. Normally, it's a sign that something is out of whack, but there is one case when a Stray Publication (as currently defined) is legitimate: a Publication that belongs to a Variant Title. Al was considering cleaning the code up so that they don't show up as strays, but I don't think he has gotten to it yet. Ahasuerus 19:13, 19 Jun 2006 (CDT)

Thanks! Do you happen to know the data situation that corresponds to "publication records that don't have work records pointing to them"? I think this means that the author named on the publication record doesn't lexically match the author referenced in the pub_authors record, but I'm not sure.Mike Christie 19:37, 19 Jun 2006 (CDT)

I don't think any major tables are currently linked lexically except reviews and serials . Works and Publications should be matched via "pub_content". Do you see two foreign keys there, pub_id and title_id? That should do the trick :) Ahasuerus 19:53, 19 Jun 2006 (CDT)

• Could you tell me if these statements are true?
  • Canonical_author links a title to the canonical author name; this may be a pseudonym though.

True.

• • Each title should have exactly one record in canonical_author for each actual author of that title. If a collaboration is recorded under a single pseudonym (e.g. Lewis Padgett, or Eando Binder) there will be two records in canonical_author since there are really two authors. Conversely if a book apparently by two authors is really by one -- e.g. John Wyndham and Lucas Parkes for The Outward Urge -- then there will only be one record in canonical_authors.

First part true. There are a set of records for the real authors and a set of records for pseudonomous authors - the pseudonym information needs to be stored someplace.

• • Each pub should have exactly one record in pub_authors for each apparent author of that title; e.g. a collaboration will have two records in pub_authors, but if it's under a single pseudonym (e.g. Lewis Padgett) it will be a single record.

That is true. In fact, publication records record the published author name. So, unlike titles (which has both), pub records only has the one. In an earlier incarnation, publication records pointed exclusively to published names, and titles to actual names. That seemed more elegant to me, but it turned out that matching up the actual names to published names was a hard problem - so bibliographies pub listings were missing "as by" data.

• • When a pub is entered:
    • If an author matches lexically, a pub_authors record is created linking the pub to that author; if not, a new author record is created and a pub_authors record is created linking them.

True.

• • • If a title (for the identified author) matches lexically, pub_contents records are created for that title and this pub. If not, a new title is created and pub_contents is loaded using that title.

Not True. Currently new publications generate new titles, even if they already exist. See our earlier discussion on automerging. I'm currently leaning to the following behavior: If a title (for the identified author) matches lexically, pub_contents records and title records are created for that title and this pub, AND a submission is created to merge the new title with it's lexical match. If there is no match, no such submission is made.

• • • The contents of the pub are treated in the same way: match author, then title for that author.

Also not true. Same recommendation as above. Alvonruff 07:28, 20 Jun 2006 (CDT)

Thanks for any help. Mike Christie 18:22, 19 Jun 2006 (CDT)

Another question. Suppose you enter the following two publications: Test Title 1, by Test Author1; and Test Title 2, by Test Author2. This creates two pubs, two titles, and two authors, and two records in pub_authors and two in canonical_author (which is the title_author cross reference). I assume that's what it creates, anyway; and I assume that if the above are novels there is nothing in pub_content. If you now merge these authors, and select "only publication records" on the merge, only the pub_authors record is updated. Now the publication Test Title2 still has an author of Test Author2, but the pub_auths record links it to Test Author1. Test Author1 will display only Test Title 1, and Test Author 2 will still display Test Title2, since this is still the state of the cross-reference canonical_author records. Displaying Test Title2 shows a publication as Test Title2 -- but why? I'm not sure how pubs are selected for display under a title. I'd thought it might be lexically, where the title matches the pub title and the author matches from pub_author, but that can't be right because pub_author has been updated. So I'm baffled. Can you explain? Thanks. Mike Christie 19:37, 19 Jun 2006 (CDT)

Also, I'm curious to know why the merge authors function gives you the choice of merging only titles or only publications. I'm sure there is some scenario that calls for this capability; can you tell me what it is? Mike Christie 21:51, 19 Jun 2006 (CDT)

The pub_content table links titles to publications. So while it's true that each content title is linked to the pub via this table, it is also used to link the novel to the publication as well (if the publication is a container, then one of the things it would contain is a work which is a novel). The purpose of the "only publication records" was to preserve pseudonyms under the old setup. As things are different now, I propose we drop that aspect of merging. Alvonruff 07:28, 20 Jun 2006 (CDT)

Spam or meta-spam?

The strangest spam we have been subjected to so far... Ahasuerus 19:39, 12 Jul 2006 (CDT)

Welcome back!

Welcome back, Al! I hope you are well rested and ready to continue with bug extermination :)

I haven't done as much work as I wanted while you were gone, but I am trying to keep up with the "Upcoming Books" page and beginning to develop data cleansing scripts, starting with Project:Author Names Cleanup. I have also written a script to extract some data that we may want to add to the list of "computationally intensive" pages -- see Project:Static Pages for details.

Other than that, not much has happened while you were gone: more Author biblios cleaned up, more spam squelched, a couple minor Project pages added, a few bugs identified.

OCLC Fiction Finder has undergone a facelift and is now in phase 2 (3?) of their beta process, but still buggy. Fantastic Fiction, arguably the main alternative to the ISFDB, seems to have been doing well lately and becoming more and more comprehensive. Ideally, we would be able to swap data, but they are commercial, so I don't know how far they would be willing to go. NESFA folks, who are working on the next generation of their database, have stopped by to see if we could share again -- see elsewhere.

Also, it occurs to me that we may need to beef up the "Disclaimers" page before we open things up so that all contributors have a very clear understanding of the intellectual property ramifications of any contributions (especially plot outlines) which they may be making. Ahasuerus 14:13, 1 Oct 2006 (CDT)

Thanks. I think the break invigorated things, so I'm actually interested in doing some extensive work now.

It often does. The old saying about variety being the spice of life has been getting a fair amount of support from psychologists lately. Ahasuerus 19:00, 1 Oct 2006 (CDT)

THE REST HAS BEEN MOVED TO Community Portal. Ahasuerus 22:21, 1 Oct 2006 (CDT)

Checking in

Al, just checking in -- I've been out of the country, and working on other things for a bit. How are things progressing towards a beta? My time is limited but I'd like to keep up with what's going on, and contribute when I can. I have built a couple of biblios for Wikipedia (Damon Knight and John Campbell) and can work on correlating that data with ISFDB data when we get to beta test over here. Mike Christie 08:00, 17 Oct 2006 (CDT)

You've been missed. Things are moving a bit slow, as I've been on numerous trips my self (with more coming up), but I have started on the verification support. The last thing to do after that is to vet the current bug list, and separate out a list of "must fix" bugs that need to be fixed before we flip the switch. Alvonruff 05:11, 18 Oct 2006 (CDT)

I have my ISFDB wiki watchlist on my home tabs, so I will keep an eye out for progress, and certainly join in when it gets to bug-culling, beta-planning and so on. Thanks. Mike Christie 12:54, 18 Oct 2006 (CDT)

Welcome back! We are still betaing the software, finding new bugs and areas for discussion -- not that the last two activities will ever come to an end :) -- but things are a little more stable now. I have some concerns about the need for multiple submissions when entering certain books/magazines (pseudonyms, etc), but that's fodder for another page. Ahasuerus 15:15, 18 Oct 2006 (CDT)

Unititled Science Fiction Novel

Al, is this Publication and its Title -- see http://www.isfdb.org/cgi-bin/pl.cgi?NTTLDSCNCF2005 -- a recent Dissembler artifact or Amazon flakiness?

I was fairly ruthless this weekend about rejecting Amazon entries, and I recall integrating a titled version of Nova Swing. I only took the Dissembler run out to March where I began to see titles along these lines. Usually the ISBN is assigned by the publisher before the title is finalized, so if you go too far out with forthcoming books you wind up with entries like the one you showed (I've seen Locus do this as well). This entry was entered about 15,000 publications ago, so it's more likely to be an old Dissembler entry from about 6 months ago. Alvonruff 20:33, 24 Oct 2006 (CDT)

Ok, sounds reasonable. Also, this sequel to Light was originally supposed to be published in 09/2005, but was delayed until 11/06, which may help explain why it fell through the cracks. I have cleaned Harrison up a bit, although Viriconium omnibuses still need to be reconciled with The Locus Index to Science Fiction and Wikipedia. Ahasuerus 01:53, 25 Oct 2006 (CDT)

Also, how far did you get with Z39.50 crawling when (ISTR) you were playing with it a few years ago? I have been putting together the foundations of a Z39.50 spider in my plentiful spare time (to be used to populate Wiki pages and such) and it occurred to me that I may be duplicating what you have already done. Ahasuerus 17:53, 23 Oct 2006 (CDT)

I recall doing something at the beginning of the summer, but I consider it rudimentary. When time permits, I'm more likely to work on the fundamental core engine that could be used by any number of spiders. So I don't think there will be too much overlap. If you get yours running well enough, I'll show you the magic to doing remote submissions to the ISFDB. Alvonruff 20:33, 24 Oct 2006 (CDT)

After playing with Z39.50 some more and seeing how dirty the data is, I am rather disinclined to do any kind of automatic submissions to the database even with the kind of moderation tools that we now have :( However, I can think of a number of ways to use Z39.50 data to support our effort, mostly by auto-generating Wiki pages to be later reviewed by humans.

At this point I have a rudimentary Windows/VB/Perl-based (VB because it's the only YAZ-based tool that I could easily get to work under Windows; I'll also try Linux/Perl one of these days) Z39.50 mongrel^H^H^H^H^H module that can simultaneously query hundreds of Z39.50 targets and generate MARC dumps. For example, here is what I get back when I query a random target for "personal name=Zelazny":

host: breeze.gmu.edu:7090
databaseName: Voyager
Number of records returned: 18
setname: default

Record 1:
001 392623
008 870120s1986    nyu           000 1 eng d
035    $a (OCoLC)15088561
035    $9 ABX5280GM
040    $a PCB $c PCB $d m/c $d VGM
049    $a VGMM
090    $a PS3576.E43 $b B58
100 10 $a Zelazny, Roger. $w cn
245 10 $a Blood of amber / $c Roger Zelazny.
260    $a New York : $b Arbor House, $c c1986.
300    $a 182 p. ; $c 22 cm.

Record 2:
001 392431
008 910503s1991    nyua          000 1 eng  
010    $a    91018153 
020    $a 0553076787 (hc) : $c $22.00 ($27.00 Can.)
020    $a 0553354485 (tp)
035    $a (OCoLC)23768529
035    $9 ABX5076GM
040    $a DLC $c DLC $d SVP $d VGM
049    $a VGMM
050 00 $a PS3569.H392 $b B7 1991
082 00 $a 813/.54 $2 20
100 10 $a Zelazny, Roger.
245 10 $a Bring me the head of Prince Charming / $c Roger Zelazny and Robert Sheckley.
260    $a New York : $b Bantam Books, $c c1991.
300    $a 279 p. : $b ill. ; $c 22 cm.
700 10 $a Sheckley, Robert, $d 1928-

Record 3:
001 418377
008 900913s1980    nyua          000 1 eng  
010    $a    90194484 
019    $a 06513101
035    $a (OCoLC)22891470
035    $9 ACA2732GM
040    $a DLC $c DLC $d OCL $d VVR $d VGM
049    $a VGMM
050 00 $a PS3576.E43 $b C48 1980
082 00 $a 813/.54 $2 20
100 1  $a Zelazny, Roger.
245 10 $a Changeling / $c Roger Zelazny ; illustrated by Esteban Maroto.
260    $a [New York] : $b Ace Book, $c c1980.
300    $a 251 p. : $b ill. ; $c 23 cm.
650  0 $a Fantastic fiction, American.

etc. Aggregating MARC records into something useful is not that hard, although there are a few surprises along the way. Non-MARC records (SUTRS and such) are much messier, although they can also be very tempting since they mostly come from British and other foreign catalogs that sometimes have otherwise-hard-to-find records.

I'll see if I can post something semi-coherent on the topic on the Community Portal page (which needs to be archived badly) in the next few days. Ahasuerus 01:53, 25 Oct 2006 (CDT)

Verification Flag

I see that you have snuck in some Web code for the Verification Flag while nobody was looking :) However, I seem to be unable to change dates from 0000-00-00 to 8888-88-88 for "unpublished" publications any more. Could it be related to this code change? Ahasuerus 19:30, 29 Oct 2006 (CST)

Dissembler gaps, part 7

I wonder why Dissembler found only 4 out of 5 books for Alex_Archer? It missed The Chosen, a January 2007 release, but grabbed Forbidden City, a March 2007 release. Ahasuerus 23:27, 3 Nov 2006 (CST)

Dissembler typically does forthcoming books in subject mode, so if the sources haven't properly categorized a forthcoming book it won't see it. When run in author mode, it did find the book in question. I think it would be useful to run Dissembler in author mode to find these, but there are currently a lot of false positives in that mode, and I'd like to set up a staging database so that a specific ISBN can be permanently marked IGNORE, otherwise I'll have to reject the same books over and over again each month. Alvonruff 08:13, 4 Nov 2006 (CST)

Help links

Once the help screens are stable, I'll go through and make a list of what screens I'd suggest we link from what cgi scripts. Meanwhile just grab anything that looks useful -- I'm inclined to link people to the detailed help in most cases, rather than the "How to" or simplified pages. Mike Christie 08:51, 25 Nov 2006 (CST)

OK, here are the help links I think are worth adding to the edit pages. Each help file should have "Help:Screen:" prepended; I've done this for the first one only. I give the help file name first, then the CGI script name I think would be associated with it.

• Help:Screen:AddPublication -- addpub
• AddVariant -- addvariant
• AuthorData -- editauth
• ClonePub -- clonepub
• EditPub -- editpub
• EditTitle -- edittitle
• MakeVariant -- mkvariant
• Moderator -- list
• RemoveTitles -- rmtitles
• SeriesData -- editseries
• Verify -- verify

Also, the newpub script could have NewPub and NewNovel linked to it, with the latter linked only for the Novel entry screen, if that's possible. If not, just use NewPub in all cases. Mike Christie 09:53, 26 Nov 2006 (CST)

TitleRemove

Thanks for nuking the bogus titles for me -- that was going to be a tedious job. Mike Christie 09:37, 26 Nov 2006 (CST)

Mo problem. I've been in that mode for a few days now, going through Astounding Science Fiction. Alvonruff 09:40, 26 Nov 2006 (CST)

Happy Birthday

I just noticed the list of birthdays on the ISFDB page. Happy birthday! Mike Christie 20:14, 26 Nov 2006 (CST)

Congratulations, you have made it to 50! Not that it means a whole lot in the 21st century (except that you are now officially authorized to make less than complimentary comments about young people who have everything handed to them these days), but it was a non-trivial accomplishment just a few centuries ago. Here is your birthday pie:

       _,..---..,_
    ,-"`    .'.    `"-,
   ((      '.'.'      ))
    `'-.,_   '   _,.-'`
      `\  `"""""`  /`
        `""-----""`

 :) Ahasuerus 21:13, 26 Nov 2006 (CST)

Thanks for the pie. Now get off my yard! Alvonruff 06:58, 27 Nov 2006 (CST)

Oct-Nov 53 Fantastic Universe update?

Al, I was just fixing the attributions on the early Fantastic Universes to conform to the rule being discussed on the Community Portal (i.e. "The Editor" instead of substituting Merwin's name), and I noticed that the title of Merwin's piece was given as "Editorial: The Aliens". I'm pretty sure I entered this as "The Aliens", but I had seen you working through Contento2, so I'm guessing that you made this change based on that source -- is that correct?

If so, I agree with Contento that it's an editorial, but I'd like to leave the title just as "The Aliens" -- that's what's given in the magazine, and I don't see a need to change it. Does he do that for other editorials? Mike Christie 19:59, 27 Nov 2006 (CST)

While going through Astounding, there were several years where Campbell wrote both an editorial and an article in the same issue, so I've been preceding the editorials to differentiate them. I don't have a strong preference, so if you prefer it without, then I'll move it back. Alvonruff 06:05, 28 Nov 2006 (CST)

I've moved it back; I think I'd like to stick to the "use what the publication shows" rule where possible. Thanks. Mike Christie 08:35, 28 Nov 2006 (CST)

Analog January/February 2007

Al, http://www.isfdb.org/cgi-bin/pl.cgi?ANLGJANFEB2007 lists the following two entries:

1. 184 • Rollback (Part 4 of 4) • serial by Robert J. Sawyer
2. 184 • Rollback (Part 4 of 4) • shortfiction by John Allemand

Is the second one a "shortfiction" or "interior artwork"? Ahasuerus 10:29, 8 Dec 2006 (CST)

Not that I ever make mistakes, it should be artwork. Fixed. Alvonruff 12:33, 8 Dec 2006 (CST)

More on help links

Al, I just noticed that the help links on "New <foo>" are still pointing to the old links -- can we update them? The best thing to link to would be Help:Screen:NewPub. Mike Christie 12:58, 8 Dec 2006 (CST)

Sorry - did the work, just didn't upload that particular app. Hopefully fixed now. Alvonruff 13:14, 8 Dec 2006 (CST)

Series deletion

Al, before I spam the Wiki with more feature requests/bug reports, let me ask you if the current lack of "empty series deletion" functionality is by design. As we continue to massage the data, we will have more and more empty "orphan" Series records with no Title records associated with them. Some of the time the ISFDB software handles empty series well and some of the time it doesn't. Would it be possible/desirable to change Title processing behavior so that when an update/deletion results in the last Title in a Series getting either moved to another Series or deleted outright, then the resulting empty series gets deleted as well? And if it is not desirable for some reason, then do we want to have a "Delete Series" option, which would only allow you to delete empty Series? Ahasuerus 17:28, 14 Dec 2006 (CST)

It's desireable, but nontrivial due to heirarchical series. For instance, there are multiple series that have no titles in them, but shouldn't be removed because they are parents to children series that aren't empty. In fact the first version did attempt to delete an "empty" series, but it kept removing needed series, and after adding hack after hack to the heuristics to determine an empty series, I finally pulled it for a later date. I'd say that we should just make this a bug rather than request a new feature - either method still requires a reliable heuristic to detect a series that's truely empty. Once we have that, we might as well clean up after ourselves when needed. Alvonruff 18:55, 14 Dec 2006 (CST)

Thanks, that explains a lot! I'll go and roll up all related bugs on the "Series bugs" page to reflect the fact that they will all go away once the "empty series" issue has been resolved. Ahasuerus 10:35, 15 Dec 2006 (CST)

10056

Al, I made some notes at ISFDB_talk:Beta#More_on_showstoppers about the showstoppers. The one in particular I wanted to ask about was 10056 -- are you OK with adding it to the list of showstoppers? I think it ought to be fixed, but didn't want to add it without checking with you. Mike Christie (talk) 07:44, 15 Dec 2006 (CST)

That look's fine. If you guys feel something should be moved to the showstopper list, go ahead. If I strongly disagree, I'll post a response. Alvonruff 08:45, 15 Dec 2006 (CST)

OK, done. I actually think only 10023 and 10056 need to be fixed for the beta, but that's partly because I can't reproduce a couple of the others on your list. Mike Christie (talk) 09:03, 15 Dec 2006 (CST)

I plan on knocking off a huge number of bugs over the next two weeks, so don't feel conservative. Alvonruff 09:43, 15 Dec 2006 (CST)

Data consistency scripts

Wow, that was fast! :-) Ahasuerus 11:31, 18 Dec 2006 (CST)

I already had a few of those, and the other was easy to put together. Some of the others will be more interesting. Alvonruff 13:58, 18 Dec 2006 (CST)

Possible dataloss bug

Al, just wanted to check this one with you before I posted it. I just addpubbed a pub to "The Tombs of Atuan", and when I updated the result to add an interior art record, I noticed that it changed the NOVEL title record from jvn to blank, though I had not consciously modified it. This is presumably an error in addpub's display. I clicked "Approve" before noticing that change, so it's been deleted. Are we using "jvn" as a length on a standard basis? I don't have anything in the help files for it; should I add something? Anyway, since it might be a dataloss bug, I thought I should let you know. Mike Christie (talk) 12:16, 18 Dec 2006 (CST)

The storylen field has been overloaded for numerous purposes. In this case jvn signified 'juvenile'. See ISFDB:Community_Portal#Using_the_Storylen_.2F_Length_field_to_note_series.2Fvolume_numbers for others. We're not currently displaying these annotations, so it might look like they appear out of the blue, but there are quite a few of these. Alvonruff 13:58, 18 Dec 2006 (CST)

Short Works issues

Al, do you intend to keep Short Works on the Long Works page or was it a side effect of another change? I don't see anything wrong with combining the two as long as the application can handle the extra load without affecting performance.

Also, do you think you could look into the problem with Variant Titles of Short Works appearing twice prior to the beta? It makes it hard to look for real duplicate titles in Short Works. TIA! Ahasuerus 17:47, 18 Dec 2006 (CST)

I'm just starting to play around with the display issues. When we converted to SQL, I was concerned about the application load, and split them into the short and long works, which turns out to be the root cause of many of the display issues. I'll keep the strictly Long and strict Short apps around for a while, but focus on the combined app. There are some other dups as well (like Serials previously printed under a previous novel listing).

Some people like to focus on long works and found the short works cluttered up the bibliography. I find being presented with "No long works available" annoying when working with authors who were/are predominately short fiction authors. Alvonruff 18:36, 18 Dec 2006 (CST)

I tend to like the unified view as well, but I can see how some people may not care about the short stuff. OTOH, I don't think bandwidth -- a consideration 10 years ago -- should be a problem now, not even for all 4 of our 56kpbs users in Grand Fenwick. Also, anything that creates multiple execution paths for the display logic is undesirable as it can lead to functional divergence, as we have already seen. Ahasuerus 17:52, 19 Dec 2006 (CST)

Remaining work?

Al, looks like the only thing left on ISFDB:Beta listed as a showstopper is the Unicode fix. From your summary line it doesn't sound like it's a quick fix. What's your opinion on what needs to be done to go live? I'm going to have a lot of free time between now and around Jan 7, so I'm inclined to say let's just do it, and stand by with mop and broom for any cleanup needed. (The Unicode fix really doesn't seem necessary to me.) This is probably a conversation we should have on the ISFDB Beta talk page, but I wanted to get your take on the remaining work -- no point in debating it if you've still got a week of other things to do before we can open up. Mike Christie (talk) 10:49, 19 Dec 2006 (CST)

I put the editing bugs on the must fix list, as dataloss seems like a severity 1 to me. The unicode stuff (like the Russian errors) are annoying, but don't prevent work from continuing. I think people need to pick out display errors that they think are showstoppers due to high annoyance, but I otherwise vote to turn it on. I'll be working on the features in the background.

I think the next step is for the current moderators to go through the bug lists, determine which ones make them uncomfortable about going to Beta, and put them on the Beta page. When there are no bugs left on the Beta page, we go live. Alvonruff 11:19, 19 Dec 2006 (CST)

Cold Print

I gather from UnaPersson that this edition is actually a collection. I saw you verified it a month or two ago, so I wanted to check that there's nothing odd going on before I correct it to COLLECTION. Was it just an error? Mike Christie (talk) 11:51, 22 Dec 2006 (CST)

Hah! The database had it as a novel and there's no indication on the cover that it's a collection, so I just validated the metadata. I'll make it a collection, and add the stories. Alvonruff 12:00, 22 Dec 2006 (CST)

Front page

When you get a chance, you might want to update the ISFDB front page to say "is now open" . . . . Mike Christie (talk) 12:33, 22 Dec 2006 (CST)

Done. Alvonruff 12:54, 22 Dec 2006 (CST)

Sorting Order for Series on an Author's Bibliography

Al - I was thinking of ways to organize omnibusses so that they don't clutter up the main list of novels too much and on Marion Zimmer Bradley's page created a new series called Darkover Omnibus. That seems to work well enough but the puzzle is that it did not sort immediately after Darkover. The series list looks like it's in alphabetical order other than Colin McLaren and I'm wondering why that's the case. Marc Kupper 15:30, 22 Dec 2006 (CST)

The SQL query doesn't perform a sort at present. What's your preferred sort? Alvonruff 15:35, 22 Dec 2006 (CST)

I’d suspect alphabetically by series name is something most people would understand. Fantastic Fiction seems to sort by the date of the earliest title meaning series get listed in roughly the order the author started them. Of course, someone will come along and say author X is well known for series Y and so that series should be at the top of the page as most people visiting the page will go looking for it meaning ISFDB could turn into a WYSIWYG wiki like thing. Marc Kupper 17:10, 23 Dec 2006 (CST)

Alphabetical sortng sounds reasonable. Some authors have written/contributed to so many series (e.g. Andre Norton) that trying to figure out which series started when would be a headache. Ahasuerus 17:15, 23 Dec 2006 (CST)

Help for the pseudonym editor

I see it's done; cool. Can you give me an outline of what it does it terms of records? I want to make sure I write the help accurately. I assume if you say A has alias B then A becomes the canonical, and all the B records are created as vt's of any corresponding titles -- is that correct? What does it do if the vts are already there -- e.g. "Ursula Le Guin" which is an existing author record which (as far as I know) has all its records marked as vts already? Mike Christie (talk) 16:14, 22 Dec 2006 (CST)

It does not modify title records (and must not) - it simply maps the pseudonym author to the parent author. Imagine what would happen if we had no pseudonyms in the database, and pointed "Alexander Blade" to "Don Wilcox" - and it changed all the records. Then we added "Edmond Hamilton", and then all the records were modified again, blowing away the "Don Wilcox" linkage, and then again for "Robert Silverberg". Pseudonym mapping is specific to each title, so we need to track it there.

Most of the pseudonym magic is already happening in the variant title support, so the actual effect of the editor is fairly underwhelming now. It will cause a note to appear on the pseudonym's bibliography labeled as "Used As Alternate Name By:", followed by a list of all the parent authors that used that pseudonym. In other words, it forms a link that allows the user to navigate from "Ursula Le Guin" to "Ursula K. Le Guin" without doing an additional search. We can also add the link "Wrote under the Following Names" to the author bibliography. We can also create a pseudonym browsing tool if desired.

The tool specifically modifies the pseudonym table, creating a mapping between the pseudonym author and the parent author. This has an effect on the summary bibliography, in terms of displaying extra links, and in the structure of the bibliography itself. (For instance, series information is currently supressed on the pseudonym's bibliography). Alvonruff 17:38, 22 Dec 2006 (CST)

OK, so adding a vt does not currently create a pseudonym record, then? This new tool is the only way pseudonym records can be created? The other question would be how they get deleted -- if either author disappears because all their works disappear, the pseudonym record presumably goes too. Is that right? Finally, since the record just stores pointers, it doesn't matter if you edit the author records in any way -- the pseudonym remains as it is.

I like the idea of a pseudonym browsing tool. The "Who's Hugh" format is pretty good; maybe we can do something like that. I'll put in a feature request and I would think this is a case where the feature definition page will be handy to define requirements. Mike Christie (talk) 08:48, 23 Dec 2006 (CST)

Hi Al – regarding “Make This Author a Pseudonym” – could you please explain the mapping process? The background questions are:

• Is it safe to do this twice on an author? For example, I looked at Brian Stableford vs. Brian M. Stableford and spotted one title in Brian M. Stableford’s list that did not have a vt relationship to the same title for Brian Stableford. I added that one by hand rather than risking “Make This Author a Pseudonym.”
• Does the mapping create title records for the parent if they don’ already exist?
• When comparing parent and pseudonym title records do you look at anything other than the title?
• Is there a record somewhere of this mapping so that as people add new titles to the pseudonym that title records will automatically get added to the parent author?
• Is there any form of linkage between the parent and pseudonym title records so that if one gets modified that the other is updated?
• Can “Make This Author a Pseudonym” be used to remove a pseudonymous relationship? If so, does it just remove the vt relationships or do you also delete the parent title records? --Marc Kupper 15:21, 27 Dec 2006 (CST)

Uptime issues and the limit on the number of Tiltes per Pub

• Our uptime has been less than sterling lately - take a look at the operations page. Would it be possible to query the TAMU IS staff and see if they have been feeding the hamsters?

• We can check.

• How many Titles do we allow per Publication? Books like Science Fiction & Fantasy Book Review Annual 1989 can contain up to 1,000 sub-page reviews. Ahasuerus 00:51, 23 Dec 2006 (CST)

• Theoretically, there's no limit on titles per pub. We have a limit on authors per title, but that's an application limit, not a database limitation. From a practical point of view I wouldn't do more than 30 to 50 titles at a time, just to limit work loss if some interesting error happens (much like when I entered The New Space Opera titles yesterday). Alvonruff 06:03, 23 Dec 2006 (CST)

• Ah, good to know! And yes, Wikipedia has been known to eat particularly elaborate updates as well -- unless your browser had crashed first, of course.

• Let me just make sure that I understand: we are pretty sure that there is nothing in the form processing logic that would result in a minor thermonuclear explosion once the Content Title count reaches, say, 1,000, right? Re-entering 999 Titles would be somewhat time consuming. Ahasuerus 11:01, 23 Dec 2006 (CST)

The forms should be able to work to a maximum postive integer value, which would be around 2 billion entries - so that's your theoretical upper limit. I just tried editing '100 Ghastly Little Ghost Stories', and was able to add 10 more stories, so we know it can do 110. There's no reason why it shouldn't be able to do 1100 (although the processing takes a bit of time). Alvonruff 14:10, 23 Dec 2006 (CST)

Sounds good! Ahasuerus 14:17, 23 Dec 2006 (CST)

Search page notice?

What do you think about adding the beta recruitment notice to the top of the search page results? I'm thinking that that's a page most users will see. Mike Christie (talk) 08:41, 23 Dec 2006 (CST)

Two new display bugs -- high(ish) priority?

Al, could you please take a look at 20062 and 20064 when you get a chance and see if they could be addressed quickly? They were just recently (yesterday?) introduced and they have confused 2 editors so far. Granted, they are "display only" bugs, but I am having a hard time explaining the "display only" concept to editors :( Ahasuerus 16:02, 25 Dec 2006 (CST)

20062 is related to the long form bibliography, which used a different SQL query that hadn't been updated. That appears to be working now; let me know if you see any other errors. 20064 - I think I've seen that, but can't reproduce (possibly fixed by 20062). If you find a page that demonstrates the problem (the current Dinosaur Park example is gone), let me know. Alvonruff 19:51, 25 Dec 2006 (CST)

Looks good on both fronts so far! :) Other than that, I have just added 20065 and also Serials are not being matched correctly under some circumstances that I am still investigating, but no showstoppers at the moment. Ahasuerus 20:03, 25 Dec 2006 (CST)

There was one missing clause in the new SQL query that caused reviews of an author's books to show up under the Reviews section (which should only show reviews performed by the author). Ditto for interviews. Fixed now; mentioned in case you saw some spurious examples. Alvonruff 20:13, 25 Dec 2006 (CST)

Oh, I see! I thought you had snuck in some new and exciting functionality :) It looks fine now, I'll keep trying to puzzle out the Serial display thingie. BTW, I seem to have run into a Kornbluth/Merril novelette (in one of the Dynamics) that has never (!) been reprinted. Could it possibly be either that bad or that overlooked? Ahasuerus 20:18, 25 Dec 2006 (CST)

I'm pretty sure we have all of the anthologies and collections indexed by Contento in his pre-1984 index, which would cover the most likely reprint years. Perhaps it's been reprinted since? Hasn't NESFA done a complete short works of Kornbluth yet? Did Merril do bad work? Alvonruff 20:24, 25 Dec 2006 (CST)

I have checked Contento's Index (now online and last updated in 2005) as well as Locus for 1984-1998. The NESFA collection that you are thinking of was indeed published in 1997, but it only collected Kornbluth's solo short fiction. That's why the title was His Share of Glory :)

Merril's solo work was somewhat spotty and not all of it has been reprinted. She did pretty well as an editor, though. Kornbluth/Merril collaborations were not among Kornbluth's best stories, but still, no reprints at all? Merril's website lists the original appearance of the story, but nothning else. It's not like Dynamic was particularly obscure, so I am a bit perplexed and will give the novelette a try tonight to see what's up. Ahasuerus 20:47, 25 Dec 2006 (CST)

Shortfiction series display messed up?

Al, are you changing the short fiction series display logic to display Long and Short Works together if they belong to the same series? AFAICT, if there are no Long Works to display, Shortfiction series titles are now displayed after all other Shortfiction Titles and without any indication that they belong to a series. For example, Noel Loomis has two shortfiction series consisting of 2 and 4 Titles respectively, which now show up at the bottom. On the other hand, T. Jackson King's The Forty-Seventh Florescence is displayed correctly since (?) it has a Long Work to anchor it. Or so it looks from this side of the fence. I will file a bug report shortly. Ahasuerus 21:06, 25 Dec 2006 (CST)

Corrupt Silver Web record

Al, could you please fix this Silver Web issue when you get a chance? It's missing Tag data and is therefore not linked properly from a related Title, e.g. try accessing it from this title. Also, could we try to address the bug with user-submitted Tags soonish so that we don't have this kind of data corruption spreading? Thanks! Ahasuerus 15:09, 26 Dec 2006 (CST)

This was an easy fix - just edit the pub (which works off the record number) and add the tag. The title launches by tag, so once the pub had one it was fine. Does this only occur when submitting a magazine? Alvonruff 15:35, 26 Dec 2006 (CST)

Editpub crash

Al, I take it you're aware that editpub is crashing with a python error? I assume you're in the middle of some edits to the code, but just wanted to make sure you knew about it. Mike Christie (talk) 16:16, 26 Dec 2006 (CST)

Give me an example - it's working for me. Alvonruff 16:18, 26 Dec 2006 (CST)

I found an example for when there's no isbn data (obviously related to isbn changes today). That one's fixed. Alvonruff 16:22, 26 Dec 2006 (CST)

That seems to have fixed it -- thanks. I was working on magazines, so of course I saw that problem early. Thanks for the Hold function too, by the way -- fast work!! I've held one of Pagadan's and will wait for consensus on the community portal before I reject it. Mike Christie (talk) 19:55, 26 Dec 2006 (CST)

Yes, that Hold button is very nice, thanks! Ahasuerus 20:03, 26 Dec 2006 (CST)

Ditching

I thought you'd like to know that your "ditch the list" addendum made me laugh out loud. Thanks for the "difficulty" notes, too. Mike Christie (talk) 09:16, 27 Dec 2006 (CST)

10069

Sorry about re-reporting 10069! It was late and it escaped me that we had this very same discussion about case sensitivity wrt author names a few months ago. I'll go and engage in vigorous self-flagellation with a wet noodle now... Ahasuerus 13:14, 27 Dec 2006 (CST)

No problem. It does mean that other people will try to do the same, but I think we're doing the right thing. And of course, no one will read the documentation (if it even mentions this situation), so we'll just have to deal with it. Alvonruff 17:25, 27 Dec 2006 (CST)

Well... I have been working with Hayford Peirce (the Analog author), trying to get him up to speed wrt using the ISFDB. He is not finding our user interface particularly intuitive, I am afraid. Here is my explanation of how to Merge two of his titles and here is his response. And if we can't get an Analog author to master one of the more straightforward aspects of the UI quickly, how much luck can we expect with F&SF, not to mention futuristic romance, authors?! :-\ Ahasuerus 23:00, 27 Dec 2006 (CST)

I guess if bibliographies were easy, everyone would be doing them... Alvonruff 05:09, 28 Dec 2006 (CST)

Translations of short stories

Al, have you seen my post on the Community Portal about translated short stories? Am I missing something or do we have a gap there? Also, have you seen T. Jackson King's question about SF art on his Talk page? Ahasuerus 21:48, 28 Dec 2006 (CST)

Hi Al, T. Jackson King intruding, briefly, here. First, thank you very much for the time you took to scan/process/OK my input of Uncle River's spec fiction digest Xizquil 16, and for correcting the Summer Breeze item to correctly show Poem. Question: I note that my info item about Uncle River now being located in Pie Town, New Mexico, does not appear in the online Note version of 16 when you call it up. Is it redundant, or simply wrong etiquette for a Note? Also, thanks for pulling out the karen verba lower case Note text, since it's only intended to last through a Moderator's review, in case there is a question.

I didn't see that particular note in the Xizquil 16 submission. Here's the note string in the submission:

   <Note>There is no ISSN number for this issue. The lower case spelling of 
   karen verba/her poem is intentional. On page 66, back cover, the Good Morning 
   Maya item should be listed as a Poem, not as ShortFiction.</Note>

At any rate, I think that a note about the current location would be fine for the magazine page in the wiki (and can include subscription information if so desired). I have similar data concerning the office address for Astounding in its wiki page. The note for a particular publication should discuss bibliographic issues for that issue (volume number, errors in the table of contents, notes about artists, etc...). A note about the current office location in issue 16 might appear to be a non sequitur in a couple of years. Alvonruff 21:26, 30 Dec 2006 (CST)

No worries, I have already created a Wiki page for Xizquil and moved the note (which Tom resubmitted in #13) there :) Ahasuerus 21:29, 30 Dec 2006 (CST)

Lastly, re Ahasuerus' comment about my genre artists' website, got any outside website guidance to offer? Tom/Tjacksonking 21:11, 30 Dec 2006 (CST)

I think I lost track of both the website and Ahasuerus' comments :) (I feel like I wandered into a discussion from across the room). Where's the website and the comments? Alvonruff 21:26, 30 Dec 2006 (CST)

I believe the art discussion was/is on Tom'a Talk page. Ahasuerus 21:29, 30 Dec 2006 (CST)

10074?

Al, have you seen Python Bug 10074 yet? If so, do you still need the corrupt publication record that errors out when you try to file it or can I delete it? Thanks! Ahasuerus 18:30, 29 Dec 2006 (CST)

Not a corrupt publication after all, just an error in processing reviews of "books by multiple authors". See Open Editing Bugs for details. Ahasuerus 14:01, 30 Dec 2006 (CST)

Pseudonym display

Looks like you're working on pseudonym displays. I noticed an oddity with this pub that I wanted to ask about before I enter a bug, since it might be connected to what you're doing. It's a vt of a Fanthorpe. If you click on "Bron Fane" in that listing, you get no indication that any books were published as by Fane. If you then click through to Fanthorpe, everything looks good. I recall some discussions somewhere about what should display for pseudonyms, but I don't recall how that was settled. Anyway, is this intentional? Mike Christie (talk) 12:11, 1 Jan 2007 (CST)

It's as designed, but that doesn't me we can't make it better. I think the more problematic area is if you follow the publication. For the title, we could put the actual author in the printing of the variant title information:

   Variant Title of: Last Man on Earth (R. L. Fanthorpe)

For publications, I'd like to see the information displayed differently (but don't know how as yet). I want to be able to directly compare the metadata to the publication, and putting an author of "R. L. Fanthorpe (as Bron Fane)" is not what someone would see when looking at the publication. The important thing with publications is that our abstractions have been focused on titles - we leave publications as is, and focus on putting relevant information into the titles to tie things together into a unified bibliography. The linkage in the other direction is tenuous (as we've seen with the "Title Reference" linkage). In general, our linkage tends to flow "downhill" (author->pseudonym->publication) and going in the reverse direction (publication->pseudonym->author) is a bit more difficult, as it cuts across the grain of the design. Alvonruff 12:38, 1 Jan 2007 (CST)

ERROR: Attempted automerge with missing title: 0

What does this message mean? It’s a new-publication that looks ok but I have put on hold so other moderators can look at it. I see at the bottom of the contents list an interiorart with manual-merge and assume it’s about that. As soon as you have seen the message it's ok with me if you want to approve the item. Marc Kupper 00:41, 6 Jan 2007 (CST)

Let's say you have two printings of a collection that are identical except for the page numbers. The first collection is already present, so you clone it, and update the page numbers in the form. In that case, all the titles are automerged. Now let's say the second collection is the same, except it added an introduction. You can add the introduction, but since it wasn't present in the original collection, that title can not be automerged (since it doesn't have a title record to refer to). When a title is added to a cloned publication, it will show up as needing a "manual merge". For interior art, this is no big deal, but if it was a short story, then someone would have to go the Wolfe's bibliography and find that story, and merge it if necessary.

The automerge error at the top of the page is NOT about that. Somehow the parent title for the collection Endangered Species was zeroed out of the form. I'm looking... Alvonruff 06:26, 6 Jan 2007 (CST)

The problems stems from cloning publications that have the wrong type. In this case, someone tried to clone a collection that was marked NOVEL. The app could not determine the referring title because there was no novel by that name. (and no - the app can't try to figure this out. There are 9 titles named "Endangered Species", and matching on the author won't work because publications can have pseduonyms, while the titles have actuals. Cloning a broken pub just propagates the error). The cloning app now generates an error if it can't find a referring title, with instructions on how to fix the situation. Alvonruff 07:06, 6 Jan 2007 (CST)

Thank you Al - Should coverart and interiorart items be merged – maybe as a cleanup project? I had not thought about merging coverart before but if multiple printings share the same cover other than the price I suspect they should be merged and a corner case would be if the font of the title changes or a publisher crops the original cover art where the look of the cover changed without the cover artist’ involvement. Marc Kupper 11:28, 6 Jan 2007 (CST)

I would not merge any artwork at the present time. Alvonruff 11:44, 6 Jan 2007 (CST)

I think that it would be desirable for cover art recods to be merged when the art is identical, but I am not sure how we can tell that they are identical unless the same person is doing physical verification of all affected Publications. ISBN, price, etc are all easily checked by looking at what we have in the ISFDB, but cover art? Hm... Ahasuerus 22:23, 6 Jan 2007 (CST)

I agree. For example, I recently saw a Del Rey reprint which had the same cover artist as the first printing, but which had severely cropped the picture for the later printing. This would look like a merge candidate just from the database information. Mike Christie (talk) 05:31, 7 Jan 2007 (CST)

That’s why I mentioned recently (I forget where but think on feature requests) for a way of caching the ISFDB images. When I do primary verification I try to always have the image displayed match exactly what I have. But, as ISFDB is using 3rd party sources for the images there’s no control over the images changing, getting moved, deleted, etc. My thinking was something that would cache and then verify the images from time to time to make sure they still exist and have not changed so that future editors can see that two cover-arts are in fact the same.

Maybe there should be a wish-list along with the feature requests as an idea would be that thumbnails could be shown in the “Cover Art” sections of bibliographies. The formatting could be done much like how FantasticFiction does it where they run the images four across on the right leading to vertical spacing that’s roughly similar to the text on the left. I suspect you can just display the first publication for a title that has an image. This would make it a lot easier to decide if COVERTART title records should be merged. And, a related thought is to enforce that only titles with verified publications could be merged and to lock the image to the publication. At present with unverified publications the image may not match the publication and so we should not be merging those. Marc Kupper 13:57, 7 Jan 2007 (CST)

Unmerge for moderators?

Would it be possible for unmerge to be turned back on for moderators only, while we're waiting for the new version? It's a darn useful tool, and restricting it to moderators should make it pretty safe (now we've learnt what to do with it). Mike Christie (talk) 14:11, 7 Jan 2007 (CST)

Dataloss – Publication Notes fields are getting silently deleted

Al - heads up on Open_Editing_Bugs EditBug:10082 Dataloss – Publication Notes fields are getting silently deleted. Marc Kupper 13:07, 9 Jan 2007 (CST)

I did a diff of ALL pub notes from 2 days ago (prior to the note change) and from about 1 hour ago. There were 4 pub notes present 2 days ago that are not present now:

• 6282 - "1st ed."; this was put back.
• 100491 - "DAW books no. 132"; this was put back.
• 61583 - "$3.50"; this was bogus and not replaced.
• 106761 - "Originally published by Victor Gollancz in hb in the UK, 1983"; this was put back.

Do you save the XML blobs so you can see if there were any pub-updates to add notes that may then have been deleted by accident? I know I have been adding notes to things though nothing comes to mind where I added/changed a note and then changed something else. Marc Kupper 19:32, 9 Jan 2007 (CST)

Title merge Python error

Al, I just approved sub 198191, a title merge of Sturgeon's "Twink"; it caused a Python error. Can you take a look and see if there was any problem with the resulting data? Unfortunately I neglected to copy the Python error so I can't give it to you here. Mike Christie (talk) 12:54, 15 Jan 2007 (CST)

The data looks okay. It involved titles: KeepId [45868] and DropId [291381]. Alvonruff 15:30, 15 Jan 2007 (CST)

Submission list parse error

Al, sorry for the heads up here, but I thought you might want to look at this. The sub list is throwing this error: "parse error on: BLongley 9650 Woody Allen BLongley Allen Stewart Konigsberg http://en.wikipedia.org/wiki/Woody_Allen http://www.imdb.com/name/nm0000095/ http://www.imdb.com/rg/name-headshot/top_center/http://imdb.com/gallery/granitz/5332/Events/5332/SoonYiPrev_Devan_11236110_400.jpg?path=pgallery&path_key=Allen,%20Woody Brooklyn, New York, USA 1935-12-01

when you display it. I suspect the culprit is sub 199821, which causes a long Python error on display. Mike Christie (talk) 18:51, 15 Jan 2007 (CST)

It's a strange URL that's not XML-compliant. Since it's hotlinking to an IMDB image, I forced it into the rejected state, as we shouldn't be linking images there any way. Alvonruff 21:12, 15 Jan 2007 (CST)

Logic to hide parent title

I took an existing collection, changed it to an anthology, and added the contents. By default the first blank title entry was type ANTHOLOGY and I forgot to change to shortfiction. After saving/approving the publication I took a look at it and was surprised that the first story was missing. I could have sworn I had entered it but had already closed out the windows that had the evidence. I shrugged, added the story, did the merges and all that, and while reviewing the stories realized that two of the stories by other authors were actually essays and not fiction meaning it was a collection and not an anthology. I changed the title/pub type back to COLLECTION and at that point the original story I had entered popped into view meaning I had not been dreaming after all.

I’m now wondering about the logic used to hide the parent title when displaying a publication’s Contents list. It looks like you hide the first title record where the type matches the publication’s type. I don’t know if this is something you can even fix as I don’t know if you have a specific link for the “parent” title or if it just yet another title in the contents that sometimes gets picked to be the “parent” fo rthe title reference part of the display. Marc Kupper 01:12, 18 Jan 2007 (CST)

Missing review?

I don't know if it's a bug or misunderstanding so I'll ask first. This review is not showing up on the author's page though it's on the reviewer's page. At first I thought it had to do with the ^ style VT in the title but that did not fix it.

Possibly related is when you edit the publication contining the review to change the title that the moderator page shows the a spurious content "change." I have left the update in the queue so you can see what I mean.

FWIW - When you are done looking/thinking we should fix review's title to The Many Worlds of Poul Anderson as the review was published in December 1974 and the VT The Book of Poul Anderson was not published until August 1975 meaning it's not mentioned in the review (assuming the reviewer did not have access to a time-warper). Marc Kupper 04:02, 18 Jan 2007 (CST)

I dropped this to remove the item from the queue. Marc Kupper (talk) 15:59, 3 Feb 2007 (CST)

Repeating titles

Al, I don’t know if something’s messed up or I just hit two strange data elements in a row. I was looking at the title The Waterfall and two of its publications 1 2 contain repeating titles. Is something messed up? Marc Kupper (talk) 18:13, 20 Jan 2007 (CST)

This is a display bug - I'll file the report in the bug reports and you can ignore this message. Marc Kupper (talk) 18:47, 20 Jan 2007 (CST)

Author page with no data

I'm trying to understand why a page for Neal Barrett, Jr (without the period after "Jr") exists. It should say "Author not found." Tests with other "Jr." authors finds that removing the dot results in an author-not-found page. There are no variant titles on Neal Barrett, Jr.'s page. A title search for "Neal Barrett, Jr" finds many records for "Neal Barrett, Jr." (with the period) and none without. The Advanced Search page is crashing at the moment when I search for publications by Neal Barrett, Jr and so I can't confirm that there are no publications but assume they would have been listed as strays on the author page. Marc Kupper 00:55, 23 Jan 2007 (CST)

He's got a bazillion award entries under the old name ([2]), so the author reference count isn't zero, so it doesn't get deleted. Would you like this cleaned out? Alvonruff 05:06, 23 Jan 2007 (CST)

Thank you Al - just knowing it was award entries is enough - I should have spotted that as now I remember this happening before in that I had a slightly different than canonical spelling for an author, got the blank page, and saw the awards which explained why it was not an "author not found." I don't think it needs to be cleaned out as I believe you are working on the awards codes these days anyway and so suspect the problem will fix itself as the awards get linked to the correct author name. Marc Kupper 23:27, 23 Jan 2007 (CST)

Warning on Fantastic Sep 64 edit

Al, I was cleaning up a weird version of David R. Bunch's story "2064, or Thereabouts" and ran into something I wanted to check with you. It is recorded as being by Darryl R. Groupe and David R. Bunch, but the original Fantastic (Sep 64) apparently has it only by Groupe (there was another Bunch story in that issue, which is evidently why he used a pseudonym). I added the correct version, but when I did remove title I got this message: "WARNING: Unable to local the title reference for this publication. Removing titles while in this state is dangerous. Check to make sure the publication type is correct (collection, novel, anthology, etc.). Then come back and remove the title in question." Is this just because it's a magazine and hence has no direct title ref? What's the bad thing that can happen? I figure the likely problem is that one could remove the title that is supposed to be paired with the publication by mistake; of course that can't happen here.

Anyway, I didn't want to proceed without checking with you. Let me know whether I can go ahead. Thanks -- Mike Christie (talk) 22:56, 26 Jan 2007 (CST)

The warning shouldn't apply to magazines, so I've changed the check. For non-magazines, if the title ref can't be determined, then it means we can't disallow the removal of the title reference. So, it would be possible for a user to remove the COLLECTION title associated with a COLLECTION, for instance (such removal would be okay for an OMNIBUS). Alvonruff 04:49, 27 Jan 2007 (CST)

Bureaucrat powers

Al, I'm getting ready to suggest one or two names as moderators on the community portal (section ISFDB:Community Portal#Busy day). Ahasuerus, Grendelkhan and I are bureaucrats, but I didn't want to assume that that meant you were comfortable with us creating moderators without referring to you. If you're OK with this approach, I will suggest a name or two, wait for a few days or until most active editors have chipped in, and then set the bit (or not) per the consensus. Let me know if you would rather be the one to make the decision about setting that bit. Mike Christie (talk) 17:39, 2 Feb 2007 (CST)

You guys should feel free to promote moderators as you see fit without getting final approval from me. Alvonruff 18:33, 2 Feb 2007 (CST)

Python error in title.cgi

Al, just in case you don't know yet, there's a Python error in title.cgi: "NameError: name 'parent_title' is not defined ". I didn't think it was worth filing a bug as I assume you're just doing a code upgrade, so just FYI. Mike Christie (talk) 14:26, 3 Feb 2007 (CST)

It's feeling much better now. Alvonruff 16:19, 3 Feb 2007 (CST)

Apostrophes in Series names bug (30015)

Sorry to hear that you will be going on an extended hiatus shortly, Al! Nice job on the subgenre keywords, BTW.

I thought I had reported Bug 30015 and was patiently waiting for it to be fixed, but I guess I hadn't. If the bug could be exterminated before you go into hybernation, it would be very helpful! Ahasuerus 14:24, 9 Feb 2007 (CST)

Things to look at before you go

I'm reluctant to ask for anything before you go, just because of the risk that a change causes problems. On thing that occurs to me would be to add the "new" stylesheet tag to show where a publication link has no text yet, as you've already done with the bios. Mike Christie (talk) 05:22, 10 Feb 2007 (CST)

That one's done. Alvonruff 07:49, 10 Feb 2007 (CST)

Thanks. If you're not gone yet, one other that would be good is 20098; that one has almost caused me to reject correct subs.

Incidentally, someone posted a complimentary note on rasfw about the new banner on the main page, and gave the original picture sources. They asked about copyright -- I don't know if you think that's going to be an issue, but I thought I'd mention that the question was raised. Mike Christie (talk) 09:14, 11 Feb 2007 (CST)

Duplicate contents

Remember when you rejected my deletion of one of the duplicate contents for "Narabedla, Ltd"? Well, I think I've come across another incident like that: "The Clockwork Traitor". You fixed the last one by hand, so I'm holding off from cloning from that for now. BLongley 09:49, 10 Feb 2007 (CST)

Fixed. Clone away. Alvonruff 11:13, 10 Feb 2007 (CST)

Thanks - any clues as to why that happens? If only so people can avoid my original deletion decision... BLongley 16:11, 10 Feb 2007 (CST)

"Internal Server Error"?

Al,

When I am trying to approve certain submissions by "Chris J", I am getting a 5+ minute wait and then the following error:

Internal Server Error The server encountered an internal error or misconfiguration and was unable to complete your request.

Please contact the server administrator, webmaster@academy.tamu.edu and inform them of the time the error occurred, and anything you might have done that may have caused the error.

More information about this error may be available in the server error log.

---

Apache/2.0.55 (FreeBSD) DAV/2 mod_fastcgi/2.4.2 SVN/1.4.0 mod_ssl/2.0.55 OpenSSL/0.9.8a mod_apreq2-20051231/2.5.7 mod_perl/2.0.2 Perl/v5.8.8 Server at isfdb.tamu.edu Port 80

This is only happening to some of Chris J's submissions and nobody else's, at least not that I have seen. Would you happen to know what may be causing these errors? Ahasuerus 01:09, 16 Feb 2007 (CST)

They only happen to titles starting with "Doctor Who and The D" as the pub tag space for that title was exhausted. Working now. Alvonruff 06:58, 16 Feb 2007 (CST)

Thanks! :) Ahasuerus 13:43, 16 Feb 2007 (CST)

The Age of the Pussyfoot

I see you just approved a title merge of this title, so I thought I'd go see what was done. But the title appears to have disappeared totally, although the serial is there and there's a review too? I have a copy of the Pub here, so I'm wondering if it got merged with something it shouldn't have? BLongley 14:45, 17 Feb 2007 (CST)

The title was changed to "The Age of Pussyfoot" (note removal of the word 'the'). That title does show up in the Pohl bibliography. The name of the serial and the review was unmodified, so they no longer match up with the new title. Perhaps you should discuss with User:Scott Latham to see what the intent was. Alvonruff 17:47, 17 Feb 2007 (CST)

That was a mistake: I meant to merge to The Age of the Pussyfoot and clicked the wrong choice. What can I do to recover? (Scott Latham 15:49, 18 Feb 2007 (CST))

It just required editing the title back to "The Age of the Pussyfoot". I've already done that. Alvonruff 09:31, 19 Feb 2007 (CST)

Star Trek the Next Generation -- series data corruption

Al, here is the note that I left on Chris J's Talk page a minute ago:

Chris, you have submitted a change to the Star Trek Next Generation Numbered sub-series of the Start Trek Next Generation series that would blank out its name. I put the submission on hold and then hit a wrong button, so it was accidentally approved. As a result, the sub-series name has disappeared -- see http://www.isfdb.org/cgi-bin/pe.cgi?1835 -- and it can no longer be edited. I will ask Al, the programmer, to fix the database manually and add an extra check in the software that would prevent this from happening in the future. For now, could you please refrain from editing/adding Star Trek the Next Generation books until the problem has been fixed? Thanks!

Could you please take a look into this? Thanks! Ahasuerus 19:48, 21 Feb 2007 (CST)

Fixed. Alvonruff 21:09, 21 Feb 2007 (CST)

Thanks! Do you have enough information to fix the software or do you need more details? Ahasuerus 22:11, 21 Feb 2007 (CST)

Two invalid subs

Al, just a heads-up on two invalid subs that can't be rejected -- 288091 and 344051. They both say "invalid record" rather than giving a Python error, as the first one used to; there's no "reject" button, though. Can you force them both to be rejected? Thanks. Mike Christie (talk) 20:34, 25 Feb 2007 (CST)

Moved 280091 (not 288091) and 344051 to rejected. Alvonruff 12:13, 26 Feb 2007 (CST)

Python error

Al, there was a poorly constructed submission earlier this evening. I discussed it with the editor and we decided to do it differently. I waited for the editor to submit the alternative edit, approved it and then went back to the "bad" submission so that I could reject it. It turns out that the submission is now erroring out in the Proposed New Publication Submission form, so I can't reject it. Could you please take a look at it to see whether it's a new bug or an existing one? Thanks! Ahasuerus 23:21, 27 Feb 2007 (CST)

Fixed. Error handling case. We'll probably go through a period of these now that there are more moderators and submissions may be approved in an arbitrary order. Alvonruff 06:59, 28 Feb 2007 (CST)

Yes, it does appear likely since our two step approval process creates openings for various inconsistencies to exist for brief (and sometimes not so brief) periods of time. Oh well, one dead bug at a time! :) Ahasuerus 23:18, 28 Feb 2007 (CST)

Another one: Variant Title ERROR breaks publication listing

The story The Street That Wasn't There by Clifford D. Simak and Carl Jacobi causes an error in all publications that contain it, like CRTRBYND7D1975:

The Street That Wasn't There • (1941) • shortstory by  --> -->

IndexError	Python 2.4.2: /usr/local/bin/python
Wed Feb 28 07:50:35 2007

A problem occurred in a Python script. Here is the sequence of function calls
leading up to the error, in the order they occurred.
/www/isfdb.tamu.edu/root/cgi-bin/pl.cgi
 453         # Contents Section
 454         #################################
 455         PrintContents(titles, publication, concise)

Lengthy rest snipped, it ends with "IndexError: list index out of range" and doesn't recover to list other contents or even properly close the HTML. Now I see that the title page shows "Variant Title ERROR: Parent Title=56956" which might be the source of the problem; however this title, story The Lost Street by those two authors, displays all right, only without any mention of a variant title. --JVjr 08:01, 28 Feb 2007 (CST)

So is the correct parent 56955, NOT 56956? Alvonruff 05:33, 1 Mar 2007 (CST)

Yeah, I guess so: now I see that 56956 gives empty "Title Record Error" and it's quite plausible that calling a non-existent parent will wreak all kinds of havoc. So this should be correctable just by changing the number in the DB; still, it might be a good idea to change the script in some way either to not break down in such a case so much (cf the title listing), or prevent such a data configuration from arising. --JVjr 06:15, 1 Mar 2007 (CST)

And now I see it's changed into much nicer

The Street That Wasn't There • (1941) • shortstory by [PARENT TITLE ERROR] Clifford D. Simak and Carl Jacobi ].

Great (& quick) work, thanks! --JVjr 06:21, 1 Mar 2007 (CST)

Broken submission

Here is an interesting submission for you to take a look at when you have a chance. We had a couple of bogus Titles in this anthology and I removed and deleted them before going back to this submission. Now that the Title records are gone, the submission is in limbo since it can be neither approved nor rejected. Feel free to zap the submission at your convenience since everything in it is already in the database. Ahasuerus 16:25, 10 Mar 2007 (CST)

Edit: And here is another submission that can't be approved/rejected. Ahasuerus 18:08, 17 Mar 2007 (CDT)

Converting Magazines to Anthologies

It looks like you can't convert a Magazine to an Anthology cleanly since an EDITOR Title can't be edited or removed. Is there a workaround? Alsol, when I merged an old (and by then Publication-less) Anthology Title and an Editor Title, we lost an associated review. Is this a known issue? Ahasuerus 02:02, 19 Mar 2007 (CDT)

Pyhton error on lost MySQL connection

Here is what I got while approving a submission:

update titles set title_copyright='1974-03-00' where title_id='17334' delete from canonical_author where title_id='435171' select COUNT(author_id) from canonical_author where author_id=5 select COUNT(author_id) from pub_authors where author_id=5 update pub_content set title_id=17334 where title_id=435171 delete from titles where title_id='435171' --> -->

OperationalError Python 2.4.2: /usr/local/bin/python Sun Mar 11 18:38:23 2007

A problem occurred in a Python script. Here is the sequence of function calls leading up to the error, in the order they occurred.

/www/isfdb.tamu.edu/root/cgi-bin/mod/ta_merge.cgi  
 208                 merge = doc.getElementsByTagName('TitleMerge')

 209                 if merge:

 210                         TitleMerge(db, doc)

 211                         submitter = GetElementValue(merge, 'Submitter')

 212                         if debug == 0:

TitleMerge = <function TitleMerge>, db = <_mysql.connection open to 'localhost' at 82a580c>, doc = <xml.dom.minidom.Document instance>

/www/isfdb.tamu.edu/root/cgi-bin/mod/ta_merge.cgi in TitleMerge(db=<_mysql.connection open to 'localhost' at 82a580c>, doc=<xml.dom.minidom.Document instance>) 
 137                 dropCanonicalAuthors(db, RecordIds[index])

 138                 dropPubTitles(db, KeepId, RecordIds[index])

 139                 deleteTitle(db, RecordIds[index])

 140 

 141                 ######################################################

global deleteTitle = <function deleteTitle>, db = <_mysql.connection open to 'localhost' at 82a580c>, global RecordIds = [435171, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], index = 0

/www/isfdb.tamu.edu/root/cgi-bin/mod/ta_merge.cgi in deleteTitle(db=<_mysql.connection open to 'localhost' at 82a580c>, id=435171) 

78 print "

", delete 79 if debug == 0: 80 db.query(delete) 81 82 def moveTitleColumn(db, column, keep, drop): db = <_mysql.connection open to 'localhost' at 82a580c>, db.query = <built-in method query of Connection object>, delete = "delete from titles where title_id='435171'" OperationalError: (2013, 'Lost connection to MySQL server during query') args = (2013, 'Lost connection to MySQL server during query')

=======

I wonder whether (a) it may have caused data corruiption and (b) we want to capture these errors and process them more gracefully? Ahasuerus 18:56, 11 Mar 2007 (CST)

Bad Title Merge submission

There is a bad Title Merge submission, which I put on ice pending your return. Ahasuerus 23:52, 13 Mar 2007 (CDT)

That IS an interesting one. Think I'll save that for an after dinner treat tonight. Alvonruff 12:36, 14 Mar 2007 (CDT)

It looks like Bill has figured it out :) Ahasuerus 19:40, 16 Mar 2007 (CDT)

Bad record in mod queue

Al, another record to force into a rejected state: here. Thanks. Mike Christie (talk) 17:35, 24 Mar 2007 (CDT)

I think I know what's causing these "bad" submissions -- see the newly created EditBug 10101 in Open Editing Bugs. Ahasuerus 18:07, 24 Mar 2007 (CDT)

Enemy Mine/Longyear Author Update

Al, could you do me a favour and take a look in the edit history for an Author Update on Barry Longyear? I have a suspicion that someone updated "Barry Longyear" to "Barry B. Longyear" when it should have been left as an alternate form of the name. If so, this might be reversible with relatively little trouble. I spotted it because of this submission, which would have made sense if the underlying author on one of them had been "Barry Longyear". I did look back through the last couple of pages of edit history but didn't spot it -- maybe we could do with larger page sizes on edit history, or a search feature? Thanks. Mike Christie (talk) 09:57, 25 Mar 2007 (CDT)

Parse error on moderator screen

Al, please take a look at ISFDB:Community_Portal#Unknown_by_Unknown. Thank you. Marc Kupper (talk) 13:51, 28 Mar 2007 (CDT)

There is what may be a related issue with this MakePseudonym submission. One of the involved Author records errors out in Edit Author. Mismatched parameters in "printauthrecord", perhaps? Ahasuerus 01:32, 6 Apr 2007 (CDT)

How to track modifications in database as there are no modified_at columns ?

Hi, I am new here so I don't really know where to ask this so I hope you can help me. I would like to be able to track changes in various table rows over time, so that I can easily identify if an author, title, publication etc. has changed over time. This tracking information could be used for external systems that need to synchronize (import AND export) with isfdb. The isfdb schema unfortunately has no timestamp columns (i.e. modified_at columns) so the usual way of doing this won't work. I am aware of the history table but I don't know if this table can provide what I need as I haven't grasped the exact meaning of the various columns. mortench 14:23, 10 Apr 2007 (CDT). Thank you.

For database table layout, this is the right place to come (so long as I'm actually paying attention to things). The history table is an experiment in progress, so I don't consider that reliable at present.

If you take a look at the Database_Schema, could you suggest the changes that you are looking for? (especially if there is "a usual way"). Given that what we might consider a single unit (such as a title) actually spans multiple tables, it would be helpful to have feedback on what additions need to be made (or better still, what result it is that you are trying to accomplish). Alvonruff 18:35, 13 Apr 2007 (CDT)

For tracking database modifications, the usual lightweight standard apprach is to add a "created_at" and "modified_at" column in all tables that we want to track changes for. For multispanning table units, like your title example, this involves adding these two columns to ALL involved tables. This approach is generally considered the bare minimum for tracking changes. It does not allow you to track exactly which columns was changed (you need shadow history tables or some alternative approach for that). However this simple created_at/modified_at method is often good enough. One significant benefit of this usual apprach is that it is very easy to implement (with no or s very small impact on your application code). You implement it by adding something like this to every table in the isfdb database:

ALTER TABLE ISFDBTABLENAMEHERE

ADD created_at TIMESTAMP DEFAULT 0,

ADD modified_at TIMESTAMP ON UPDATE CURRENT_TIMESTAMP;

Now the updated column will automatically be set to the current time when a row is updated without ANY changes to your application code (your application code does not have to touch this column at all unless you want to expose the data in the GUI). Unfortunately, to make sure the created column is also set, you either need to modify SQL inserts so that an explict NULL value is inserted in the column (f.x. "insert into ISFDBTABLENAMEHERE (columnX,columnY,created_at) values('value1','value2',null)") or you need to define a trigger (only possible in MySQL 5+).

If you need additional help/information let me know - you can find my contact info at my personal page if you want to communicate outside the wiki. I am also available if you want to discuss some other database schema stuff. I will be happy to help with what I can.

You might also want to visit this link [3] for some general discussions on implementing created_at/modified_at in mysql.

mortench 06:55, 14 Apr 2007 (CDT)

I think the history table in the isfdb schema would do what you want, but right now it only tracks changes to the AUTHORS table (although it has a column to allow it to track changes to other tables as well). I think the other necessary mechanism for tracking ISFDb changes would be something to keep track of merges and deletions, which are pretty frequent. --WimLewis 12:31, 14 Apr 2007 (CDT)

Yea, I also figured at first that the history table could be a candidate but as you said it is incomplete (and awfully complex which is hard implement). The advantage of created/modified columns is that this scheme is very easy to implement with no/little changes in application code. As I already written separately to Al, if you want to track individual columns, the recommended next step would be to create shadows history tables for each of the tables in the database and implement triggers for create/update/delete. Although more work, this also has the benefit of keeping changes out of the application code.... But anyway just for starters, I think implementing automated created/modified columns would be a big (and easy) step forward. mortench 02:38, 15 Apr 2007 (CDT)

Welcome back!

Welcome back, Al! I was beginning to worry that the Eddorians had finally gotten you, but clearly they still have a long way to go.

As you pointed out over on What's New, we have reached a critical moderatorial mass, although it was a little hairy a few weeks ago when we had enough almost-but-not-quite-self-sufficient editors to make the moderators' life difficult. At this point, of the 14 people who have made more than 1,000 edits only four are not moderators and two of them (Kraang and WimLewis) are getting close to self-sufficiency.

While you were gone, I spent some time playing with Python (I blame long flights with no net access!) and feel that in a few more weeks I should know enough to be dangerous. I have compiled a list of minor issues that I thought I could start working on. Any thoughts/suggestions/cries of despair? Ahasuerus 21:39, 14 Apr 2007 (CDT)

The Eddorians haven't finished me off, but they did make me perform unspeakable managerial acts over the last few weeks. There will be some related followup activity in May, but after that things should be more normal (whatever that means these days).

It does seem that the database and programming sides are now the most obvious weak links. I'm not too concerned about having 2 people work on the code base in the short term, but in the long term I'll probably have to put the code up at Sourceforge, so that people can do checkouts/checkins of the latest code without lobbing tarballs of code at each other and trying to figure out which code belongs to which version. This will entail working in CVS, and while I've occasionally used CVS (more comfortable with Subversion or ClearCase), I haven't set up a repository on Sourceforge before, so I might take some time with that one. We also need to figure out how to build a sandbox of some sort, so that people can test out changes without taking the site down (I do this on the home system).

Nonetheless, I will endeavour to keep my code snapshots on the website uptodate, and whenever you feel like hacking please do so and send the changes back to me. Alvonruff 07:04, 15 Apr 2007 (CDT)

Sounds good, I'll send whatever I have your way! By the way, when you have a few minutes, could you please review the first half dozen submissions stuck at the top the queue due to errors and force them to be rejected? Ahasuerus 15:32, 15 Apr 2007 (CDT)

FWIW, I intend to set up a ISFDB development sandbox for myself eventually, and once I've done that I'd be willing to provide ssh logins for other people who want to bang on the isfdb code. I could also host the source-code-control system, though I think Sourceforge might be better for that. I think Sourceforge supports Subversion these days. OTOH, since the current system seems to be RCS, it seems like CVS (which is just a wrapper around RCS) might be simpler ... or we could be crazy and modern and go with Darcs or Monotone or Codeville or Bazaar or or or ... :) --WimLewis 21:41, 16 Apr 2007 (CDT)

If and when we get enough programming "staff" to handle the software end of the project, I will be all too happy to concentrate on contents issues. Entering and verifying my 20,000 volumes in the ISFDB will be enough to keep me busy for a while once I am finally done with my wanderings. Ahasuerus 00:29, 17 Apr 2007 (CDT)

Pesky DisplayBug 20048

Do you think you could bump DisplayBug 20048 up on the list of priorities, perchance? It has confused at least 2 of our moderators over the last few weeks. It causes pseudonym data to be displayed in a very confusing way that is probably indirectly responsible for some data degradation because it misleads our editors, who then create bad submissions. Ahasuerus 17:07, 16 Apr 2007 (CDT)

Hmmm.... I'm probably being dense, but what exactly is the problem? The author of the work in the example is "Dave Wolverton" and there is a variant title as by "David Farland". Is the issue that the ORIGINAL publication was by Wolverton, and in that case we shouldn't display "as by David Farland"? (That seems busted to me as well; if someone has read the version by Farland and is looking for associated data, they'll never find it). If so, I take it that the relevant parts of the bibliography should read:

   *  Golden Queen
         o Worlds of The Golden Queen (2005) [O/1,2] [as by David Farland ]
         o 1 The Golden Queen (1994)
         o 2 Beyond the Gate (1995)
         o 3 Lords of the Seventh Swarm (1997) 
   *  The Runelords
         o 1 The Sum of All Men (1998)
               + Variant Title: The Sum of All Men (1998) [as by David Farland ]
               + Variant Title: The Runelords (1998) [as by David Farland ] 
         o 2 Brotherhood of the Wolf (1999) [as by David Farland ]
         o 3 Wizardborn (2000) [as by David Farland ]
         o 4 The Lair of Bones (2003) [as by David Farland ]

That is, only "The Golden Queen" and "Beyond the Gate" are affected? If so, this will be nontrivial as it requires the work bibliography to have deep knowledge of the publication bibliography, including which authorship is relevant. To be workable, this probably requires an override of some sort to command the display routines to bypass the display of the pseudonym data. Alvonruff 18:35, 16 Apr 2007 (CDT)

That's right, the affected titles are in the Golden Queen and The Runelords series. As an aside, Wolverton started his career in the late 1980s as a science fiction writer, but it pretty much petered out by the late 1990s. At that point he reinvented himself as "David Farland", a fantasy novelist, and has done reasonably well since then.

The way I see it, there are currently two problems with his biblioraphy. The first problem is that The Golden Queen and Beyond the Gate were originally published as by Wolverton and then reprinted as by Farland. As of this morning, the software was displaying the title as by Farland only and there was no mention of the Wolverton version. Now the software displays the Wolverton version, but not the Farland version, while I think that the preferred behavior would be to display:

   *  Golden Queen
         o Worlds of The Golden Queen (2005) [O/1,2] [as by David Farland ]
         o 1 The Golden Queen (1994)
               + Variant Title: The Golden Queen (2005) [as by David Farland ]
         o 2 Beyond the Gate (1995)
               + Variant Title: Beyond the Gate (2005) [as by David Farland ] 

The second problem is that The Sum of All Men was never published as by Wolverton. It was published as by Farland, although it had two different titles in the US and the UK: The Runelords and The Sum of All Men, respectively. I would think that we would want it to be displayed as:

         o 1 The Sum of All Men (1998) [as by David Farland ]
               + Variant Title: The Runelords (1998) [as by David Farland ]

The latter problem appears to pop up only when there is more more than one Variant Title. Does the proposed approach sound reasonable/doable or am I missing something? Ahasuerus 19:40, 16 Apr 2007 (CDT)

I'll have to look at the second case; it might be a simple matter of an extra check (well at a minimum 2 passes through the variants). The original case is now being handled by a new control bit field, which allows exceptions to the normal display behavior. There is no easy way to implement the desired effect via an algorithm - every title would have to look up all publication, and look up the authorship on each publication, and understand canonical vs. variant vs. pseudonyms, which would vastly complicate and slow down normal pages. Instead we'll have the ability to tag peculiar circumstances like this one and just ignore variant pseudonyms in this case. This field is extensible to other cases, but the current case would be SUPPRESS_AS_BY_DISPLAY.

I don't have a user interface to set the bit yet - I have added the new field to the title table, added the code to handle the case, and tweaked the two Wolvertons and a couple of Heinleins by hand. Alvonruff 21:13, 16 Apr 2007 (CDT)

Actually, I think I'll have to look at this in more detail. This isn't so much a bug as we haven't written down what should be displayed in specific cases like this, and some of the requests seem to contradict others. The variables people seem to care about (that I can think off of the top of my head) are:

• How many variants there are. We have requests to suppress the Variant Title label if there is only 1 variant and that variant is a pseudonym variant.
• The publication order of the variants. We have a request to NOT suppress the Variant Title label if there is only 1 variant and that variant is a pseudonym variant, BUT it was published at a later date.
• The number of author variations. We also have a request to NOT suppress the Variant Title label if the publications were all performed under a pseudonym, but never under the canonical name.

I'd like to see all of the possible cases thought out in advance, or otherwise we're just hacking on hacks to get the display to work the way everyone wants it, and we'll wind up with unstable code which no one can understand as to why it's so complicated. Alvonruff 21:28, 16 Apr 2007 (CDT)

Sounds lile a reasonable approach! I'll try to remember to post about it on one of the Community forums tonight, but I am a little short on time today. Ahasuerus 17:14, 17 Apr 2007 (CDT)

Strange duplicate submissions

Just a heads up that there have been reports of multiple duplicate submissions the last few days. Here is the latest report. Ahasuerus 21:47, 16 Apr 2007 (CDT)

Adding author gender information to authors table (and web) ?

As far as I know there is no gender (male/female) informations about authors in isfdb which I think is a little annoying. I would propose to change that my adding a nullable gender column to isfdb's author table.

I have noticed public lists like this one [4] and I also think I saw somewhere more general public lists of commen male and female names. With these kind of background data, I think I could make a good computerized estimater that could be used initially to fill out the blanks... I am willing to do a first stap at calculating the estimated genender codes along with a change script for the database if someone else will do the GUI and apply it to the isfsdb install (i.e. I can do the data gathering + database stuff but not the python web stuff). Interested? mortench 12:46, 19 Apr 2007 (CDT)

I have copied this to the feature request list plus started Feature:90145_Add_Author_tags. Marc Kupper (talk) 20:03, 2 May 2007 (CDT)

Problems installing isfdb2 locally

I've been taking a stab at installing the isfdb code locally so I can hack on it a little. (My that sounds violent.) Anyway, I've run into a few snags so far, mostly easy to fix, but I've run into one larger problem: the distributed Makefile invokes make -f dbase.mk install_htmls but there is no dbase.mk.

If I copy the files into htdocs by hand, though, I get an almost-functional isfdb on my local machine. Wheee! --WimLewis 01:48, 22 Apr 2007 (CDT)

(On a related note, is there a better way for me to send patches than by posting them on the wiki? email maybe? WimLewis 14:27, 22 Apr 2007 (CDT))

13 digit ISBNs

One of our EU moderators is reporting that new (2007) books that he is encountering are no longer carrying the 10 digit ISBN, so he has to manually convert ISBN-13s back to ISBN-10s for data entry purposes. Do you think we could make adding direct support for ISBN-13 data entry a higher priority? Thanks! Ahasuerus 11:04, 24 Apr 2007 (CDT)

I, too, have encountered this with new publications from Baen Books - unlike other publishers, they print only the ISBN-13. (Scott Latham 10:41, 30 Apr 2007 (CDT))

Purging a bad submission from the moderator queue

When you get a chance, could you force this submission to reject? Thanks. Mhhutchins 20:14, 3 May 2007 (CDT)

Done. Seems like we need a tool to allow moderators to reject submissions that doesn't do any XML parsing. Alvonruff 12:03, 4 May 2007 (CDT)

Analog June 1965

I just had a look at this magazine (I don't have many to get through, so thought I'd do mine now for practice before they're all gone!) and I'm in awe at the level of detail you went to! (And I thought Marc Kupper was a bit fanatical about detail... ;-) ) So I hate to niggle, but can you double-check the editorial? My copy has it as Laws (plural) rather than Law (singular) throughout, even in the continuation on p160-162. BLongley 15:49, 9 May 2007 (CDT)

I'm on the road today, but I can check it Saturday, since the 1965 issues are still laying out. Alvonruff 18:47, 9 May 2007 (CDT)

It is indeed "Laws" not "Law". I've fixed it. Alvonruff 17:12, 11 May 2007 (CDT)

Ta Muchly! BLongley 20:08, 12 May 2007 (CDT)

Analog 1965 July & Aug

While your at the above request can you have a look at the 1965 July & Aug Analogs. I would like to know if Poul Andersons Trader Team is spelt "Trader" or "Trade", my pb notes it as "Trade".Thanks:-)Kraang 21:35, 9 May 2007 (CDT)

It is "Trader". In the July issue it is listed as such on the cover, ToC, and on the story page; in the August issue it is listed as such on the ToC and the story page. I didn't find any errant spellings within either issue. Alvonruff 17:17, 11 May 2007 (CDT)

Thanks that helps :)Kraang 20:13, 11 May 2007 (CDT)

Invalid submission - 589425

Can you please delete submission 589425 [5]? I suspect a title in the contents was merged before this update was applied and it errors out with no hold/reject available. Thank you Marc Kupper (talk) 22:05, 17 May 2007 (CDT)

Place to post covert art acknowledgements?

Al, we have secured Phil Stephensen-Payne's permission to link to the images that he hosts. He responded with "Sure, no problem, as long as there is a note somewhere (not on every image!) and a link to my site". I suppose we could have it mentioned somewhere in the Wiki, but it seems to be too obscure. Do you think we should start a new page with a list of cover art acknowledgments similar to the page that you have set up for the art used on the main ISFDB page? Or perhaps we could combine the two under a slightly different title? Ahasuerus 00:47, 25 May 2007 (CDT)

It's actually no problem to generate a special link under each image (we already do that for Visco images), if you have the base URL that images will use. Otherwise an acknowledgments page seems fine. Alvonruff 20:39, 26 May 2007 (CDT)

As far as I can tell, all of his image URLs start with www.philsp.com/data/images , e.g. this image. Also, we are having a discussion of Publisher case sensitivity over on User_talk:Mhhutchins. Could you please weigh in when you have a chance? Thanks! Ahasuerus 21:46, 26 May 2007 (CDT)

I've also accumulated a few permissions, see van Vogt bibliographic notes. Marc did even better and got permission for Fantastic Fiction. He even seems to have got the code ready for you: I don't know how you want the permission-grantings recorded officially though? BLongley 16:10, 7 Jun 2007 (CDT)

Bouncing the servers and Series Bug 30016

Al, we have been having serious response problems since Friday. Do you think we could ask the server folks to bounce the database server and the Web servers or at least try to diagnose what's going on when they come back on Tuesday?

Also, when you get a chance, could you please confirm that my understanding of the newly created Open Series Bug 30016 is correct? It's not urgent, but it would be nice to have an accurate statement of the problem before the underlying data changes. Thanks! Ahasuerus 22:09, 28 May 2007 (CDT)

P.S. It looks like the problem was, as suspected, with the sorting algorithm, and our clever editors have found a workaround, so this is no longer a priority. Ahasuerus 12:45, 30 May 2007 (CDT)

Wiki Blocking software acting up again

The Wiki blocking module went berserk earlier today blocking a bunch of users as discussed here. I have undone the 5 day old block of a link spammer that seems to be the cause of this problem, but I am not sure it has completely solved the problem. Ahasuerus 12:45, 30 May 2007 (CDT)

Everything seems to be back to normal for now, no need for Sysop intervention. Hopefully they are not making you do anything too unspeakable in the management area! Ahasuerus 20:06, 30 May 2007 (CDT)

Backup and mw_archive

I installed MySql and the public backup file and don't think the mw_archive table needs to be included in the backups. It's a large table that mostly contains link spam. Marc Kupper (talk) 13:39, 7 Jun 2007 (CDT)

Dissembler question

FYI - I posted a question to User_talk:Dissembler#Roleplaying_with_Kids:_Bringing_up_the_Next_Generation_of_Gamers. Marc Kupper (talk) 13:11, 9 Jun 2007 (CDT)

Python question

I have posted another question to Talk:ISFDB_Downloads#Python_and_CGI. TIA. Marc Kupper (talk) 23:28, 10 Jun 2007 (CDT)

When spambots attack

Al, the Wiki has been under spambot attack for the last few days. It started slow, but we are up to 15 spam edits per day now -- see Recent Changes. Would you happen to know of any way to stop the spammers short of disabling automatic user registration? Ahasuerus 19:23, 11 Jun 2007 (CDT)

The only two methods are captchas and moderated registration. Alvonruff 20:17, 11 Jun 2007 (CDT)

Captchas would be nice, but does our version of the Wiki software support them? If not, how much of an obstacle do you think moderated registration would be for our potential editors? Ahasuerus 22:21, 11 Jun 2007 (CDT)

http://www.mediawiki.org/wiki/Extension:ConfirmEdit talks about version 1.5 and 1.6 as being the baseline for Captcha related extensions. We are at 1.4.5 which I don't even see in the download archives. Marc Kupper (talk) 03:37, 12 Jun 2007 (CDT)

Bring this over from the spam discussion on Ahasuerus page. I've just helped setup several space related wikis, http://www.lunarpedia.org/ Lunar wiki, http://www.marspedia.org/ Mars wiki, http://www.exoplatz.org/ general space wiki, http://www.exodictionary.org/ space dictionary. So I have recent experience and would be glad to help. Dana Carson 07:02, 17 Jun 2007 (CDT)

Al - If you are willing to point me to a full db backup I can look into creating scripts to migrate the existing tables to the current Mediawiki structure. I'd also like to collect the MediaWiki related code in common/mediawiki.py rather than having it scattered through ISFDB. Once we are on a current version that we get many options. For example, on MediaWiki commons the default logic is if you have a link to an external site it asks for a captcha. Thus it does not intrude on signing up, posting messages, etc. We could modify the code for that trap to allow for links to the ISFDB database as those are common. Marc Kupper (talk) 17:51, 17 Jun 2007 (CDT)

User page protected

Al - I believe you are being held hostage in another space-time and won't see this right away but as a temporary measure I've protected your user page to keep the vandalbots out as it's one of the frequent targets. Your talk page is still available for editors and other members of the public. Marc Kupper (talk) 02:27, 19 Jun 2007 (CDT)