ISFDB:Proposed Design Changes

This page is intended for consideration of proposed changes to the Design of the ISFDB.

Specifically, at first, for those changes intended to provide what Al has described as "features considered necessary for solid bibliographies".

Much of the initial text here is taken from ISFDB:Community Portal/Archive/Archive11#Intended Order of Work, with editing to combine experts from a discussion into a set of coherent issues. Further discussion will no doubt take place, hopefully leading to a consensus (or a decision from on high) and action. -DES ^Talk 11:07, 29 Jan 2008 (CST)

Roles

This would encompass ideas like Narrator, Illustrator, Editor, Cover Designer etc.

• Some people have suggested that a translator creates a new work (title) that can be published in multiple formats (publications), but that might be debated. Even if a translation is considered to be a new work, we might want to record both the translator and the original author, as both are essential to making the final work what it is, and both are of bibliographic significance. -DES ^Talk 11:22, 29 Jan 2008 (CST)

• This might be implemented by defining a fixed set of roles, and allowing an entry into each. This has the disadvantage that any new roles would need a code change, and that empty entries would need to be stored for most roles in most records. Another way would be to have a varying number of "role" entries in a publication record, just as we have a varying number of review, interview, or content entries. Each role entry would consist of a role type (either freeform, or, perhaps better, picked from a dropdown list like the story length entries) and one or more canonical names for the person/people in that role for that publication. This would be flexible going forward, and might be able to reuse code from the way reviews are implemented now. -DES ^Talk 11:22, 29 Jan 2008 (CST)

See the MARC21 list of "relator codes" for an example of what these roles may look like. Ahasuerus 04:16, 4 July 2009 (UTC)

Interesting list, obviously including many we would not need here. This raises the issue of whether there should be a fixed set of possible roles, or the role name should be freeform text, or pick-from-a-list, but with an 'other' item allowing free-form entry? -DES ^Talk 04:41, 4 July 2009 (UTC)

Printing records

Should we try to split the concept of publication into "edition" and "printing"? This might allow display of only distinct editions/bindings, with printing-level detail only shown if wanted. A problem with this idea is that it is not always clear which printings are actually different in content until we have full data for multiple printings. -DES ^Talk 11:56, 29 Jan 2008 (CST)

Based on

A title (or perhaps a publication) should be able to be "based on" 0 or more other works. A "Based on" record might consist of a link to the work (publication?) based on, and the nature of the relationship (expanded from; includes; expands; abridged from; revised version; experted from; restored) picked from a dropdown list. This would handle fixups, expanded versions, revised versions, screenplay->novel conversions and the like. -DES ^Talk 12:04, 29 Jan 2008 (CST)

This has already been proposed for variant titles, but this proposal is wider, and would extend to a relationship between works of different types, or that are not "the same work" in the way that a variant title normally is.

• The link should be two way. That is, If Work A is "based on" work B, this fact should be shown in the display of work B as well as in that of work A, if possible. -DES ^Talk 12:05, 29 Jan 2008 (CST)

New Fields

Possible new fields to add:

• Printing number (to a publication)
• Physical dimensions (do we care?)
• juvenile title. Possibly just Yes/No, Possibly a recommended age. Currently this is specified by overloading the "length" field, which is probably unwise. Should this be a title-level field, if added?
• Catalog number. Currently this overloads the ISBN number field, and must be placed in the notes when both are present. (Publication level?)
• Publication date (or first printing date) (title level or publication level
• Copyright date?
• Simplified title? (see talk page)
• Date of Variation, for variant titles? (See ISFDB:Community Portal/Archive/Archive10#Variant Titles and Dates for more discussion.)
• Subject (see ISFDB talk:Proposed Design Changes#Subject for discussion)
• Volume Number / issue number (for magazines)

Fields to restrict

Fields that possibly should be restricted to valid entries from a table, rather than being free-form:

• Binding (hb, tp, pb, chapterbook, audio, ebook, perhaps others)

Publishers

We currently do not have the ability to edit publishers (like we do with authors), merge them, have supporting fields like web sites, dates of existence, or related bibliographies. We also don't link imprints to publishers like we do with pseudonyms to authors (or even variant titles).

Update: we now have publisher edit and merge, and supporting fields for Wikipedia page and other websites, and one notes field. Dates of existence, bibliographical notes like company ownership, addresses, distributors, known ISBN ranges, and publisher series are being recorded in the Publisher Wiki pages, and some linking between imprints/publishers/companies has been started there. See here for all Publisher Namespace entries so far.

Regularization of publisher and imprint names has started but lacks direction, with some editors overloading the publisher field with imprint AND publisher info, and others recording the imprints and noting publisher in the wiki. One emerging standard is " / SFBC" after original publisher name to indicate book club editions.

Future directions are unclear: variant publisher names seem unlikely to happen, and a hierarchy of companies, publishers and imprints like we have with series and sub-series seems unlikely in the short term too. Workarounds include Wiki Redirects and Wiki Links. There is a fear of data loss if over-regularization occurs too soon, and possibly cloning the publisher field to an additional new field would allow one to remain for "exactly as stated" (or at least "editors preference") and the other field could be used for grouping into a smaller number of useful publisher-related information. BLongley 19:30, 1 July 2008 (UTC)

Edit History

Allow viewing and (perhaps) reversion of the details of each edit transaction / submisison. Al had announced that this was in process, but not simple. Highly desireable if possible.

Support for grouped works

It happens sometimes that a number of short works with separate titles form a group, with a collective title, that is published as part of a collection or anthology. It would be nice to be able to capture both the group and the individual titles, and show the relationship. One example is the three "Distant Friends" stories in Distant Friends and Others. Another is "Ghost Stories of Chapelizod". This could also be a way to deal with "braided" stories such as those in Busted Flush or Fever Season. Series have been used in such cases, but this is not always a satisfactory solution. -DES ^Talk 06:10, 14 June 2009 (UTC)

See also Rules and standards discussions/Archive/Archive06#Section Titles in Story Titles special case? for detailed documentation of some cases I ran into with one pair of collections. --MartyD 10:30, 14 June 2009 (UTC)

A Preliminary List of Major(ish) Projects

Can we add to this list, or is this what you are proposing as the preliminary list? (Because I really want/need a better implementation of chapbooks and other short works independently published as ebooks and special printed editions.)Kevin 01:30, 14 June 2009 (UTC)

The list is wide open, add away! :) Ahasuerus 04:15, 14 June 2009 (UTC)

Reversing Canonical Name and its Pseudonym

Clearly very useful since it would help eliminate a great deal of manual work, but may be time consuming to implement. We will need to make sure that the option can't be used on house names, collective pseudonyms and such. Ahasuerus 01:06, 14 June 2009 (UTC)

If we implement removal of Pseudonym relationships (below), a reversal becomes just two steps (1. delete; 2 add the other direction) and just a few clicks, and that might be less work and less dangerous. Or we could restrict it to mods like publisher edits, I suppose. -DES ^Talk 01:12, 14 June 2009 (UTC)

What I had in mind was reversing all affected VT relationships, including Series and other frequently "left behind" areas. The process tends to be quite time consuming at the moment. Ahasuerus 01:23, 14 June 2009 (UTC)

I see. that is a bigger kettle of fish. Larger coding, but more payoff. -DES ^Talk 01:27, 14 June 2009 (UTC)

Add the Ability to Merge Contents Titles with Pre-Existing Titles at Submission Time

After selecting Submit on the Add New Collection/Anthology/Magazine/Non-Fiction screen, the editor will see another screen, which will give him the the option to merge Contents Titles with pre-existing Titles when available. It shouldn't be too time-consuming to code the new screen, but the approval screen for this type of submissions could be tricky. Ahasuerus 01:06, 14 June 2009 (UTC)

Sounds like a good idea to me, if it can be implemented without an overly complex interface. For it to work well, i think that links must be available so that the editor (and the approver) can navigate to the records for the works proposed for merger. Often seeing such details from the current merge screes (which do have such links, both when using "Show titles" and when using "Dup candidates") I find important to determine which candidates should be merged. -DES ^Talk 14:06, 16 June 2009 (UTC)

One thing to think about here is the case of a proposed inappropriate merge. Having to redo a rejected submission is not very pleasant, and I think at least some of these cases will be more likely to generate rejections (vs., say, the moderator's letting it through and fixing it afterward). --MartyD 10:46, 3 September 2009 (UTC)

This might be less of a problem when and if FR #2799118 Ability to 'Edit' Proposed submission and FR #2799077 Add new screen 'Review Submission' for Editors are implemented. That would allow a mod to fix or remove the inappropriate merge without needing to force the editor to redo the entire edit. -DES ^Talk 15:36, 3 September 2009 (UTC)

Automatic Deletion of Empty Series

Al tried this at one point and it didn't work well because of embedded Series. For example, if Series B is embedded within Series A and the last entry in Series B is deleted, do we want to delete Series A as well as Series B is Series A has no other data? We may want to consider adding an option to delete Series records instead. Ahasuerus 01:06, 14 June 2009 (UTC)

Yes to either method. Automatic deletion of empty serial trees upon moderator approval of title deletion (or title series change)(Make sure the empty tree check is after the new series gets applied...it might have been moved to the parent series); or Delete this Series from the view of an empty series. Either one is fine with me, but it's pretty obvious which one will be easier to code. Kevin 02:07, 14 June 2009 (UTC)

I think one of the problems was that just because B becomes empty it doesn't mean that it's not a parent of series C, i.e. you have to check the series hierarchy, not just whether there are any titles left in the series. BLongley 11:46, 14 June 2009 (UTC)

I can also see cases in which an editor is juggling series assignment -- particularly within a series hierarchy -- and leaves a given series empty for a moment, but intends to reuse it shortly. And before you "very well an new series of the old name will be created" recall that the wikipedia isfdb series template links to series by record number so such changes would break such links. -DES ^Talk 14:19, 14 June 2009 (UTC)

It sounds like the least controversial and the least time consuming fix would be to add a new option to the Series-specific Navbar which you see when you pull up a Series. The new option will let you delete a Series if and only if it has no Titles in it and if it has no "child" Series. If there are no objections, I will create a Feature Request tonight. Ahasuerus 16:44, 16 June 2009 (UTC)

Feature request 2809640, "Allow empty Series deletion", has been created. The text reads:

Add a new button on the main Series screen, "Delete This Empty Series". The button will only be displayed if the Series has no Title record associated with it and has no "child" Series. Add an extra check to the Moderator approval screen to avoid creating orphan pointers when submissions are approved out of order.

Implementation details:

When the user clicks this button, a regular SeriesUpdate submission with a blank series name will be created. Since Feature 2807944, which needs to be implemented first, will prevent users from blanking series names, this will allow us to re-use "SeriesUpdate". If we don't do it this way, we will have to create "SeriesDelete", which will require more extensive modifications to the main Submissions screen and probably the Web API.

Anything else that may be missing? Ahasuerus 00:58, 21 June 2009 (UTC)

Sounds good to me, nothing else I can think of. -DES ^Talk 01:35, 21 June 2009 (UTC)

I deleted a series name this morning and left a note on Bills page. Here's the one that's gone "24734 Rod McBan" renamed "Formerly Rod McBan - to be reused". There's only one Rob McBan now this record "24733 Rod McBan" (current). If I send in a blank name now it deletes the name. This didn't work in the past, so why does it work now?Kraang 01:18, 21 June 2009 (UTC)

You didn't delete it, you just renamed it to blank. 24734 shows that it still exists and should probably be renamed "To be reused" or some such. Renaming to blank is not a good idea, it makes dangling series harder to find. -DES ^Talk 01:33, 21 June 2009 (UTC)

Yup, unfortunately, it's not gone, it's just that its title has been set to an empty string. Here is what happens when I run this query internally:

select * from series where series_id = 24734;
+-----------+--------------+---------------+-------------+
| series_id | series_title | series_parent | series_type |
+-----------+--------------+---------------+-------------+
|     24734 |              |          NULL |        NULL |
+-----------+--------------+---------------+-------------+

Ahasuerus 01:43, 21 June 2009 (UTC)

That's interesting because I couldn't do that before.Kraang 01:41, 21 June 2009 (UTC)

We'll have to take a look at the history of the Series page and see if anything has changed... Ahasuerus 01:43, 21 June 2009 (UTC)

(unindented)I thought it best to leave it at the one I did and find out if this causes any problems and why it can be done now and not in the past. No changes to series that I read would lead me to think this was possible or intended.Kraang 01:52, 21 June 2009 (UTC)

In another discussion, Bill (I think it was him) reported cleaning up some 20 such series not long ago, so someone has been able to do this for a while. In any case Feature 2807944 (Prevent series name blanking) will prevent any more such when/if it is implemented. -DES ^Talk 01:48, 21 June 2009 (UTC)

Ah yes, I remember it now. I looked and found 6 2008-2009 submissions by ChrisJ and one Bob Hall that did that, so it's been possible for some time. Ahasuerus 01:50, 21 June 2009 (UTC)

See your comments here also. -DES ^Talk 01:52, 21 June 2009 (UTC)

Implement Chapbook Title Type

Bring back, recreate, or undo the removal of support for Chapbooks (Or make another title type) that can serve as a 'short length title' container for all types of contents, and that can be edited without reverting to anthology. This is needed to document the explosion of short fiction being released as individual works on the web (downloadable), and as chapbooks by specialty presses these days. This will include display support on Author biblio pages, and series support, etc. Kevin 05:18, 14 June 2009 (UTC)

I fully agree. Such a type should be a true container type, much like an anthology or collection. This means there must be both a title type and a matching publication type (some such works may have multiple publications.) It should be used for publications smaller than a usual book, and for publications of any physical size that contain only a single work of fiction (with possibly essay and interior art contents) that is shorter than novel length. Such works should appear in a separate section of an author's summary biblio page, just as collections and serials do.

As to nomenclature, i favor SHORT WORK, but could live with CHAPBOOK, or some other term. Please don't leave it as CHAPTERBOK. -DES ^Talk 05:36, 14 June 2009 (UTC)

SHORT WORK, works for me. Kevin 16:17, 14 June 2009 (UTC)

I prefer SHORT BOOK, unless you are proposing that these are for single-content works only? (Which seems to be how some people are misusing the current broken situation.) As I've seen anthology and collection chapterbooks here, but we can make those real collections and anthologies and record the fact that they have been labelled chapterbooks by the publisher in notes. How far do people want to go to eliminate the phrase CHAPTERBOOK? BLongley 17:40, 14 June 2009 (UTC)

I think there are two different things that usually but not always go together. The standalone publishing of a short story or two. The type of physical container. We distinguish between mass market paperbacks, trade paperbacks, hardbacks and magazines. A way to distinguish chapbooks where the book is printed and folded with-out a separate bound on cover is useful. Omnibus has a slight problem with this also since they can contain a mix of novels and collections/anthologies. Dana Carson 21:33, 14 June 2009 (UTC)

I thought that was what 'ph' Pamphlet binding was for? Kevin 21:36, 14 June 2009 (UTC)

Not Single Content - Multiple Content items, as DES described. As for CHAPTERBOOK, that is DES's windmill, not mine. Kevin 21:35, 14 June 2009 (UTC)

I think the most frequent use in practice would be for publications having only a single content item. But accompanying essays, interiorart and other non-fiction contents would be far from unheard of, i would expect. I could see a convention (which i don't think needs to be enforced in software) that this type be limited to items containing only a single work of fiction. If there are multiple fiction works, then the anthology or collection type can be used, perhaps with a binding of ph. -DES ^Talk 18:57, 15 June 2009 (UTC)

As to the name, i think everyone who has commented on the matter agrees that the name "Chapterbook" was a mistake and should be changed. As long as we are making code changes affecting this type (assuming that we are) that seems the moment for correcting that mistake and/or choosing a better name. "Chapbook" at least arguably describes a publication/binding format, while the main use of this type would be to describe the nature of the contents (a basically stand-alone publication of a work of short fiction, possibly with essays or other non-fiction). Thus a name other than "CHAPBOOK" seems like a good idea to me. -DES ^Talk 18:57, 15 June 2009 (UTC)

While such a type could be used for a small/short publication, bound as a pamphlet or chapbook in the traditional sense, containing multiple works of short fiction, I would not be unhappy if we agreed not to use it in that way. Our existing types, with a binding code of "ph", would hadle that case well enough, IMO. -DES ^Talk 18:57, 15 June 2009 (UTC)

I can think of a few permutations where having a Chapbook (or whatever we decide to call it) container may be useful:

• 1 work of fiction + 0-n Essays + 0-n Interior art
• 1 essay + 0-n Interior art

On the other hand, I think that 2+ pieces of fiction would be probably better displayed as Collections even the total page count is low. What about a 25 page publication collecting 32-4 Essays, though? Is it Non-fiction or Chapbook? Ahasuerus 19:56, 15 June 2009 (UTC)

I agree with your two cases, and i incline to think the 25 page book with 32 essays is best recorded as simply Non-fiction, possibly with a "ph" binding if appropriate. But that would be something of a judgment call, IMO. -DES ^Talk 20:11, 15 June 2009 (UTC)

Sorry, that was supposed to be "2-4" Essays :-( Ahasuerus 20:27, 15 June 2009 (UTC)

I see. Well the same answer, I would prefer non-fiction for such a work. -DES ^Talk 20:38, 15 June 2009 (UTC)

Instead of trying to imagine all the permutations (how many poems for example) - perhaps we can put a general page count. I believe I have seen <100 pages to be defined in some places as a chapbook length publication (This is also probably related to the definition of 100 pages = a novel). I imagine there will be things above 100 pages (A singe previously published short work plus art plus intro for example) that we want to call chapbooks instead of novels, but <100 pages for multiple items seems like a 'fine' objective line in the blowing sand for multiple item publications. This definition could be applied for 'any' contents where the page count is less than 100 pages. Kevin 23:52, 15 June 2009 (UTC)

Well, the main reason that the Chapbook Title type has been so frequently requested is to prevent works of short fiction that have been published in standalone format from getting "lost" in the sea of short fiction at the bottom of the Summary screen. This is what many encyclopedias do (e.g. Clute) and what many ISFDB editors have been advocating. However, if a chapbook contains multiple stories and appears in the Collections section -- or contains multiple Essays and appears in the Nonfiction section -- then it's already visible as a "book" and the same concern doesn't apply. By putting collections with <100 pages in a separate section we would arguably make them harder to find. Perhaps we could even consider listing single-Essay chapbooks as "Nonfiction", which would mean having two Titles per pub: one Essay and one Nonfiction. Ahasuerus 00:32, 16 June 2009 (UTC)

(Also it should be used when the publication calls itself a chapbook - just as we use Novel for things that call themselves novels) Kevin 23:52, 15 June 2009 (UTC)

Oh no, we do not rely on what the publisher claims because their claims can be wildly off the mark. As Help:Screen:NewPub says, "Note that frequently a magazine will describe a story as a complete novel, even though it may be substantially below the 40,000 word mark. The description in the magazine should not be relied upon for this distinction." Ahasuerus 00:32, 16 June 2009 (UTC)

I agree with Ahasuerus, we don't and shouldn't go by what a publication calls itself, or what the author or publisher calls it, because their usage is way too inconsistent. The kind of publication I think we really need a "chapbook-like" type for is: publications with exactly one fiction contents item, shorter than novel length, with 0 or more non-fiction contents items, such as essays, introductions, cover art, interior art, poems, etc, etc. Such a work isn't a NOVEL, because the work of fiction is shorter than novel length. It isn't a collection or an anthology (although it is functionally similar) because those both normally include multiple fiction works. It isn't an omnibus, for similar reasons. It is desirable that such works appear in their own section of an author's biblio page, distinct from both novels and shortfiction, because they do exist as separate publications, but are not really novels. The only case i can see these being useful for beyond "exactly one work of fiction" would be to handle ACE Doubles and similar publications (e.g. TOR Doubles) when they are made up of exactly two shorter than novel-length works of fiction. And most people seem fairly content with the current handling of ACE and TOR doubles. -DES ^Talk 02:02, 16 June 2009 (UTC)

UNINDENT - I'm almost in agreement with both of you. My take: Max 1 item of Prose. 1 Item of Prose, or 1 Item of Fictional Essay, or N items of Poetry required. Unlimited Fictional Essays, Poems and all other title types. If Prose/Fictional Essay or Poetry absent, then it's a Nonfiction container pub and so be it. If it has Prose greater than 1 (excluding fictional essay) then it's an Anthology/Collection container pub. - I'm quibbling over the Fictional Essay now before we create it because it will probably come up later. I'm quibbling over poetry because I moderated a 2 poem, 2 essay chapbook the other day and it felt right at 25 pages or so. Kevin 03:26, 16 June 2009 (UTC)

I could also easily be talked into 2 being the max number for prose, with an upper page limit Y (Maybe 50?) that would exclude all the Ace doubles, etc. I could also be talked into an exemption for Any publication with fewer than Z (Maybe 25?) pages being a chapbook no matter what the contents. Kevin 03:26, 16 June 2009 (UTC)

Clarification - The upper limit Y I mentioned was only for when two non-essay fiction titles were present, in order to exclude the Doubles. I think I wasn't clear. Kevin 22:07, 16 June 2009 (UTC)

As to relying on the publisher... the concern is generally cases of 'inflation' where the publisher calls it something bigger than it actually is. I don't think Novels hiding as chapbooks will be a very common occurrence. Kevin 03:26, 16 June 2009 (UTC)

I assume that by "prose" you mean non-verse fiction, as technically an essay is prose. I can live with your take above, except thst zi don't see the need for the upper page limit -- pagee sizes vary soo much, and I could see s profusely illustrated 20,000 word story on small pages being 80 pages long or more. And I don't think your "any pub smaller than 25" pages clause is needed, but i won't argue the matter further. -DES ^Talk 12:52, 16 June 2009 (UTC)

I'm glad agreement is close: what you're talking about now won't affect the coding, only how to use it. Binding is a red herring - it won't affect the container type. So if we agree a SHORTWORK is a container type like an ANTHOLOGY or COLLECTION or NONFICTION or OMNIBUS, then we just have to make sure that when Publication Type and Title Type are both SHORTWORK then we treat it the same way as those: hide the content entry for such. We can reuse the CHAPTERBOOK code that makes such appear in their own section, above the shortfiction that I hate seeing such lost in. Yes, some books will no doubt get downgraded - I guess "The Trouble with Tycho" is a casualty of that, being a standalone novella, but it's not as bad as not finding that it was published as a standalone book. BLongley 20:18, 16 June 2009 (UTC)

Again, we can break this down into smaller steps:

• I think it's only a one-line change to stop CHAPTERBOOKS breaking as soon as you edit them (add CHAPTERBOOK to the list of content title type options, rather than forcing a change to ANTHOLOGY by default).
• Hiding the CHAPTERBOOK content so that people realise that it's just the container type, NOT the shortfiction/essay, is a little more work.
• Being able to ADD the container type to existing broken publications a bit more work still. (Think of all the Magazines with missing EDITOR records - we're working on those, and will have to do the same with these too. A fix script should be possible though.)
• (Optional step) Change all display logic to show "SHORTWORK" (or SHORTBOOK, or whatever people want) rather than "CHAPTERBOOK". (Would demand changes to every place the term is used, but saves us making a big database update.)
• Finally - eradicating the phrase "CHAPTERBOOK" entirely in favour of SHORTWORK (or SHORTBOOK, or whatever people want) will demand changes to every place the term is used, and a mass database update. BLongley 20:18, 16 June 2009 (UTC)

I'd prefer to see small incremental changes, but appreciate that people would see more "CHAPTERBOOK" entries appear in the meantime: but it allows people to demonstrate what they will use them for in the future before we do anything dangerous. Nothing in this set of coding proposals enforces any rules on what SHORTWORK/BOOK will finally be used for, any more than we enforce the number of NOVELs in an OMNIBUS currently, or SHORTFICTION works in a COLLECTION or ANTHOLOGY. BLongley 20:18, 16 June 2009 (UTC)

That sounds fine to me, I agree that usage of a CHAPTERBOOK/CHAPBOOK/SHORTWORK/SHORTBOOK type is a rules-and-standards issue, and that there is no need to enforce this via code, just as most of our current conventions are not so enforced. (Some idea of how we would use it probably affects what we call it, and maybe any special coding wanted, however, so it was worth discussing here a bit, IMO.) Changing the wiki help will be easy. Is a mass database update really needed? Could the display code be set to "translate" the CHAPTERBOOK type stored internally to whatever name we want, so that only raw SQL or Data dumps would see it? -DES ^Talk 20:33, 16 June 2009 (UTC)

We could stop at step 4 and leave the software to translate everything, but that still leaves confusion for developers and people using the raw data. I'd prefer we eradicated the term entirely, to avoid potential future confusion for people that arrive later and don't know the sordid history. And to avoid translation on every screen where it might be displayed. BLongley 19:43, 18 June 2009 (UTC)

I agree tht would be ideal, unless a mass data change is seen as too risky or onerous. -DES ^Talk 20:10, 18 June 2009 (UTC)

Implement new short item Title Types

Big Picture

Before we spend a great deal of time discussing the new sub-types, we may want to decide just how far (or "deep") we want to go with this. Limiting the number of title types and "storylen" selections to a relatively small set was a conscious decision since we didn't like what we saw in Contento's list of abbreviations. This decision was made very early in the project and it affected everything else that was done later, so undoing it now may not be easy.

What we may want to do first is outline what's wrong with the current approach at the functional level, e.g. the fact that there is no dropdown list in the storylen field or that it's overloaded with "jvn" and "nvz", and then see how we can address these issues conceptually. There will likely be certain trade-offs involved if we add more subtypes, e.g. it will make Publication and Title pages more clear, but it may also make it harder to run some searches. Ahasuerus 16:08, 14 June 2009 (UTC)

I think that we (as a community) have gotten particular (picky?) enough that there is now often a desire to go to that further level of detail. We also have enough data, that the rare instances of particular types, appear in the tens, if not hundreds within the database, or in the type of material now being entered/produced. In using Contento and Locus as a secondary source, we have been exposed to the benefits of having that extra detail as well. Kevin 16:16, 14 June 2009 (UTC)

I think some people have got picky and want to go to a further level of detail - e.g. magazines are now being entered to a level of detail I have no interest in, and this discourages me from working on them. But I won't stop them, as their activity isn't harming anyone else. Some of the suggestions below though are things that mean massive amount of rework - OK, there's only 200 or so maps but about 5000 Introductions, about 1000 Afterwords and 1000 Forewords. (And 20 "Foreward"s I'm suspicious of.) In most cases I can't see the benefits outweighing the rework. But I've tried to respond on each category. BLongley 20:53, 14 June 2009 (UTC)

There's closer to 750 Map titles when you include 4 variations: "Map (", "Maps (", "(map)", "(maps)" Kevin 21:27, 14 June 2009 (UTC)

Yeah, I just filed a bug report on that. :-( Title includes "MAP", type is "INTERIORART" works for two pages of results then forgets the title on page 3. BLongley 22:09, 14 June 2009 (UTC)

I am not sure we want to go to that level of detail either, but I'll have to think about it some more. BTW, it so happens that I verified a "Foreward"s just the other day. Who needs those pesky spelling rules anyway?.. Ahasuerus 21:30, 14 June 2009 (UTC)

I at least would like a somewhat greater level of detail than our current structure supports well, but perhaps not as detailed as some of the proposals below get. It seems to me that these suggested new title types mostly fall into two categories:

1. tools/types to enable capture of data not easily captured by our existing system, requiring awkward one-off kludges or not being recorded at all. These include, IMO:
   • Section titles
   • Group titles
   • Facetious/Fictional Essays
   • Non-Genre Shortfiction
2. tools/types to make finer distinctions between things we already are or could be capturing adequately if perhaps not optimally. These include, IMO:
   • Lists
   • Maps
   • Script
   • Play
   • Diagram
   • Audio Reading
   • Lyric
   • Frontispiece
   • Introduction
   • Afterword

It seems to me that the first group is more important to implement. Of the second group, it depends on just where it is judged worth making finer distinctions. (I would implement some of these but not all, if it were my sole decision, which of course it isn't.) Also, items in the second group would possibly require revising existing data is the db is not to be inconsistent in its handling of such things, as existing entries for many of them have been recorded under one or another of the existing types. That is mostly not true for the first group with the exception of Fictional Essays, where I think the change would be worth it.

I think the initial design decision Ahasuerus speaks of is now worth reconsidering, at least to some extent, but i also think we do not want to approach the number of types and subtleties of distinction that Contento supports. -DES ^Talk 19:17, 15 June 2009 (UTC)

One thing to consider is that we already have two hierarchical levels of Title Types. The first one is NOVEL, SHORTFICTION, ESSAY, etc and the second one is Novella/Novelette/Short Story. At the moment, only Short Fiction Titles can have "sub-types", but we could conceivably create new sub-types for some of the proposed Types. For example, "Frontispiece", "Maps" and "Diagrams" would become sub-types of INTERIORART, "Lists", "Introduction", "Afterword" (and perhaps "Fictional Essays") will become subtypes of ESSAY, etc. This would require the use of JavaScript, but we already require it during editing (but not while browsing) if you want to be able to add Authors and Titles. I suspect that this approach would make "title type" herding much easier and alleviate many concerns associated with the potential proliferation of "first level" Title Types. Ahasuerus 00:43, 16 June 2009 (UTC)

I gues that I have never really considered Novella/Novelette/Short Story to be types at all, merely values of the length property. But we could also use the "storylen" field, or create a new similar "sub-type" field, to make these distinctions. Just as the display code now shows the story length on some screens, the code could display such sub-types in proper cases. Sextion titles, group titles, and non-linking reviews could not be handled in this way, i would think. -DES ^Talk 01:51, 16 June 2009 (UTC)

Sub types is fine with me, it's both a database design choice and a coding choice. In fact it will probably remove some objection to the various types proposed IF we can code the drop down list to only present the proper options for each Top Type. Kevin 03:39, 16 June 2009 (UTC)

Conclusion?

Copied from #Non-Genre Shortfiction below, because it really applies to all types and so i wanted to answer it here. -DES ^Talk 20:59, 18 June 2009 (UTC)

Given the number of new types under discussion, it's probably wise to settle the ones that we think will work and which won't, and make one big change to the software to allow the ones we want. I think the mandatory database change is small - just add some more options to the enumerated types. (Unless people are expecting the introduction of such to magically change all current worked-around entries, and I don't think that's going to happen. People will have to fix things record by record.) What we will have to do is make sure the new types appear in all the right places on our current screens, or define where they should appear - e.g it should be fairly simple to include Non-Genre Shortfiction with Genre Shortfiction everywhere, but as the point is to clarify the difference it's probably wise to consider where such should appear. (IMO, somewhere near the bottom, but that could be at the bottom of Shortfiction or at the bottom of the page, and I guess there's always the possibility that Genre and Non-Genre Shortfiction end up in the same series.) BLongley 20:37, 18 June 2009 (UTC) End text copied from #Non-Genre Shortfiction below -DES ^Talk 20:59, 18 June 2009 (UTC)

And weren't you the person arguing for incremental changes in connection with "based on"? ;) That said you may have a point. Are we ready to make decisions on these yet? Should we set up a list and indicate approval or disapproval on each? Or do we need to discuss either the individual suggestions or the whole matter more first? -DES ^Talk 20:59, 18 June 2009 (UTC)

Some things can be best done incrementally, this doesn't look like it can be broken down easily. I don't think we want to rework all the same pages each time we approve a new title type. I'm not sure we're anywhere near conclusion yet on all types, but a summarisation of the options that have been raised should reveal that some ideas are already dead or withdrawn, some are still under discussion, and some are unresolved and either need more discussion that isn't happening or should be withdrawn. I see no harm in a neutral summary being posted for now: I don't think we're going to see new additions to the list, but maybe we should let people have the weekend to make sure of that. I know the discussion list is already getting too long for my taste, but we should have a possibility of "we'll do this, won't do that, and will leave the other worms till later" soon. Which is sort of an incremental change, as it allows for the possibility of a second set of changes, I'm just a bit sceptical about whether the second will ever happen. But the first should. BLongley 21:26, 18 June 2009 (UTC)

Facetious Essay

For those in-universe info-dumps that are not really factual, and so don't really fit "essay", but have no plot and are not realy stories. Things like the "Interludes" in Hammer's Slammers or the various appendicies in many of David Weber's books. For the matter of that, most of the appendices to LotR would fit this category. -DES ^Talk 05:48, 14 June 2009 (UTC)

Yes Kevin 06:31, 14 June 2009 (UTC)

I'm mildly tempted on this. But "Facetious Essay" makes it sound like one of the "Probability Zero" types to me, whereas "In-Universe Background Data" is the sort of thing I want it for. BLongley 20:03, 14 June 2009 (UTC)

How about 'Fake Essay' or 'Fictional Essay'? Kevin 21:10, 14 June 2009 (UTC)

'Fictional Essay' would suit my purposes. BLongley 21:14, 14 June 2009 (UTC)

'Fictional Essay' would cover many hard-to-categorize cases, so it looks useful. Ahasuerus 22:22, 14 June 2009 (UTC)

I'm fine with that name ("Fictional Essay") for this category. It is perhaps better than "Facetious". -DES ^Talk 13:00, 15 June 2009 (UTC)

I'd prefer "Fictitious" to "Fictional", but otherwise agree. -- Dave (davecat) 20:46, 16 June 2009 (UTC)

I prefer "fictional". Also, not sure if it's been raised but I have several fictional whole books that are aren't novels, e.g. reference books on dragons, travelogues of space trips.) ... clarkmci/--j_clark 08:43, 28 June 2009 (UTC)

Lists

To handle glossaries, Dramatis Personae, and other things that are lists, not really essays. -DES ^Talk 05:48, 14 June 2009 (UTC)

Yes Kevin 06:31, 14 June 2009 (UTC)

Maybe -- Dave (davecat) 20:52, 16 June 2009 (UTC)

Maps

As separate from other interior art. -DES ^Talk 05:48, 14 June 2009 (UTC)

Yes Kevin 06:31, 14 June 2009 (UTC)

Section titles

Sometimes collections and particularly anthologies are divided into sections, with section titles. There is currently no way to capture these titles, but it would be nice to be able to do. Interiorart entries have been abused for this purpose, but not often or consistently. -DES ^Talk 06:14, 14 June 2009 (UTC)

Sure Kevin 06:31, 14 June 2009 (UTC)

Whom do we credit the section titles to? And can we 'Hide' them on the Author's Biblio page if we credit them to the Author? Kevin 06:32, 14 June 2009 (UTC)

In an anthology, to the editor(s) I would say. In a collection, to collection ed if ther is one, or to author. Or better, make them an authorless type that displays only in pub records, never on author biblio pages. -DES ^Talk 06:55, 14 June 2009 (UTC)

I like the authorless type idea. Kevin 07:29, 14 June 2009 (UTC)

Essays and Shortfiction and Serials and Series have been abused for this too. I'm slightly positive towards this one. "Authorless" would be be good for such, but it might be simpler to work around with a convention for now before we go break "titles always have authors" rules. BLongley 20:08, 14 June 2009 (UTC)

Eh, maybe just give them all the author "Title, Section", with code in the contents display to never display Mr. Titles, name, just his work. Kevin 21:09, 14 June 2009 (UTC)

The Locus Index does something similar and we may want to review their approach to see if it will work for us. For example, would you say that the way "P.S.: Ideas, Interviews & Features..." is displayed in Ballard's Cocaine Nights would work? It seems to be lacking something extra that would make it stand out as a section title. I guess we could bold or italicize it at display time as long as we have a separate code for it internally. Also, if we make it author-less, it may necessitate other changes since the software currently requires at least one author for each Title. "section title" as a new reserved Author name in addition to "uncredited" and "anonymous", perhaps? Ahasuerus 22:32, 14 June 2009 (UTC)

How about just Prepending and appending "-----" to whatever the section title name should be? Kevin 23:38, 14 June 2009 (UTC)

Well, as long as it's just a display issue, we can easily change it later. We just need to decide what we need to do at the data level. Ahasuerus 01:44, 15 June 2009 (UTC)

I could live with a convention of entering these with a generic author, which would not be displayed, if that would be technically easier than making them truly "authorless". I like Kevin's idea of displaying these surrounded by "-----", but as you say, the display format can be changed more easily than the db format goign foreward. -DES ^Talk 19:22, 15 June 2009 (UTC)

Script

Script, Teleplay, or Screenplay, or other type of 'story directions for recorded visual medium' Kevin 06:17, 14 June 2009 (UTC)

I think I would combine this with "Play" but not a big deal. -DES ^Talk 06:58, 14 June 2009 (UTC)

Traditionally, Play's have been a more accepted form of fine literature (See Shakespeare), but in SpecFic, the Script adaptation is much more prevalent. By separating the two, those interested in the (percieved?) differences can pick and choose whether to read the Plays of Terry Pratchett, or the Screenplays of Heinlein by type. (Scripts often have much more detailed, set layout descriptions and opening and closing effects etc described for each scene, and are frankly a less 'involving' read, than Plays IMHO). Kevin 16:32, 14 June 2009 (UTC)

I think that is true is some cases, but far from all. Light disposable farces, and scripts for fan dramas presented at SF conventions, are just as much "plays" as Shakespeare is. I am always suspicious of a type distinction largely supported by arguments about quality. For the matter of that, soem stories have been written partly but not entirely in dialog/play format. For example Blish's "More Light") consists largely of the text of a play, with a framing story indicating how the narrator came to read it. -DES ^Talk 19:33, 15 June 2009 (UTC)

I wasn't trying to denigrate Scripts, only to point out that more people are used to reading LIGHTS UP - MAIN CHARACTER IS AT THE CONTROLS OF THE SHIP as opposed to FADE FROM BLACK WITH A DESCENDING ZOOM TO MAIN CHARACTER AT THE CONTROLS OF THE SHIP. (shrug) - Camera direction (if present) I find to be distracting. Some folks may not mind the differences. (shrug). Kevin 00:05, 16 June 2009 (UTC)

I am inclined to think that merely recoding plays and scripts as works of shortfiction (or novels, if long enough) with a note about their dramatic form is enough, but if more distinction is wanted then i think that a single "Dramatic Work" type covering plays, video/audio scripts (when published as such) and any other works published in dramatic form would be sufficient. -DES ^Talk 19:33, 15 June 2009 (UTC)

I Could live with a single type for both. I don't have a significant concern. I just wanted to make sure both options were on the table before a decision was made. (That's how you make good decisions in my book). I could go with 'Script' for both, or 'Dramatic Work' for both. I think 'Play' for both types would be misleading. Kevin 00:05, 16 June 2009 (UTC)

Play

Story directions for a live acted visual presentation. Kevin 06:17, 14 June 2009 (UTC)

I've got by without this and "Script" fine for now. Are people unhappy about the way the Pratchett Plays are entered? I think I got a comment on some of the Quatermass books too, but that's the only way you can read them so they've been left in. BLongley 19:12, 14 June 2009 (UTC)

I'm saying I can't search for a play. I have to search the title for 'play' and sift through the results (If I want to find a play). Kevin 19:42, 14 June 2009 (UTC)

So you CAN search for a play then. Yes, the results will get messed up with things like "The Game-Players of Titan", but that's what advanced search is for (and not the broken bits either). Have we got a load of unemployed thespians searching us daily for new ideas? I suspect not. I haven't even had people ask why the Plays aren't variants of the main titles. (Which I'm beginning to lean towards myself, actually.) BLongley 19:59, 14 June 2009 (UTC)

Thats a circular argument. By that standard, all title specific information should be overloaded into the title field, and we don't need any Title Types. You CAN find everything... you just have to work at it. Kevin 02:47, 15 June 2009 (UTC)

No, I'm just saying that the "- The Play" suffix convention is currently working. I wouldn't use that to justify removing all title types in favour of "- a collection", "- a novel", "- an omnibus" suffixes with tens of thousands of edits to get to that state. It's just that as we have a workaround already I can't see the need to change. "It ain't hardly broke, so don't hardly fix it". I'm sure there are hidden plays still, but I haven't seen anyone asking about how to find them in the years I've been here. Not worth the effort, IMO. It's not even been important enough to really consider whether the Plays I do know about should go under the main title or be variants of them or totally separate as now. BLongley 17:33, 15 June 2009 (UTC)

If we had a standard of '- The Play' then I might agree that it isn't broken. But the good news is after checking my local database I can now agree that two title types for plays/scripts is overkill. I found less than 200 titles for "the play", "(play)", " script", "teleplay", and "screenplay", Kevin 00:16, 16 June 2009 (UTC)

Well, '- The Play' wasn't my standard, fortunately PTerry or publisher seems to have adopted that anyway. BLongley 19:34, 16 June 2009 (UTC)

Diagram

Technical type Interior Art (Ship Schematics, Timeline of Events, Family Tree of characters) - Any visual presentation (other than map) where there is information in the artwork, as opposed to artwork for, err well artworks sake. Kevin 06:17, 14 June 2009 (UTC)

If this is desired, I think it could cover Maps too. (Even though I do find the presence of a Map in modern Pub contents as a useful warning sign that I'm probably not going to like the book.) BLongley 19:21, 14 June 2009 (UTC)

That would be possible, but I suspect that maps far outnumber all other diagrams, and may be worth marking as such. But I am not wedded to either "map" or "diagram". -DES ^Talk 19:34, 15 June 2009 (UTC)

I agree maps far outweigh other 'specific' interior art types. My primary concern in addressing this was to see if there was support for Family Trees , Time-lines, and Organizational Charts, that now often get a mishmash of 'essay' or 'interior art' designations depending on the editor. I'm fine leaving ships schematics etc as interior art. If we do not implement this type, we should address the help to try and steer them towards one or the other for those sub-items. Kevin 00:24, 16 June 2009 (UTC)

Audio Reading

Podcast, download, etc. A short work, read to the listener, by a single reader (or a group of readers who do not alternate readers based upon the dialog or actions in the story). Kevin 06:17, 14 June 2009 (UTC)

I don't think this is a type of title. A give work, printed, read aloud, and dramatized, should none-the-less have only a single title record IMO, and we already distinguish audio books by binding. -DES ^Talk 07:06, 14 June 2009 (UTC)

I can't distinguish with Binding when it's included in a publication already. See Subterranean Online, Spring 2007 and Subterranean Online, Summer 2007. I also imagined that it would be a variant title of the printed published title. Kevin 07:28, 14 June 2009 (UTC)

I hadn't thought of that situation, and it makes a bit more sense now. Still I think a note might handle this rare case. -DES ^Talk 13:32, 14 June 2009 (UTC)

About half the Subterraneans online that I haven't entered yet have similar audio contents. I believe Clarkesworld also always issues a podcast of one of its stories. It's very likely to be something more and more of the better webzines will do. Also some ISBNs in the near future may include the printed word, the ISBN, and the audio at a single bundled price (Or more likely the ebook and the audio download, skipping the print). Several of the early Baen CD Bind in Extra's included the audiobook as part of the same purchase. Things are being 'bound' together today in ways that were impossible before (Which coincidentally I think I recall also helped give rise to the modern novel... it could be bound as a single volume inexpensively) Kevin 02:50, 16 June 2009 (UTC)

Hehehe - First we had words, then people started putting pictures with them, and now they are including sound - I'm just poking fun, but what will we do when they sell a single Blu-Ray disc with ebook, audio book, audio dramatization, graphic novel (ebook) and the feature film all together? Maybe we can call the film Interior art? - Chuckle - Maybe we can talk the IMDB into linking back to US for a change?) Kevin 02:50, 16 June 2009 (UTC)

I'd be against doing this as a title type. Possibly we need to do something more with pubs to allow for it. But the title as such should refer to the words, whether printed or read. -- Dave (davecat) 21:15, 16 June 2009 (UTC)

Audio Dramatization

Podcast, download, etc. A short work, read to the listener, by a cast of readers / audio actors (who alternate readers based upon the dialog or actions in the story), or that includes sound effects (other than simple opening music). Kevin 06:17, 14 June 2009 (UTC)

I don't think this is a type of title. A give work, printed, read aloud, and drqamatized, should none-the-less have only a single title record IMO, and we already distinguis audio books by binding. Moreover I think true dramatizatiosn should be OUT of scope altogether, just as video dramatizastions are -- ther are too many data elements that woukld nee to be captured to so such justice (cast by role, director, producer, musicians, sound effects, author of script, author of work from which script was adapted, etc). -DES ^Talk 13:30, 14 June 2009 (UTC)

Any rule which excludes The Sagan Diary by John Scalzi (Read by Elizabeth Bear, Mary Robinette Kowal, Ellen Kushner, Karen Meisner, Cherie Priest and Helen Smith) is a silly rule. But the distinction between a single reader, and multiple readers, and again the distinction between a simple narration, and one that includes radio style sound effects is worthy of documenting, as evidenced by your reaction to the same. To each their own I say, and why don't we catalog to a level of detail that allows each to find their own. I tried to draw the line between the two title types at the happy medium. Single narrator only (for the purists like yourself), and all other type (for those wanting something different). Would you really seek to exclude this gem of Tolkien's Hobbit and LotR. Kevin 16:24, 14 June 2009 (UTC)

One could equally argue that any rule that excludes Forbidden Planet is silly. But we exclude all videos, of what ever quality and significance. Similarly i would exclude all dramatizations (at least ones with multiple readers) and would think they could be deleted on sight, just as RPG modules are. We have Scope and RoA rules for a reason, and it is not appropriate to have things be IN or OUT depending on who is doing the entry or moderation. (of course no one has to enter things they don't choose to enter.) That said the example (The Sagan Diary) you cite I might well call an audio-book and count as IN. I would exclude the Tolkien, although I would love to have a copy, and have entered some JRRT audio books in the past. -DES ^Talk 16:38, 14 June 2009 (UTC)

"Forbidden Planet" is in, but only the novelization. If we have a valid dead-tree version of a work I don't mind an audio version of it. But that would be for comparable versions - e.g. it doesn't justify the soundtrack of the film. The number of performers doesn't matter to me: a relationship to the written work does. So I can cope with "Unabridged" and "Abridged" audiobooks, but not "a dramatization for radio" that bears no relationship to a particular book or story. Some Doctor Who "audiobooks" probably cross that line for me already. BLongley 19:34, 14 June 2009 (UTC)

I think this type is about "dramatizations". And that is another reason for them to be OUT, they are often sufficiently different from the underlying printed work that they would need to be separate titles, not merely variants of or publications of the underlying written story. Dramatizations often (although not always) elide descriptive text or move it into dialog, for obvious reasons. In some cases much more rewriting than that is done. -DES ^Talk 19:37, 15 June 2009 (UTC)

The problem is drawing the line. Straight read = in. Multiple readers (by Chapter) or 'Voices' plus narration true to the text = in (Put here or under Reading Above). Straight Read / Alternating voices + 'some' limited music = in (Put here or under Reading above). Read or voices + some sound effects = Limbo. Partial Rewrite (for length, or for effect) = Limbo. Complete re-write = out. I pointed out all these combinations because I think that Multiple 'readers' is in because it's the same text, and many audio books supplement with mild music or rare sound effects. Couldn't any 'abridged' audio version have just as much anonymous changes as your average radio dramatization? I fully agree that complete new 're-imaginings' of a story in Audio are out. My problem is that I think you have drawn the line so close, that half the audio books out there are now 'out', but we can't tell they are out unless we listen to them... so keep some sanity by moving the line to where we can identify it without doing a minute comparison of the work, but this moves enough 'dramatized' things (Some music, some sound effects, some re-write/abridgment for time/format) 'into' the database, that now we need to separate 'Single Reader' from 'Combinations of Readers / sound effects / but not a re-imagining' in order to document that some are more pure than others, and Cast of 1 vs cast of 1+N is easy to identify without listening to the complete work. (PS This line I'm moving, is an example and not my line. If I was king for a day, almost everything published without moving pictures would be 'in' in one shape or form, but I think you guys know that by now.)Kevin 04:04, 16 June 2009 (UTC)

On audio, I tend to have an "exclusionist bias" and if you reallu want a bright-line rule, I'd be willing to draw it at "single reader" -- anything with multiple readers is out -- although I think that is probably too conservative. But I think the two of us have probabvly hashed though our views prett throughly, and i at elast intend to wait for the input of others on this type. -DES ^Talk 12:36, 16 June 2009 (UTC)

Lyric

Spoken words of a poem or song, intended to be accompanied by music (Music not required in notation, score, or audio track form). Kevin 06:17, 14 June 2009 (UTC)

Is there really enough difference from "Poem" here to make the distinction? A good many of Kipling's poems, for example, seem to have been intended to be sung, and I have heard them performed, set to popular (music-hall) tunes of his day. Other examples could be cited. -DES ^Talk 13:55, 14 June 2009 (UTC)

I can't see a lot of need for this, and it doesn't cover the situation where the musical notation is included. BLongley 19:09, 14 June 2009 (UTC)

Actually I purposely defined it ("Music not required...") to apply in those cases where music is present. I figured with all the SpecFic exclusionism (Too many pictures.. Out. Too many voices... Out. Too many sounds... Out. Too many musical notations... Out.) going on around here, the title type of 'Score' suggestion might get me in hot water. Kevin 20:43, 14 June 2009 (UTC)

I don't think I'm in favour of "out" on any of those grounds. I just don't see a distinction between our current use of POEM and this that couldn't also be used to demand a separation for POEMs that do have musical notation. (All those filkers that started in Merovingen Nights, for instance.) BLongley 21:18, 14 June 2009 (UTC)

I agree with Bill here. i wouldn't exclude lyrics with notes (scores), merely record them as poems. -DES ^Talk 19:39, 15 June 2009 (UTC)

That was kind of my point in proposing this; Getting songs out from under poems. I'm honestly curious about songs that may be printed in books, but most 'poetry' leaves me bored. This was intended to cover Songs (with music printed), songs (without music printed), songs (without music printed, but commercially available elsewhere). It was also intended to cover the extremely rare case where a well known author writes Speculative Fiction Songs, and releases them as commercially available rock music. Kevin J. Anderson released an album Terra Incognita: Beyond the Horizon 2 weeks ago where "Anderson developed and expanded one of the novel's storylines to form the basis of an epic rock CD, under the band name "Roswell Six." He and his wife, bestselling author Rebecca Moesta, wrote all the lyrics to the songs." It shares the universe with his most recent novel Terra Incognita: Edge of the World. Heck, I have a CD of Non-genre songs sung by Travis S. Taylor, but since it's non-genre and not officially released and I'm not sure on the authorship it's not appropriate, but it's a darn cool footnote for a complete bibliography. This is one of those things where if you have a title type, it just might bring some bibliographical goodies out of the closet of history. (Shrug - Who knows). I'm not going to be upset if this doesn't go in this time. Kevin 02:01, 16 June 2009 (UTC)

Well, if we go with the "sub-types" approach outlined above, we may decide to create a "song" "sub-type" for POEMs. Ahasuerus 02:35, 16 June 2009 (UTC)

Sub Type is fine here (Above I said 'out from under Poems' but a subtype of poem is just dandy). Of course if this goes live, I'll have to bring up The Who's 'Tommy', Wagner's 'The Rings', and perhaps Rush's 'Cygnus-X' as spec fic. - Chuckle - Just kidding (Sorta) Kevin 04:43, 16 June 2009 (UTC)

Frontispiece

Introductory interior artwork (A higher class of interior artwork) - An alternate choice might be to go with 'Int. Artwork (Pen and Ink)' and 'Int. Artwork (Color/Painting/Photograph)' - An Aside, this alternate option could be adapted for the few Graphic Novels we let in to document the various contributors since they usually separate tasks. There have been some fantastic frontispieces over the years (And more to come as the specialty presses keep chugging along), and relegating them to the closet of 'interior art' along with Jack Gaughan's doodles and similar sketches seems such a shame sometimes. Kevin 06:30, 14 June 2009 (UTC)

I see endless arguments over "quality" if we implement this. And a good many books, particularly from specialty presses, have some very high quality interior art that is not a frontispiece. I can see some reason to distinguish between Interiorart (Color) and Interiorart (B&W) though, or perhaps to distinguish "plates" (set on their own page, nothing on the verso, possibly do not get a page number). And there is currently no good way to indicate multi-page interior art. -DES ^Talk 13:52, 14 June 2009 (UTC)

Why not go all out? (I like the Plate Idea!). Frontispiece, Plate, Interior Artwork (Color or Photograph), Interior Artwork (B&W), and Interior Artwork for all undifferentiated items. Kevin 16:06, 14 June 2009 (UTC)

Ohhh - Just had a (great?) idea. What if we overloaded the 'length' data field for Interior Artwork? When no length is specified, it's just 'Interior Art', but if we add a length of Frontispiece , etc then it will modify the display inside a publication (And on the Artists Biblio page?) just as Novella, Novellete, SS do now for short works. May require some other monkeying with the system (Change Length to Subtype in the entry pages). That solution might actually work for ALLOT of both our suggestions in this section. Kevin 16:06, 14 June 2009 (UTC)

I have no desire for this. If people really want to distinguish Artist efforts this much, I think they would be demanding links from the titles on their page to the actual uploaded images first. As it is, nobody's really addressed artist pseudonyms, and people are rarely merging coverart or such to tidy up artist pages. Very low priority, IMO, not just from my perspective but from what I see people doing with artists at the moment. (Which is mostly just trying to identify them!) BLongley 19:44, 14 June 2009 (UTC)

I'm generally against distinguishing types of interiorart. I wouldn't yell & scream if we had subtypes for color & monochrome, but I don't think it would add enough to be worth it. -- Dave (davecat) 21:35, 16 June 2009 (UTC)

Signature Page

It's a stretch, but it would be nice to document the bound in signature pages of the various special editions as a content level item when viewing the publication in the database. Kevin 06:30, 14 June 2009 (UTC)

Yes, it's a stretch. I think I'd support "Cigarette Ad page" above this, just to keep the magazine page-counters happy. BLongley 19:47, 14 June 2009 (UTC)

How about 'Special'. In the rare instance that there is a 'Special' page in a publication (Signature Page, Cardboard Advertisement, Contest Entry Form, etc). Anything that would/might be of special interest to a collector (Due to presence or absence), but doesn't fit under any particular title type. The page itself will probably be descriptively titled, "Kent Cigarettes" type 'Special', "Signature Page" type 'Special', etc etc such that the type description itself WOULD be redundant, but documenting it as special removes the need for faking it up as an essay, or leaving it malingering just in the Notes. Aha! - Could this be another Author-less type, like the Section Separator discussed above? Could be used to document 'ANY' type of Bind in or tip in feature of a book. Would make documenting errata pages easier when they exist, or 'certificates' that might accompany some special super uber limited editions. (Gosh wow - If I buy this One of a Kind Super Limited Edition, I get the Authors Grocery List in his own hand from the day he delivered the manuscript). Thoughts? (Again, I fairly well expect this to be not added, but at least we discussed it) Kevin 02:37, 16 June 2009 (UTC)

I guess if we have "sub-types", then we could have a TYPE of "Special" (?) and a list of "sub-types", which would include "Section Header", "Signature Page", "Contents Entry Form", etc. Keep in mind, though, that Help is currently written to exclude most of these sub-types, so it would require a great deal of re-verification. Ahasuerus 02:43, 16 June 2009 (UTC)

It's going to happen sooner or later. Any OCD Based Collection hobby (bibliography included) will always get more detailed with time. The work isn't finished until each artifact/item/feature is photographed and filed along with a description of them item in excruciating detail (Look at Taxonomy, Archeology, Ornithology, etc). On a more serious tone, if we are clear in our discussions of verifications that 'All verifications prior to X date may have excluded Y data types' we will be on solid ground for leaving those verifications alone until someone comes along who already wants to reverify that work. Same as has been happening with Cover Art since uploading was enabled last year. Kevin 04:32, 16 June 2009 (UTC)

First off, I don't know what's meant by a "signature page". But there is a problem with including (say) cigarette-ad pages (& SFBC-ad pages, etc. ad nauseum), in that these were often perfed to be torn out. It's often not obvious that something is missing. Potentially, there goes a lot of verification. (Um. I wonder whether these things were ever geographically targeted, different in different copies.) -- Dave (davecat) 21:43, 16 June 2009 (UTC)

I think that a "signature page" is the page in a special signed & numbered edition, that carries the author's signature, and sometimes the signatures of the artist, editor, publisher, or other people significant to the work. Such pages are part of the printed book, just like title pages, and no more likely to be ripped out. I think Bill's suggestion of "Cigarette Ad page" was satiric, though Keven ran with it. -DES ^Talk 22:21, 16 June 2009 (UTC)

Many of the Limited, and Special editions, as opposed to the Trade editions of works coming out these days from some specialty presses (Subterranean is one I have an example of) have an extra 'signature' page bound into the book, with an inked signature. This is usually an unnumbered early page. Kevin 22:20, 16 June 2009 (UTC)

Movie/Album/Graphic Novel Review

I hesitate to suggest this, because so many of the science fact books (which read like science fiction - and I love having cataloged here) and the premier level Graphic Novels could get lumped into it, but how about a 'Review of Non-ISFDB Material'. This would be a combination of essay and review. The Authors and titles would not be hot linked, but it would display as a review (indented, with title, and reviewed work authors shown) inside a publication. Gives proper credit to the reviewer, documents the review (In case the RoA ever change and we can enter them - (Like published filk CD's as an example of a borderline exclusion). Kevin 07:18, 14 June 2009 (UTC)

There is some merit to the non-linking reveiw IMO. Or perhaps just a "do not link" checkbox/flag on the existing reveiw type, to reduce the interface complexity and the number of distinct types. -DES ^Talk 13:46, 14 June 2009 (UTC)

I like the checkbox solution. It also removes this change from this Title Type change discussion. Thanks. Kevin 15:55, 14 June 2009 (UTC)

What about things that are 'In', and reviewed, but not a 'book' (Webzines, Fanzines, Audio Dramatization) Do enough of those exist to fix the 'Book Review' Language with a second 'In' review type? Kevin 07:18, 14 June 2009 (UTC)

Just change the language to simply "Review". I have already entered and linked a number of reveiws of shortfiction works that are not "books". (And I don't agree that Audio Dramatizations are IN. Where do the RoAs include them? See above for why they shouldn't be in, IMO.) -DES ^Talk 13:37, 14 June 2009 (UTC)

I like changing the language to just 'Review'. It also removes this change from this Title Type change discussion. Thanks Kevin 15:55, 14 June 2009 (UTC)

I think Reviews will work so long as there is an ISFDB title to link to: Books, Magazines, Fanzines all work. The display of such could be clarified dynamically I think to show "Magazine Review" or "Fanzine Review" - or just simplified to make it clear it's not always a book. BLongley 20:18, 14 June 2009 (UTC)

To clear up a a lot of our data it would be less effort to change a review to a non-linking review (have you ever tried to change one to an Essay?) so that might be something to look at anyway. But it's not that simple - e.g. we have "Authors" here that exist only because somebody reviewed their single or album. And those Musicians and Bands should go, IMO. BLongley 20:18, 14 June 2009 (UTC)

Don't forget that some Editor title records are merged. See example. The only way you would be able to link to a specific magazine or fanzine is a by link to the pub.--swfritter 00:06, 15 June 2009 (UTC)

I've never felt too bad about linking a magazine review to the year, I'm sure most users can pick one title out of 12 after that. (Or are there weeklies or fortnightlies merged at year level)? Offering links to publications might be good but then people would use them to link books to the exact publication being reviewed, which makes finding them from title level a bit more of a pain. This might be the right thing to do, but I've seen many reviews that refer to both the hc and tp versions. And such should probably apply to all exact reprints as well. The level to link to is "edition" probably, but we don't have that. BLongley 17:43, 15 June 2009 (UTC)

Anyway, as I see it there's enough problems working out what to fix when a review becomes non-linking. BLongley 17:43, 15 June 2009 (UTC)

Short Short Fiction

Defined as being between 1000 and 2500 words in length per Wikipedia though I could easily live with combining this with the flash category below. Kevin 07:18, 14 June 2009 (UTC)

I think this would at most be an additional storylen setting, not a title type, just like Novella, Novelette, and Shortstory. But don't we already have enough arguments over what length a given work is? why add more? and AFAIK none of the major awards distinguish these super-short lengths. -DES ^Talk 13:42, 14 June 2009 (UTC)

You are correct, I was getting myself wrapped up in the different 'types' of items I would like displayed separately on the Author's biblio page, or that would have the 'length' call outs we have discussed elsewhere. I retract the suggestion. Kevin 15:51, 14 June 2009 (UTC)

I'm happy to look at this as a Shortfiction story length, but again, I'd want some justification. The category below has at least 300 titles that I know could be mildly usefully adjusted. How many in this category would benefit? BLongley 22:20, 28 June 2009 (UTC)

I haven't a clue. I threw it out for discussion, then realized it wasn't a title type, so stopped thinking about it. (shrug) Kevin 02:46, 29 June 2009 (UTC)

Flash Fiction / Super Short

Defined as being between 1 and 1001 words in length per Wikipedia. Kevin 07:18, 14 June 2009 (UTC)

I think this would at most be an additional storylen setting, not a title type, just like Novella, Novelette, and Shortstory. But don't we already have enough arguments over what length a given work is? why add more? and AFAIK none of the major awards distinguish these super-short lengths. -DES ^Talk 13:43, 14 June 2009 (UTC)

You are correct, I was getting myself wrapped up in the different 'types' of items I would like displayed separately on the Author's biblio page, or that would have the 'length' call outs we have discussed elsewhere. I retract the suggestion. Kevin 15:51, 14 June 2009 (UTC)

This designation actually would be very valuable for a lot of magazine entries; Ferdinand Feghoot, etc. The awards distinguish stories with these lengths by almost totally ignoring them. The arguments over story lengths are very rare except in the case of novella vs novel.--swfritter 11:57, 28 June 2009 (UTC)

I can support this as another SHORTFICTION storylength setting if it's going to be of significant use. There's at least 300 book entries that would benefit: Drabbles are exactly 100 words and would fit nicely here, but I'm not asking for Drabble as a separate category. I'll fix Drabbles if someone else will fix the rest though. (And yes, I do mean I'd look into the coding changes as well as fixing the 200 Drabbles already here, and maybe adding the other 100 as they're Spec-Fic too.) BLongley 22:08, 28 June 2009 (UTC)

I can see the value in clearly identifying qualitatively different types of Titles, e.g. fictitious essays, but I am not sure under what circumstances a higher level of granularity for story length would be desirable. Are there some uses for it that I haven't thought of? Ahasuerus 02:23, 29 June 2009 (UTC)

Introduction

And possibly 'Author's Introduction' to separate between 1st party and third party introductions. Kevin 07:33, 14 June 2009 (UTC)

Um. Why? I'm inclined to think that having ESSAY as a type, & "introduction" or "preface" or "foreword" in the title, is adequate. Admittedly, authors (& editors & other 2nd-party introducers) don't always use such a word; but I think the potential gain is still very small. -- Dave (davecat) 21:50, 16 June 2009 (UTC)

Afterword

And possibly 'Author's Afterword' to separate between 1st party and third party introductions. (And yes, I'm stretching the envelope all over with these suggestions, because it will probably be a 5 years before we go through this type of content item expansion again, if ever) - Blame DES. He started it (chuckle). Kevin 07:33, 14 June 2009 (UTC)

What is the value of distinguishing between Introductions, afterwords, and other essays? I don't see it, but maybe after you point it out... -DES ^Talk 13:39, 14 June 2009 (UTC)

Introductions discuss the 'work at hand' with the understanding the reader hasn't read it. (No spoilers). Kevin 15:48, 14 June 2009 (UTC)

Afterwords discuss the 'work at hand' with the understanding that the reader has probably read it (Usually a more critical analysis - with spoilers). Kevin 15:48, 14 June 2009 (UTC)

Essays do not need to have a 'work at hand' and can (and often do) stand alone. There is no required associated longer/other work. Essay's are more likely to be reprinted elsewhere, referenced, etc. while introductions and Afterwords are nearly always tied to a single work. Essay's can also be about something totally different, like a science fact article or other non-fiction type short work. Kevin 15:48, 14 June 2009 (UTC)

I think that is not merely as general a rule as you may believe. I have seen a number of "critical introductions" that discuss the work or works in detail, not avoiding spoilers. I can slso cite at least one case of an introduction later reprinted away from the work it origanally introduced. (See 118815.) Come to think of it, both Clarke and Le Guin have included introductions from other publications in collections of essays. Also, I have seen anthologies that include a "Foreword", a "Preface", and an "Introdution". Would all these be of type "Introduction"? With granularity this fine, it might be much harder to find a given work, and I can foresee lots of argument over borderline cases. (More types make more borders and hence more borderline cases, and i cna think of a number of texts that don't exactly fit any of the proposed subtypes but aren't exactly critical essays either.) I think this is probably going too far in differentiation. -DES ^Talk 20:02, 15 June 2009 (UTC)

All good points, and perhaps valid enough to exclude this title type. (as to being reprinted, I just said that essay's are more likely to be reprinted elsewhere). Kevin 02:16, 16 June 2009 (UTC)

As for my general reasoning, I attempted to recall Every 'type' of extra information we overloaded into the Title Field, because we didn't have a Title Type (Introduction (STORY NAME), Afterword (STORY NAME), Map (STORY NAME), etc.) and we created a title. We used to overload the title field for Variants, then we upgraded. This is just another upgrade. Now we can have STORYNAME of type MAP, STORY NAME of type Authors Introduction, STORYNAME of type Afterword, etc, just as we can now have STORYNAME of type Interior Art. AND Bonus - We can break all those introductions, etc out of Essay on the Author Biblio page. Now any critical essays they write can stand out from the crowd of introductions etc. Kevin 15:48, 14 June 2009 (UTC)

The need for parenthetical additions is not to subtype but to disambiguate. There are thousands of pieces of text (whether we call them essays or whatever) whose only title is "Introduction" If we don't put "(storyname)" into the title, the author's biblio page may have a dozen or more entries that can't be distinguished except by drilling to the publication level. If we changed these all to type "Introduction" we would still need to keep the "(storyname)" in the title, or there would merely be a section on the biblio page "Introductions" with a bunch of things not usefully distinguishable from one another. And don't suggest that the disambiguator can be automatically picked up from the enclosing publication. This makes no provision for intros to individual stories, of which I have entered quite a few (when they seem significant enough to record, not mere blurbs -- a judgment call) Auto-disambiguation also won't handle well the situation when an anthology or collection is reprinted under a different name, but with the introduction unchanged. -DES ^Talk 20:02, 15 June 2009 (UTC)

You are correct. I wasn't considering the disambiguation completely. It will still be needed on some cases (many cases?). But some cases could end up going the way of interior art and sharing the name of the primary work with a different title type. How does this strike you (This is my last argument for an Introduction and Afterword Type). Picture a basic novel, with an Intro, Novel, and Afterword. With a separate Intro and Afterword type, it could be entered as 'NOVELNAME type Intro', 'NOVELNAME type Novel', and 'NOVELNAME type afterword'. Now disambiguation isn't needed. Feel free to say 'I Hate it, we'll never do it that way'. As I mentioned earlier tonight and several sections above, I posed most of the multiple 'slices' here to get it all out on the table, and trim it down to what we wanted. Better that, than to get done and realize we should have put in something not considered. Kevin 02:16, 16 June 2009 (UTC)

I think 'NOVELNAME type Intro' and 'NOVELNAME type afterword' might encourage people to break the 'exactly as stated' principle. Did it actually say "Intro" in the pub, or "Introduction" or "Introduction to Novelname"? Was the afterword called "Afterword" or "Author's Afterword" or "Editor's Afterword" or "Publisher's Afterword"? Yes, I know our current convention is only "Exactly as stated (plus a possible suffix)" but we've lived with that quite a while now. BLongley 18:20, 16 June 2009 (UTC)

I agree with Bill here. Kevin's suggestion might well work for the sort of "basic" case he describes, but I thing it will fall down on more complex cases, cases where an into or afterword is reprinted as an essay later (I cited several above) and cases where an intro or afterword, particularly by someone other than the author of the work being introduced, has its own title. It is in some ways a nice idea, but I think it falls down on inspection. -DES ^Talk 19:42, 16 June 2009 (UTC)

I can see the reasoning, but I've never really heard any clamour for differentiation of INTERIORART or ESSAY before and there would be an awful lot of rework of existing data needed for little benefit that I can see. BLongley 20:26, 14 June 2009 (UTC)

Clamour? No. Not when What's New is 365 days - 1 whole year out of date (as of today oddly enough). There hasn't seemed to be much point. I'd probably be happy enough with Intro/After, Fake Essay, Map/Diagram, and the old Essay and Interior Art, but you make hay when the sun is shining, and it appears right now that the sun is shining. Hence all the noise. Kevin 21:06, 14 June 2009 (UTC)

Good point about "What's New" - updated now. Thanks! Ahasuerus 21:47, 14 June 2009 (UTC)

Non-Genre Shortfiction

I have several times seen people try to enter the Shortfiction contents of non-genre or mixed-genre anthologies or collections, and they often try "NONGENRE" contents, which just creates new book-level titles. Someone desperately wants such in, judging by Maxim Jakubowski's page. I suspect a lot of shortfiction here isn't actually genre but is added for completeness. Could we use such a category, or just encourage people not to enter such? BLongley 20:37, 14 June 2009 (UTC)

I Like the idea. It documents the contents (when someone has entered them) so they don't get entered again, and properly classifies them. Many of the collections in Bleiler's checklist are collections with 1-3 'in' stories, but he doesn't identify the stories explicitly. Kevin 20:48, 14 June 2009 (UTC)

Separating "genre"/"non-genre" from Title Type has been often requested and I agree that we need to do it. Like some other proposals here, it probably belongs on the ISFDB:Proposed Design Changes page since it will require a database change. No hurry, though; we are accumulating a lot of good info here and can sort it out later. Ahasuerus 21:38, 14 June 2009 (UTC)

I also like the idea, but agree that it does not need to be on the "right away" list. -DES ^Talk 20:03, 15 June 2009 (UTC)

I agree that this would be a Very Good Thing, having run up against the what-to-do-with-nongenre-contents problem a few times. I also agree that it's a database structure change, to be undertaken when possible, not one of the things we could in principle hash out & expect someone to just do one of these days. (I guess I'm just metooing here.) -- Dave (davecat) 21:56, 16 June 2009 (UTC)

Given the number of new types under discussion, it's probably wise to settle the ones that we think will work and which won't, and make one big change to the software to allow the ones we want. I think the mandatory database change is small - just add some more options to the enumerated types. (Unless people are expecting the introduction of such to magically change all current worked-around entries, and I don't think that's going to happen. People will have to fix things record by record.) What we will have to do is make sure the new types appear in all the right places on our current screens, or define where they should appear - e.g it should be fairly simple to include Non-Genre Shortfiction with Genre Shortfiction everywhere, but as the point is to clarify the difference it's probably wise to consider where such should appear. (IMO, somewhere near the bottom, but that could be at the bottom of Shortfiction or at the bottom of the page, and I guess there's always the possibility that Genre and Non-Genre Shortfiction end up in the same series.) BLongley 20:37, 18 June 2009 (UTC)

I have copied Bill's commetn to #Conclusion? above. I urge that people respond to it there, rather than here. -DES ^Talk 21:04, 18 June 2009 (UTC)

(unindent) It may be that we should be sure we have a consensus on the RoA before proceeding with this. See Rules and standards discussions‎#Nongenre shortfiction for current discussion on this point. -DES ^Talk 21:04, 18 June 2009 (UTC)

Excerpt

Just thought of this after looking at the "disambiguation" suffixes we use on Introduction and Afterword essays and such. Not all excerpts have the same title as the full work, e.g. Amazon Fragment (Excerpt from the first draft of Thendara House). An EXCERPT could be just another subtype of SHORTFICTION maybe (I tend to leave the Story-Length blank for such anyway, might as well use it for something useful) but I'm also thinking that if given its own type we could Link the EXCERPT to the title it's actually an excerpt of, same way as we link REVIEWs. Thoughts? (Flame away - I only thought of this 5 minutes ago and there's probably some big flaws I haven't thought of yet.) BLongley 18:32, 16 June 2009 (UTC)

The major objection I can see is that it might encourage more recording of routine promotional excerpts which I gather we have been tending to discourage of late. If we are going to record those, a special type or special length/sub-type might be a good idea. Not IMO as urgent as some of the other suggestions, but worth considering. -DES ^Talk 19:45, 16 June 2009 (UTC)

I'm not sure we have been discouraging them recently - in fact, I think people have been entering them more often now, if only to explain the differences in page-counts between various sources. I know that I always used to ignore promotional excerpts after the numbered pages ended, but found oddities worth recording, like an entire short-story being used as a promotional excerpt for a Collection, or a serialised story only (originally) available by buying several books in the Star Trek series (didn't matter too much which book you bought, so long as you bought ONE of the Star Trek titles in a particular month, you'd get that episode). Rare oddities, but worth consideration, even if it ends up in the "most of us don't really care" bucket. BLongley 21:09, 16 June 2009 (UTC)

I think Excerpt as a Short Fiction 'length' or subtype is a fine idea. (And I agree with Bill - I don't find them in old records, but I do find them in new records - Makes me think we are entering them more. Kevin 22:24, 16 June 2009 (UTC)

I think we have to choose one method or the other: if EXCERPT becomes a true content title-type it's not too difficult to introduce the linking to the parent title. (Just extend/reuse all the REVIEW coding. Well, once we've fixed that.) If it's just a new "length" for SHORTFICTION then it's probably a bit less coding, but we'd still have to overload the title occasionally and make people go hunt manually. And some people might want to know how big an excerpt it actually is. I don't think we've got a lot of excerpts that need attention, but as we're on the topic it's still worth thinking about. BLongley 20:22, 18 June 2009 (UTC)

Large Issues: Roles & "Based on"

There are two large issues that I for one would like to see at least some planning/design work on. In fact I regard them as rather more important than any of the title type issues discusses above, although probably involving more db structure changes. These are field support for additional "roles" and support for a more general "based on" relationship. Roles would also be highly useful, if not essential, to some of proposed the new types discussed above. -DES ^Talk 13:54, 16 June 2009 (UTC)

Roles

Roles have already been discussed a bit at ISFDB:Proposed Design Changes#Roles and elsewhere. My current view is that these should ideally be flexible, with different publications having different numbers of roles recorded. When editing there should be a "role" pulldown, with the option of also entering free-form text if possible, and a corresponding edit box to enter the name of the person filling the role. There would be an "add role" button which would add another pulldown/editbox pair to the form. On display the roles would be part of a publication's metadata, perhaps in the form of a bulleted list. -DES ^Talk 13:54, 16 June 2009 (UTC)

Roles could support capturing such information as translator, cover designer, book editor, editor of single-author collection, narrator for an audio book, series creator for sharecropped works, etc. If we implement an audio reading type, use of roles might be very important. -DES ^Talk 13:54, 16 June 2009 (UTC)

I think this needs to be broken down a little too. Some of the roles are publication level, some are title-level, some may be either or both. E.g. Cover Designer is almost certainly publication level. Collection Editors are probably title-level, but I've seen Silverberg and Clarke both credited as Editors of the Same Anthology in different countries, this might occur for Collections too. Translators and Readers may be at publication level or at pub-content title level: for a Novel it's probably Publication, but for Anthologies and Collections there may be different translators or readers for each content title. There's probably an overlap with Variants or "Based on" relationships to consider too. But there's evidence of oddities already planned for: e.g. BACK Cover Artist. BLongley 19:29, 16 June 2009 (UTC)

Some text moved to #Splitting Authors and Artists below -DES ^Talk 23:33, 17 June 2009 (UTC)

Still, again these are good suggestions and should be on the discussion list. BLongley 19:29, 16 June 2009 (UTC)

Good points. We may well need roles at both pub and title level, but I suspect that pub-level are more urgent and would wind up being more common. I agree with your examples. Translators should probably really be at edition-level, except that we don't have that. (I have seen multiple publications of each of several different translations of a given Verne work, for example.) But then the same could perhaps be said of cover artist. -DES ^Talk 19:55, 16 June 2009 (UTC)

Some text moved to #Splitting Authors and Artists below -DES ^Talk 23:33, 17 June 2009 (UTC)

[after editing conflict] I don't think authors (in our current use of the word which includes artists and editors) should be assigned roles, but that roles should be assigned to authors. There's a difference there. Let me explain. Robert Silverberg should have one record for all his credits. His relationship to the publication is defined by the role we assign him. The system automatically assigns him the role of editor when the pub type is ANTHOLOGY, as it assigns him the role of author when the pub type is NOVEL. If he were credited with an INTERIORART credit, we would assume him to be the artist, even though there is no such credit explicitly assigned by the system. I would suggest a entry form that allows a drop-down menu to assign pre-defined roles then allow the editor to add another credit (as it does now for cover art). That way we can assign as many roles to a publication as are necessary. The system will combine these credits into one "author" summary page, but only display the assigned role on the publication record. To bring up a recent example: Robert Conquest edited The Robert Sheckley Omnibus, but there is no credit on his summary page. Once role assignment has been implemented, it will. The question is: how will it be displayed? Perhaps under a newly created Book Editor category. Which begs another question: how will this is differentiated from books listed under Anthologies which at the moment assumes the author to be the book editor? MHHutchins 23:13, 17 June 2009 (UTC)

Collection editor perhaps? -DES ^Talk 23:39, 17 June 2009 (UTC)

Based on

My views on designing a "based on" relationship are at ISFDB:Proposed Design Changes#Based on, ISFDB talk:Proposed Design Changes#"Based on", and at Feature:90164. Such a feature could be combined with an improved handling of variant titles, or separate, leaving variants for cases where the text is more-or-less identical but the title and/or author credits are different. My preference would be for the second (separate) option, but either could do the job. -DES ^Talk 13:54, 16 June 2009 (UTC)

A one to one "based on" relationship with more flexibility should be fairly easy to implement. E.g. the current variants are simply based on a title having a title_parent. Add a "title_parent_type" column and you can specify the nature of the relationship - default this to the "current" or "usual" meaning of "variant" (whatever that is: same text with different title or author-pseudonym?) both for existing data (populate with default when we add the new column) and new entries (add the latter to the make variant title screens). Provide some tools on title-edits to change the link type on the variant title so we can correct/refine the existing relationships. Making the parent '0' to break the link should also still work (although making that more intuitive should also be a long-term goal.) BLongley 19:06, 16 June 2009 (UTC)

The one to many relationship (e.g. for fix-up novels based on several short stories) is more complex. Compare it with our problems with author pseudonyms: we can't currently undo such (although that feature is also requested). We do have a table for title_relationships at the moment, for reviews and interviews, which already seems to have some suggestion of a "translation of" relationship. But we haven't yet fixed the problems caused by pseudonymous reviewers or interviewers. (Well, I have, and have provided a fix script for existing data too, but the code fix is r2009-04 and the data fix is not necessarily going to be implemented on risk grounds). But that area is still a bit too "Here be Dragons" for my comfort. BLongley 19:06, 16 June 2009 (UTC)

Still, I think both parts are highly desirable, but probably should be split so that we novice ISFDB developers (and I only mean novice on ISFDB, I think most of us actually pretty experienced at development elsewhere) can go from small fixes to medium fixes to large fixes rather than jump from small to large. BLongley 19:06, 16 June 2009 (UTC)

Actually, a one to one could do the job, as long as it isn't exclusive. A work can be the parent of many variants now, each a separate "variant" relationship. Similarly, a work could be "based on" more than one other work, each a separate "based on" relationship, to handle fixups. The real problem is the other way, i think. Currently a title record that is a variant has exactly one parent, while more than one work may be based on a given work, there might be a condensed version, a later revised version, and still later it might be used in a fixup. (I can find such examples easily enough.)-DES ^Talk 20:12, 16 June 2009 (UTC)

I'm not sure what you mean by "one to one could do the job, as long as it isn't exclusive"? There's no problem with "A work can be the parent of many variants" , but there IS a big problem with a variant being the child of more than one parent. BLongley 20:40, 16 June 2009 (UTC)

I see your poijt, i was thinking as I wrote and was thinking only of the "parent" work i fear (although in the case of a "based on" relationship i think that our notion of "parent" and "child" may be reversed, or it may be sufficiently symmetrical that such designations don't make sense at all). My comment was premature.-DES ^Talk 21:12, 16 June 2009 (UTC)

In any case, however the code is done, I urge making the interface (and display) separate from, even if similar to, the current variant interface. This will allow for an enhanced fully-flexible version of based on later, after a simpler and more limited implementation earlier. I fully understand the need to take small steps, i was not calling for a full implementation of my vision next week. -DES ^Talk 20:12, 16 June 2009 (UTC)

I think clarifying the current "make variant" reasons will help us decide which are most needed in the long run. Let's do that, and see what people complain about. If somebody says "I can't make this a 'budget edition' variant of the main title!" we can point them at the price guidance, or if they complain about not being able to make it a "translation of" we can point them at the rules on how to enter foreign editions. Or change those rules, if needed. (We might end up with a better way to enter foreign titles, currently banned from variant rules unless it started as being not in English.) BLongley 20:40, 16 June 2009 (UTC)

The problem is, that implicitly makes the decision that a based-on relationship is a form of variant. -DES ^Talk 21:12, 16 June 2009 (UTC)

No, it will clarify that we already have variants here trying to show a "based-on" relationship. E.g. Belin's Hill (revised 2005). The first step would hopefully show up such anomalies: if the common understanding of variants is currently "same text, differently captioned" then the misuses we currently have should stand out a bit better, and be fixable, or inspire discussions of why "variant title" isn't the way to go in a particular situation, and encourage more effort into designing "based on (generic solution)". I suspect the current one-to-one relationships like this are most easily clarifiable this way, but showing up the faults and dealing with the results looks a useful small step. BLongley 22:12, 16 June 2009 (UTC)

Yes we do indeed have such variants now. Weather this is a misuse of the current mechanism might be debated -- i have argued in the past that it is, but others have disagreed. If the full "based on" relationship is implemented as i suggest, such variants will eventually be improper and, in time, corrected. I fear, however, that rather than encouraging "more effort into designing 'based on (generic solution)' " it would more firmly legitimize such variants ("See we already have variant text for that") and discourage a fully flexible "based on" or similar feature, and leave no good solution for recognizing fixups, split novels, and other relationships that a full "based on" solution can handle but a "variant text" feature cannot. -DES ^Talk 16:01, 17 June 2009 (UTC)

There is also the problem that "text variants" and "title variants" are at least potentially orthogonal, which IMO is another reason to have them be separate relationships. For example, a revised work may be published with an author credit other than our canonical name. To keep the author's biblio intact, a parent record must be created and the published work listed as a variant -- a "title variant". But if it is already a "text variant" of the original work, this would mean that it would be a variant of two parents, which the current mechanism does not support, and neither would the suggested "enhanced variant" ("text variant") mechanism, as I understand it. Similarly, if a work ABC (shortfiction) is expanded into ABC (novel), but the expanded version is later republished as DEF, an "enhanced variant" mechanism would require a variant chain, or a work that was a variant of two different parents.

These issues, as well as the display differences i discuss in my response to Kevin below, are why i think that implementing an "enhanced variant" now would be going down a blind alley -- at best it would require an almost total rework later, and at worst it would make it less likely that a full implementation would ever happen. -DES ^Talk 16:01, 17 June 2009 (UTC)

I'd say that that the biggest barrier to the full solution is trying to jump to that in one go. Big changes take a lot of time to develop, and a lot of testing, and get a lot of complaints in the meantime. Small changes, even if they may make things too "comfortable" in the short term, won't stop some people pushing for a further improvement. And we'd get those tested by everybody using it, not just volunteer testers with their own local ISFDB. I'm also not sure why you think an Author-name variation means "two parents", can you clarify that please? (It might be good to indicate titles that actually only exist due to the need for a "Canonical Author" version for the main Author page, but we'll also have to cope with the fact that such may eventually get used when people discover that say, Richard Bachman was Stephen King.) I don't consider such changes a blind alley, I consider them a useful experiment at worst. BLongley 20:07, 17 June 2009 (UTC)

As a professional developer myself, i do understand that insisting on a huge all-at-once change is often a recipe for getting noting at all done. I am not saying that I don't want any change less that a full implementation of my dream feature. Not at all. But I also know that making an early "easy change" sometimes tends to constrain future development because of the reluctance to 'pull it out and start over' and even more the accumulated data that might need to be reworked or even tossed to fit a new design. In short i am not saying don't take incremental steps, i am saying have at least a rough plan so that those steps are headed towards a desirable final goal. I strongly suspect that piggybacking a new "text varient" or "nature of the relationship" expansion of the existing variant code would tend to constrain future development in ways I think unwise. Suppose that instead, as an incremental step, a different mechanism, similar to an expanded variant feature,. but using a new db column were implemented. it might, as a first stage, still be limited to having all "child" records point to a single parent record, rather than being a many-to-many connection. It might start with an entry interface effectively cloned from the existing variant interface. But it would allow for orthogonal intersections of variant relationships and the new "related work" or "based on" relationship, and it would be possible to experiment with different means of displaying the new relationship without changing the display for existing variants. would that be a sufficiently small incremental change, in your view? -DES ^Talk 21:14, 17 June 2009 (UTC)

Let me clarify some of the plausible scenarios that make me thing there would be a collision if "text variant" is built on the existing variant system. I could probably find actual examples of all these, but inventing them will be easier.

1. a) J. Random Author writes (and publishes) "Spaceship!", a novella. b) Later he expands it into Spaceship!, a novel. c) Still later this is published under the title Spaceship to the Stars with an author credit for "J. R. Author". Now b is a text variant of a. c is also a text variant of a, but is a title variant of b. This means that c has two variant parents, or at the least is a variant of a variant. Not good (within the linmits of the current mechanism.
2. a) J. Random Author writes (and publishes) "Superbeing" a novella. b) He expands this into a novel, which is published as Hyperbeings under the name "J. R. Author". Hyperbeings is never published as by our canonical author name ("J. Random Author"). Hyperbeings is a text variant of "Superbeing". c) But to get Hyperbeings listed on JRA's canonical bibio pag, we must create a record for "Hyperbeings as by J. Random Author", and make the title record for Hyperbeings as a actually publiahed a variant of this. So b must be a child of both a and c in variant relationships, albeit with a different "nature" for each. If "text variant" just adds a "nature of relationship" column to the existing "variant" mechanism, this situation cannot be handled properly. While if "based on" is separate, even if it does not support many-to-many relationships yet (and thus cannot handle most fixups) it can handle these and many similar cases, because a work can have a parent in a variant relationship, and a different parent in a "limited based on" relationship.

In general, any time a derived work involves both text changes and title or author name changes, a variant+"nature of relationship" solution will not handle things well.

Also, i do thank that the display of "text variants" or "based on" works should be quite differetn from the display of "title variants". Our current variantes are pretty much never displayed separately -- only on the same line as or adjacent to the parent work in biblio or series displays. I think this is correct for title (or author) variants. But IMO text variantges should display in their own places on biblio or series pages: they may have different types (novel vs short story, say) or different publication dates, and should be found in the correct place. Can you imagine if 2001 could only be found in the shortfiction section as a variant of "The Sentinal"? But that is what a text variant system would do if the current display rules are retained. I suppose that an enhanced variant feature could change its display behavior depending on the relationship chosen. But I suspect that would create overly complex and fragile display code.

Having said all this, all i really ask is that the longer term goal, and some of these scenarios, be kept in mind when creating whatever interim solution is first implemented. Creating a change that will need to be pretty much ripped out for the next step is perhaps not a step foreward, so please plan not doing that. That is all I really ask. -DES ^Talk 21:14, 17 June 2009 (UTC)

I very munch want to reserve the current variant mechanism for cases where the two works are identical in content (not counting trivial corrections of typos and such). Once we start down that path, i fear that the clear distinction between records that are for the same text, differently captioned (meaning different title or author credit or both) and records that are for different but related texts, will be lost or at least blurred. Also i strongly feel that title records for "different but related" texts should not be hidden on author pages, as now happens when an expanded version is marked as a variant. I suppose the display code could do that only if the variant reason is "altered title" or "author credit" or whatever, but not when the reason is "expansion" or "revised version" or whatever. But i really do want any changes to be made with an eye to making that distinction quite clear, and i fear the easy-to-code method you discuss with have the unintended consequence of foreclosing such a clear distinction. If that can be avoided, i have no problems with an incremental approach. -DES ^Talk 21:12, 16 June 2009 (UTC)

I don't think the first step I proposed will hide anything, it should make it clearer which current variants are inappropriate. Yes, that may make some things look worse in the meantime, but it doesn't actually change anything that we already have. BLongley 22:12, 16 June 2009 (UTC)

It won't hide the existing variants, no. Indeed in the short term it will make things better, by making it clearer which existing variants are "title variants" and which are "text variants". What i fear it will do is make things just enough better to reduce the pressure for a more comprehensive solution, and leave more data to be changed in more complex ways if that solution is ever implemented. I also fear that by conditioning ISFDB editors to think of revised, expanded, condensed, etc works as "text variants" the differences between these and "title variants" and the similarities between these and fixups, split and rejoined novels, and other things that could fall under a general "based on" relationship will be obscured, making the implementation of such a general relationship less likely. -DES ^Talk 16:01, 17 June 2009 (UTC)

Of course it is tempting to combine this with the existing variant mechanism, and it could be done much as you suggest above. But I fear that if we ever went to a more full-featured version of "based on" it would result an an over-complex interface to handle all the possibilities. I fear it would also lead to confusion between several concepts: variants that may never have existed but which we record to get all publications listed under the canonical author and canonical title; variants that had a different title or author credit, but essentially identical text; and versions or related works with significantly different text. Note that a variant is not listed separately on an author's biblio page, and is not even noticed by "dup candidates" (although that might change), while a "based on" work should, IMO be listed separately because it is a different work, not just a different name for (description of) the same work. That difference in display behavior may suggest different database storage, but that is up to the developers. -DES ^Talk 20:12, 16 June 2009 (UTC)

This is why I'm suggesting small steps, and pointing out possible overlap problems. I've actually introduced a slight change to displays of variant titles recently (I'll let people find it for themselves - it's what some people have been asking for, but I think it'll help if people compare all the displays and decide which is best, or whether we should work on allowing people to choose which style). But we have no overall designer now, and I'm trying to be cautious. BLongley 20:40, 16 June 2009 (UTC)

I wish you would explicitly point it out to me -- on my talk page if you don't want to alert others -- so that I can look for examples and see what I think. -DES ^Talk 21:12, 16 June 2009 (UTC)

I don't think I'm revealing too much if I suggest you look at a Series, and how the Series displays on Author pages. I know there's been complaints about the "As by" making titles look as if they've only been published under that author name when it's been published by canonical author and pseudonym. But there's a lot of differences in displays, and I'm not going to make it obvious which I've introduced: they're all valid options. BLongley 22:12, 16 June 2009 (UTC)

Well, all recent software changes have been posted on the Development page, so presumably Bill is talking about the addition of VTs to the Series page (listed as "Subset of Feature 2804561".) My guess is that Bill is referring to the fact that his new logic always displays a VT on a separate line while the Summary page displays VTs on the same line using the "[as by Robert Heinlein]" format (in most cases). I noticed this difference in behavior during my testing and let it slide since this was a "quick and dirty fix" and we have to do a more generic rewrite to get the logic behind the two pages reconciled anyway.

Having said that, I am afraid that your comments above were very hard to decipher, Bill. Even though I tested the change and created a new Feature request for a more comprehensive solution just ten days earlier, it took me 20 minutes to figure out what your were talking about. Any editors who were not familiar with this change were in an even worse position. With the number of things going on and the number of proposed changes (in this section alone!), any non-specific suggestions "[to] compare all the displays" will likely be given a brief frown and then ignored. Ahasuerus 03:34, 17 June 2009 (UTC)

Being "hard to decipher" was exactly the intention. Making you spend 20 minutes on it was not. Sorry for that. :-( I gave pointers so that if people are really interested they can go look at that area with unbiased views. If it looks too hard, don't bother. If nobody bothers, I'll "compare all the displays" myself and put up some proposals later, but it'll be like Kev's Menu change proposals - lots of screenshots and lots of explanation needed, and I'm not up to that much work till the weekend at least. And there are plenty of other things to look at in the meantime, so that may be several weekends away. It's low priority stuff and I really only wanted people severely interested to look at it. BLongley 20:29, 17 June 2009 (UTC)

Frankly i find comments like "I'll let people find it for themselves" and "I don't think I'm revealing too much if I suggest you look at a Series" frustrating. I find even the listings under "outstanding changes" a bit terse. When a new patch/release is installed, or a recently revised feature is discussed, as one was here, I would like a clear explanation of what has been changed so that I can go into the DB and find (or create test) examples, and see how they work and if I like them. I understand that repeating all the details in the "outstanding changes" table might take too much room, but a link to the sourceforge item would be helpful, and a comment there or somewhere linked explaining how a change was implemented when this might not be obvious from the bug or feature listing. An added comment in the SF item would do, i should think. -DES ^Talk 16:01, 17 June 2009 (UTC)

Sorry for the frustration - the intention was that people would either be immediately interested or not. (Explained above.) BLongley 20:45, 17 June 2009 (UTC)

As to the listings on "outstanding changes": well, I'm open to suggestions. Make them too big and people are scared away. Make them a link to a different site like Sourceforge and people are scared away. Duplicate stuff here and on Sourceforge and people will only read one or the other. And even if the request/bug is standardised, we'll probably only get discussions on proposed fixes in one place or the other. :-( (Yes, it's a mess.) BLongley 20:45, 17 June 2009 (UTC)

The After installation messages aren't something I do, you'd have to take those up with Ahasuerus. But if we can put all the bug/feature reports, proposed fixes/enhancements to such, and discussions of such, in one place, we'd give him one thing to link to. Which would probably be a major help, as we're overworking him at present! BLongley 20:45, 17 June 2009 (UTC)

I can see that small steps are needed -- I am NOT urging that we rush into major change, and I'm sorry if it sounds as if i was. I quite agree that possible overlap and interaction issues must be considered. I am not dismissing your issues. Do you see the distinction i am trying to make by suggesting that "based on" not piggy-back off variants, or at least not appear to to the user? Perhaps there is a better way to make this distinction very clear while using the existing code and taking an incremental approach. -DES ^Talk 21:12, 16 June 2009 (UTC)

Having reviewed all this above and links, it seems that 'Based on' for text changes is "Variant Text" in sheeps clothing. While we currently have a single 'Variant' that is intended for 'Variant Title' and is also used for 'Variant Author'. If it's going to look like a variant on the authors biblio page, and the page to create it is going to replace the make variant page, then I'm afraid it a Duck, err "Variant Text". The 'fixup' problem can also be solved by having code that causes a 'Variant Text' relationship to appear many times on the Authors biblio page. The Novel appears first, and then where each Short Work appears, have the 'Variant Text' line appear. Why re-invent the wagon when we already have a horse and a yoke? Variant can be expanded to 'Variant Title', 'Variant Author', 'Variant Text', 'Variant Form' Perhaps for Audio Books?, 'Variant Language' (For translations). We could even be so bold as to split 'Variant Text' into 'Variant Text (minor)', Variant Text 'Expansion', Variant Text 'Abridgment', Variant Text Fixup, and illuminate all the (pardon the pun), Various Variants. Kevin 03:54, 17 June 2009 (UTC)

Part of my point is that, if I were designing it, "based on" would NOT look like the current variant on biblio pages. On a biblio page a variant appears on the same line (or I gather in a series display on an adjacent line) with the parent text. It does not appear in its own place in the biblio page, neither its alphabetical place in an alpha display, nor its chronological place on a summary display. In a title record display of the parent, all publications under a variant are displayed together with those under the parent record, and indeed it is not always obvious which are under the variant, particularly if it is an author variant. Whereas in my vision "based on" would work more like our current review display. A work that is "based on" another work would display in its own place in all biblio and series displays, it might be in a different series from the work on which it is based, it would have its own date, and would be in all ways a separate work. Indeed it might well have variant titles. At the title level, there would be a line with a link, something like "Based on: XYZ Work by ABC Author and QRS Work by DEF Author" where the underlines represent links. There might be some indication of this at the biblio level, but quite possibly not. And at the title record level of the corresponding work there would be line reading something like "Based on this work: GHI Work by ABC Author; LMN Work by ABC Author; XXY Work by ABC Author and DEF Author".

Also i don't think the page to create a based on relationship should replace the make variant page, precisely because the restrictions that should apply are different. Make variant can (and should be able to) create a new parent; make based on should not be able to. All variants should be children of a parent that is not itself a variant (although the code does not now enforce this), this is not true for "based on". A variant should be of the same type (Novel, shortfiction, Nonfiction, etc) as its parent (although the code does not now enforce this); this is not true for "based on". A variant can be a variant of at most one title (both in the current code and in my vision of the ideal code), whereas a work may be based on many other works, and still have others based on it.

In my view thinking of "based on" as an expansion of variant tends to obscure these differences, and risks leading to a design that locks in the pattern of the cur3ent variant feature, and may preclude or at least make harder an implementation of a full-featured "based on" feature. -DES ^Talk 14:12, 17 June 2009 (UTC)

I agree with DES here. Variant text (whether "based on this", "expanded from that", "revised from this", "made up of these") should have its own identity away from variant title. Users, editors, moderators should all see this distinction clear enough in their heads (and on the screen) not to confuse the two. Having a different entry interface and data display would bolster this distinction. Because a work can be revised several times or become part of several fix-ups (van Vogt from the beyond is nodding) it can't be handled like a variant in title or author, and the display should reflect that. I have no idea how this could be done, perhaps on the title record page, if not on the author's summary page. Ideally, a title might appear in different categories on an author's summary page. I don't see why a short fiction title in a series can't also be displayed under the short fiction category as well. A database user might not know a certain story is in a series and go toward the short fiction list and not find it. I'm going off on a tangent here. This is a display issue and should probably be discussed under a different topic. MHHutchins 15:54, 17 June 2009 (UTC)

Thank you for explaining it again and in greater detail. I see what you are talking about, and agree that what you are picturing is different from our current variant linking. I'm gonna let this one percolate for a few days. Kevin 00:39, 18 June 2009 (UTC)

Link Reviews to Titles or to Publications?

There have been a few discussions of Review linking over the last year. Some editors like the fact that we currently link Reviews to Titles while other editors would prefer them linked to individual Publications. I think the fact that the Title page currently lists all reviews for the displayed Title is very valuable and we will want to preserve this functionality regardless of what else we do, but I can see value in identifying the Publication being reviewed as well.

Unfortunately, the relationship between the Title and the Publication in question is not always one-to-one since you can review only one (or two-three) Titles in a Publication. This means that we if decide to capture the Publication record that is being reviewed, we also need to capture the reviewed Title separately, which will be quite awkward. However, since in 90%+ of all cases "one reviewed Pub = one reviewed Title", we may be able to come up with a clever way to automatically link them. We'll have to think about it.

On the display side, a review line in a Publication currently has the following components:

1. Review: - hyperlinks to the Review record
2. Title - hyperlinks to the Title record
3. by Author(s) - hyperlinks to the author(s) of the reviewed Title
4. book review by Reviewer - hyperlinks to the author of the Review Title

I guess we could add components 3a and 3b to the list above, which would read "in Publication by Author", but that would make the page look very busy. Alternatively, we could replace the currently displayed Title with the reviewed Publication, but that wouldn't work too well when the review covers just one Title in a Publication. Hm, clearly this needs more work... Ahasuerus 17:16, 16 June 2009 (UTC)

Well, as swfritter pointed out, the main argument for linking to publication is for magazines and fanzines, due to the merging of EDITOR records. BLongley 18:09, 16 June 2009 (UTC)

For books, the 'review' is usually of one publication, but may specifically refer to a couple of different bindings for instance. And the review will also actually apply to all reprintings of those, until there's a new 'edition' of noteworthy change. If we get printing numbers sorted out that will help group things into this nebulous idea of 'edition', but it won't help unless we also sort out relationships between hc, tp, pb, ebook and audio versions of that edition. (At which point we might want to reconsider whether "Abridged' audio books are actually a separate edition or even title. I think they probably should be, but I'm in no hurry to rework them all, although I have tried to leave notes when I know whether it's Unabridged or Abridged.) BLongley 18:09, 16 June 2009 (UTC)

Still, I think most book reviews are of one title - e.g. I rarely see one part of an Omnibus reviewed, unless it's been entered in the magazine with separate reviews for each part rather than of the 'Omnibus' title. However, there are publications that have reviews of Shortfiction: in fact, most Magazine, Anthology and Collection reviews I read go story-by-story and don't always review all the contents. Yes, an overall opinion of the quality of the containing work is given, but really, some content titles have not been reviewed. I'm sure this is a level of detail somebody will want sometime, so might be worth considering at this point. In the meantime, I think most of our workarounds are good enough for most purposes. BLongley 18:09, 16 June 2009 (UTC)

The entries I made of James Blish's reviews in The Issue at Hand and of Damon Knight's reviews in In Search of Wonder, and still more the reviews i have yet to enter in those pubs, are largely of short fiction, and those are some of the most significant collections of critical writing on SF to date, IMO. I linked tose to the appropriate shortfiction title records, and that seems to work reasonably well, although it means more entries than a single review record for a collection or anthology would. I have seen reviews that clearly are of a particular publication, but that is rare. What is more common is a review is written shortly after a work is first published, then a revised or expanded form of the work is published, and the initial review really must be noted as being of the earlier version only. (Dates will often make this clear if a reader is alert, but can be easily overlooked.) But in such a case it is probably best if we have a separate title record, perhaps a variant, for the revised or expanded work anyway, and reviews of that version can and should IMO link to that record. Overall, I don't seem many cases when a link to a pub or editor record, rather than to a title record, would be needed. There might be some, however. -DES ^Talk 19:36, 16 June 2009 (UTC)

Novels That Were Split for Subsequent Publication

We don't have a good way of recording Novels which originally appeared in one fat volume and were later split into 2+ volumes when reprinted in paperback or perhaps in another country. We typically turn enter the latter as VTs of the original novel, but it can be misleading since it suggests that the book has been reprinted multiple times rather than in multiple parts. The situation gets particularly messy when dealing with foreign language translations, which don't even have a separate Novel Title, so we are forced to stuff them under the original Title or some such.

At one point I wondered if introducing a "Split Novel" (?) sub-type for Novels might help, but now I am thinking that we could use the "Based On/Relationship" mechanism (if and when it's implemented.) Ahasuerus 23:23, 16 June 2009 (UTC)

I think so. A split novel is in a sense just the chronological reverse of a fixup. And then there are novels like Cyteen which was split and then unsplit again, so to speak. But this would require the full many-to-many based on, also able to handle fixups, i think. -DES ^Talk 23:42, 16 June 2009 (UTC)

I often find the reverse in older novels from the 1800's. First publication in 2 to N volumes, and modern publication in novel form. Kevin 00:52, 17 June 2009 (UTC)

It's not just novels. See The Early Asimov published in one, two or three volumes. Not too bad to solve with "Based On", as there is a single volume that could be a parent of a one-to-many relationship. But I suspect there are probably two-volume sets republished as three or four but never as one, which would require the full "many-to-many" solution DES mentions. BLongley 19:25, 17 June 2009 (UTC)

And I see from the above, and from The Best of Isaac Asimov, The Best of Isaac Asimov 1939-1952, and The Best of Isaac Asimov 1954-1972, that we're currently not always using VTs for a workaround. Given that this looks like a big change to fix them all, should we be standardising the workaround for now? BLongley 19:25, 17 June 2009 (UTC)

Books/Essays about individuals

We have all kinds of records and conventions to support Title level information, including Title level Reviews displayed on Title pages, but at this time the only type of supported records about Authors is Interviews. We also have hundreds of Essays and Nonfiction books about Authors, but unless they are Interviews, there is no way to access them from that Author's Summary page. In other words, if a user wants to see a list of all books and articles about Heinlein, he is out of luck even though the information may be in the database.

We may want to consider adding this functionality -- call it "Author Level Reviews"? -- to the database. I don't think it will be that difficult to do from the technical perspective since we already support Interviews, which are quite similar structurally. Interviews also support multiple authors (both interviewers and interviewees), which we can use as a template for this feature. The only catch I can think of is Essays/books that have non-Author specific contents in addition to one or more Author Level Reviews, but we could use the same rules that we use for regular Essays and Reviews. Ahasuerus 17:04, 20 June 2009 (UTC)

I mentioned a similar idea near the (current) end of the discussion in Rules and standards discussions#Entering Reviews of items that are not 'Books' or 'Magazines' (and I am sure I saw someone else discuss it earlier, but I don't recall where). There I called the relationship "subject" and I would extend it to books or essays that are about, but are not reviews of, a work of SF. In particular book-length critical studies or critical essays that are not really "reviews" could be linked with the works they discuss, as well as biographies and works such as Heinlein in Dimension could be linked with the authors they are about. This might also be a way to link critical essays that also serve as story intros to the stories that they are about, when the title does not obviously do so (as it sometimes does not) and the link biographical/memorial essays sometimes found in a collection to the author they discuss.

I like the idea. I would not like to call it "author reviews" however, because a "review" suggests evaluating something, which many of these do not really do. I can't think of a better term than "subject" as in "Robert A. Heinlein is the subject of Heinlein in Dimension or "The Lord of the Rings is a subject (one of several) of The Road to Middle-Earth". -DES ^Talk 17:46, 20 June 2009 (UTC)

For completeness' sake I should mention that "tags" could be used to tag all books/articles about Heinlein "heinlein", all books about LOTR "Lord of the Rings", etc, but this information wouldn't be accessible from the Summary page and you would have to do a Tag Search to find it, so the added functionality would be quite limited. Ahasuerus 17:57, 20 June 2009 (UTC)

And since no editor can edit another's tags it would be fragile. There would also be no link to the subject author or work. For Heinlein that is not a major issue, perhaps, but for a less well known author, or one with a less unique name, this might reduce the value significantly. (Is a work tagged "Smith" about George O, Thorne, George R., or Valentine Michael?) -DES ^Talk 18:12, 20 June 2009 (UTC)

True enough. I think the definition of the term "review" could be stretched a bit to accommodate some borderline cases, but there may be more extreme cases where it would not be appropriate. Something to keep in mind as we ponder this area. Ahasuerus 02:27, 21 June 2009 (UTC)

I don't think a book-length critical study or biography can usefully be called a "review" and no one much speaks of a review of a person, even when a brief article, comparable in length to a standard book review, is about that person. -DES ^Talk 02:33, 21 June 2009 (UTC)

Yup, that's exactly what I meant by "more extreme cases" :-) Ahasuerus 02:49, 21 June 2009 (UTC)

I understand the complexity of the request may put it closer to the horizon than the near term, but I can see a 'Linking Essay', and a 'Linking Nonfiction' title type eventually. It could have an Author as a required data field, and a title as an optional data field. I would support such an expansion once we get some higher priority medium and largish code changes under out belts. (I think this would be the largest / most complex change discussed to date). I also imagine now that the idea is in my head, I will see a need for it more frequently than I expect. Kevin 03:45, 21 June 2009 (UTC)

I don't see the need for a separate type. I think of this as a new property that can be applied to any existing type, and can thus be added to existing records without the need to re-enter or convert them. Indeed I see this as being very similar to the "based on" concept, and it could perhaps reuse some code from that feature. Like "based on" the link would probably be at the title level, not the publication level.

Note that a given item might have more than one subject. A non-fiction book might discuss the work of several authors in detail, and each would be a "subject" of the book. Or a book might discuss several specific works, and each would be a subject.

Note that when a work deals primarily with a single work, it should probably have as its "subject" the work and not the author, although it could have both. It would be rare for a work of fiction to have a subject link, but something like Rocket to the Morgue where characters are acknowledged portrayals of actual SF authors might qualify. But I could see this being available only for non-fiction and essay titles. -DES ^Talk 05:19, 21 June 2009 (UTC)

Do you see this 'subject' property being applied to all introductions and afterwords (linking them to the title being opened or closed)? Could it also be applied to Interior and Cover art (Cover art especially, as it often has a subject, 'beyond' the magazine / Anthology / Collection it illustrates.) The tough part is imagining a single data field accepting both TITLE and AUTHOR as valid 'subject's Kevin 06:24, 21 June 2009 (UTC)

It could be so applied; whether it would be worth the coding and data entry effort to so apply it is a different matter. Most introductions and afterwords seem to be clearly enough linked to the associated works now that I doubt there would be enough practical gain to be worth introducing such links. Cover art often illustrates a particular story in a magazine, collection, or anthology, a fact we are currently not capturing in most instances. It would be possible to use such "subject" links to indicate this. Whether it would be worthwhile to do so is something we would need to discuss. There would also be the question of how and where such info would be displayed if we did capture it.

As to the matter of having two different kinds of targets: it might be necessary to have to different varieties of subject links, or to have an indication in each link as to the type of target it is for. I envision a pull-down with two choices, say "Person" and "Title". Or record numbers might be entered with a preceding letter: "P1145" or "A2278". Or some other method might be used. But this is an implementation/detailed design choice, which need not be made at this point. -DES ^Talk 14:53, 21 June 2009 (UTC)

Serial Dates - Redesign

The Titan discussion reminded me of something that I have been thinking about ever since we killed lexical match (RIP). Here is what I have arrived at so far: (some parts originally posted on Marty's Talk page):

One of the underlying issues with the way we handle serialized texts is that they have two important dates associated with them:

• "date of first serialization", and
• "date of first book appearance"

The fist date is important for obvious reasons and the second one is important to book collectors and bibliographers. You can't adequately handle both dates with a single field the way we currently do it with "Title Date". Any attempt to do so is liable to be a band-aid. Ideally, the Summary bibliography page should display something like:

Golden Blood (1933, book pub. 1964), also appeared as:
   Magazine appearances:
     Golden Blood (Part 1 of 6) (1933)
     Golden Blood (Part 2 of 6) (1933)
     Golden Blood (Complete Novel) (1943)
     etc

for serialized novels, which is similar to how Clute/Nicholls and others record these cases.

In order to make this happen issue, we can add another field to the Title record and call it First Book Publication Date. We would then have two date fields to work with, which will address the underlying issue.

The First Book Publication Date will only need to be entered if it's different from the Title Date. It will only appear on the Edit Title page and will have an appropriate label. Once we do this, we can also change our policy to allow the creation of "placeholder" Novel records for Serials that have never been reprinted so that they could be made into VTs of these placeholder Novel records. I believe it should be OK because, after the last few round of changes, the Summary page clearly states "only appeared as:" when there was no subsequent book publication. This means that we will no longer have a separate "Serial" section on the Summary page, which seems like a good thing.

If we go this route, we will need to create two conversion scripts. The first one will find all Serials that do not have a Novel parent and create one. This is somewhat problematic since there exist short serials which are nowhere near novel length and that have never reprinted. We will need to figure out what to do about them.

The second script will find all Serial/Novel VT pairs, change the Novel's Title Date to the date of the first serial (if it precedes the Novel date) and populate the First Book Publication Date with what used to be the Title Date.

In addition, there are a few relatively minor design decisions that will need to be made. First, what should we do about partial serializations similar to Titan above and Anderson's The Brain Wave? Second, if the original serialization spanned 2 years (e.g. November 1941 - February 1942), what should we enter as the Title Date?

From the software standpoint, the catch is that we haven't added new fields to our records without Al's support and there are quite a few things that will need to be done, but I think it should be doable. The rest of the software changes are fairly easy to make.

So, what do you think? Ahasuerus 03:08, 28 September 2009 (UTC)

Yes. Serials are definitely of more significance in the s-f genre; a great deal of unpublishable material was published in serial format in the early magazines. The question in my mind is whether there should be two entries per title. The serial version with all the serial information listed as above and sorted by serial date and the book listing sorted by the book publication date but without the serial information perhaps as "Golden Blood (1964, Serial pub. 1933)". My own preference is for the date of the last part. We will also have to make allowances for titles like Titan which are short fiction.--swfritter 13:04, 28 September 2009 (UTC)

I'd hold off on this for now. Because Serials and other types are different title records, I think we could (mostly) derive the desired display without database changes (although performance may suffer). (The "mostly" is because if the desired serial date is of the last part, we have to cope with missing last parts.) If we want to try a core database change, then make it a small one first: for instance, "translator" is already present in the database but not used, as is "Back Cover Artist". I'm not sure if they'd be much use but they're safer options for the first experiment. A slightly more useful one would be "Editor" support for collections. Angus Wells is exiled to notes for a lot of "Best of..." collections for instance. BLongley 19:40, 28 September 2009 (UTC)

I would think of the desired date for the serial as the date of the first installment, myself. I think the proposed design above is pretty good, but Bill may well be correct on starting more slowly. And if we are going to be messing with the date support, can we consider date ranges at the same time? Maybe not implement them at the same time, but at least keep them in mind so that a future implementation is not made harder by any changes now. -DES ^Talk 20:45, 28 September 2009 (UTC)

Perhaps this discussion should move to ISFDB:Proposed Design Changes‎? -DES ^Talk 20:45, 28 September 2009 (UTC)

(unindent)Many good points above, I'll try to answer them one by one.

• Can we accomplish the same result without database changes? There are a couple of ways to get to the "(1933, book pub. 1964)" end point. First, we could change the Summary Page logic to find the earliest known Novel publication and display it as the "book pub." date. Unfortunately, it would kill performance for Authors like Silverberg and Asimov since we will be checking both Title records and Publication records. Alternatively, we could get around one part of the problem by making the "Title Date" uneditable by humans. Instead, we would automatically update Title Date with the earliest known Publication whenever Publication level data changes. However, there are times when we don't have a Novel Publication record on file, but we do know the date of the first book publication because it is listed in a secondary source. As long as our Publication coverage remains incomplete, we can't rely on it to accurately derive Title Dates :(

• Timing. I am not suggesting that we use Serials as the test case for database changes. I think we will be better off using User Preferences, a completely new table, as our guinea pig since that way there will be no risk of breaking anything else. I just want to start a discussion of the overall approach so that we would know where we are going with Serials and add then to our road map.

• First installment vs. last installment. As DES reminded us, at some point we will want to add date ranges to Title records so that the example above will change to something like:

Golden Blood (1933-1934, book pub. 1964), also appeared as:
   Magazine appearances:
     Golden Blood (Part 1 of 6) (1933)
     Golden Blood (Part 2 of 6) (1934)
     Golden Blood (Complete Novel) (1943)
     etc

The easiest way to handle date ranges (that I can think of) will be to add a new field to the Title record and call it something like "End Date", which I think this is an argument in favor of using the date of the first installment for now. Also, there are a few cases when serializations were interrupted and not resumed until some years later, which I think is another argument in favor of using the "first installment date".

• Two entries per title. I believe there is much to be said for keeping all "instances" of the same text under the same Title, in this case by means of the VT mechanism which we currently use for Serials. Having said that, as long as the data is stored the way we want it to be stored, we can always tweak the way it is displayed and see what works best. Heck, we could even make it one of the User Preferences once we have them implemented!

• Move to ISFDB:Proposed Design Changes‎? Once we have agreed on the direction we want to take, we will probably want to move this discussion there and it will serve as a "list of requirements" to be consulted by the people who will end up implementing the new approach. Ahasuerus 00:24, 29 September 2009 (UTC)

When considering date ranges, note that two very common forms will be "Sometime before X", and "Sometime after X" where only one end of the range is notes with any precision. Granted we will usually be able to find some value for the other end -- at worst the authors birth date for a start, and the date of data entry for an end. But the before X and After X forms may be common enough for a special display and special coding to be used. this might influence design choices. I was thinking of date ranges mostly for cases where a dat isn't know with precision, but something is known. For example an undated 4th printing, known to be after the dated 2nd, and before the dated 6th. -DES ^Talk 00:36, 29 September 2009 (UTC)

For the reasons you mention, i think deriving First Book date from publications would be a mistake, and it would be better to wait until we are ready to do this as a database change. -DES ^Talk 00:36, 29 September 2009 (UTC)

Oh, that! Yes, that's a whole different can of worms. Fuzzy dates will be non-trivial to implement in a way that will support display and searches seamlessly :( Ahasuerus 15:34, 29 September 2009 (UTC)

I don't think deriving dates dynamically is practical, both for reasons of performance and incompleteness of data. Propagating information from an edit is not so bad, although the issue of how to cope with a previous value (if present) is a sticky one. --MartyD 16:52, 29 September 2009 (UTC)

I like the idea of capturing two dates. Perhaps "first serial publication" and "first complete publication"? If it were more like first magazine vs. first book, and we wanted to display both dates, we could use one order and label for "large" (say Novel, Non-Genre, and we'd let any oddball Collection or Omnibus fall into this category) works and another order and label for "small" (Short Fiction, Essay, Poem) works -- i.e., novels could be listed book-publication-date first, short fiction could be listed magazine-publication-date first. --MartyD 16:52, 29 September 2009 (UTC)

FWIW, this is an outlier, but I had a Poe play that was (a) never completed and (b) only partially serialized, where the "serialization" consisted of printing of some of the scenes in one issue, some scenes in the next issue. Later collections published those previously serialized scenes plus some additional ones. The "end" date wouldn't have worked out so well. --MartyD 16:52, 29 September 2009 (UTC)

As Marty was speaking. I think this can be done without adding any data fields. Novels and Serials would have separate master records. Make the Serial master title a variant of the Novel master title. That way you can make a db connection. At run time use that connection to collect the novel date from the serial so it can be noted in the serial entry. Vice-versa for the serial entry. Do a virtual disconnect of the variant connection and treat the the novel and serial entries as primary entries. Date ranges: overheated programmer minds at work. Use the first or last date of the earliest serial publication.--swfritter 16:55, 29 September 2009 (UTC)

The reason for date ranges is not for serials that extend over a year break, i agree that serials should use either the first or last installment date as the overall serial date (I favor the first installment, but IMO it doesn't matter that much, as long as there is a rule). The need for date ranges is for a quite separate problem, publications where we don't have exact dates, but can set upper or lower bounds on the date, or both. The only reason that I mentioned date ranges in this discussion is to ensure that if date storage or display is redesigned, it should be in a way that does not make the eventual implemtation of date ranges for such pubs harder. -DES ^Talk 17:27, 29 September 2009 (UTC)

IMO the attempt to suggest methods to determine the title date or dates (book vs serial) dynamically at display time from publication dates is fundamentally misguided. There are too many cases where we have good secondary sources to set a first title date earlier than any publication we have recorded. In all of these we would be reducing the accuracy of our info, or have to enter excessively incomplete stub publications. That is not even counting the possible performance problems. What is the rush? Lets wait until we are able to implement the database changes needed to capture separate serial and book publication dates (or whatever we eventually call them) at the title level. -DES ^Talk 17:27, 29 September 2009 (UTC)

(unindent) I am trying to visualize what Swfritter describes above and I am not sure I understand the following two sentences:

Novels and Serials would have separate master records. Make the Serial master title a variant of the Novel master title.

If all related Serials have a separate master record -- presumably also a Serial Title -- then how can that master Serial Title be a variant of the Novel master Title? That would mean that the real Serial Titles would be "grandchildren" of the Novel Title and our software doesn't support grandchildren. Am I missing something? Ahasuerus 22:14, 29 September 2009 (UTC)

But not for display purposes. The link would be broken for display purposes - after mutual data collection between serial and novel titles. Novel and serial titles would have discrete display entries. The serial grandchildren would actually display as serial children of the serial title and should be accessible from the biblio screen. Need a chalkboard. I would really like to see separate novel and serial entries sorted by date and this is one possible way to do it without doing it on the fly.--swfritter 23:17, 29 September 2009 (UTC)

Sorry, I am afraid I am still confused :( We can certainly do all kinds of things at display time -- although even the current display logic makes Marty's head hurt -- but what is the proposed underlying relationship between different Title records? Anyone got a spare chalkboard?.. Ahasuerus 17:10, 30 September 2009 (UTC)

Unified user rating of a title and its variants.

Have a look at the user ratings of these two titles: Nineteen Eighty-Four VS 1984. I've observed this eralier, but now I discovered that this novel is listed twice in the Top100 - with a quite different ranking. That is definitely weird. My suggestion is that read/write access to the ratings should be redirected to the parent title's rating - whenever a parent exists.

There may be special cases where two different ratings of one novel could be useful. Like haveing two clearly different versions. So users get a glue which version might be better. But on the other hand often these alternate versions aren't even variant titles - Heinlein's Stranger in a Strange Land to name one. --Phileas 14:21, 18 February 2010 (UTC)

Artist and Coverart handling in Pub Screen

This discussion is aimed at coming up with a proposal for fixing the problems of artist/coverart manipulation in the Pub Screen (both for adding new pubs and editing existing pubs).

For background, see any/all of:

• User talk:Mhhutchins/Archive/2010Jan-Mar#Artist credit on The Phoenix and the Mirror
• ISFDB:Community Portal/Archive/Archive18#Opinions urgently needed: handling of Multiple Cover Artists
• ISFDB:Help desk/archives/archive 12#Bug in changing cover art credit.

Before we get to proposing and discussing solutions, I'd first like to identify the complete set of problems to be solved. --MartyD 12:58, 28 February 2010 (UTC)

Problems with Artists and Coverart in the Pub Screen

Below is a list of the various limitations and inconsistencies in the Pub Screen's handling of artist credits and the underlying COVERART titles generated and maintained from those credits. Add more sections for additional problems. Comment on/clarify/embellish/challenge/etc. specific problems within the appropriate section.

Coverart title's DATE is not apparent or accessible

The coverart title record's date cannot be seen or manipulated from the pub screen. When a pub's date changes, the coverart's date does not. If the coverart title is not shared with any other publication, this is wrong (well, not the best it could be). If the coverart title is shared, this is wrong if the pub being edited is the first publication of the cover; otherwise, it is ok.

Coverart title's TEXT is not apparent or accessible

The coverart title's text (the "title") cannot be seen or manipulated from the pub screen. If a pub's title is changed, the coverart's text remains with the old title's text. This is wrong. The coverart title is meant to reflect the publication's title.

Side note A: The coverart title should be shared only by publications having the same title. If the coverart is the same across publications with different titles, variants should be used.

Multiple artists get separate coverart titles

Crediting multiple artists creates separate, individually-credited coverart titles instead of one jointly-credited coverart title (see also ISFDB:Proposed Design Changes#Cannot distinguish single joint artwork from separate individual works).

Automatic title generation inconsistent with treatment of authors

For new publications, the behavior of artist credits is not consistent with the behavior of author credits in terms of the automatic creation of underlying title records. A new publication will create a single new title record jointly credited to all listed authors.

Changing an artist's name breaks joint credit

In a pub with an existing jointly-credited coverart title, "changing" one artist's name removes the original artist from the jointly-credited title and adds a new, individually-credited coverart title for the new version of that changed name.

Cannot distinguish single joint artwork from separate individual works

It is not possible to distinguish a single piece of cover art done by more than one person from multiple pieces of cover art done by different people.

Side note B: The current behavior of creating one coverart title per artist actually makes supporting this scenario possible. One a pub is set up with multiple artists, the individually-credited title records could be left to represent multiple pieces of cover art, while the records could be combined to represent a single, jointly-credited piece of cover art. Unfortunately, that requires the most work (and most difficult work) for the far more prevalent jointly-credited case.

Cannot tell if coverart is shared by other pubs

It is not possible to tell from this screen if the artist attribution is shared among publications. Right now, manipulation of artist credits is always allowed and severs any sharing with other publications (the pub being edited gets its own, exclusive set of coverart titles).

Artist manipulation inconsistent with author manipulation in Pub and Title screens

Manipulation of artists in this screen is not consistent with manipulation of authors anywhere on this screen (whether for the pub or its contents). It is also not consistent with "author" manipulation on the title screen, where for coverart records, the artists are displayed as authors.

Proposed Solution

TBS once problem list is agreed to. --MartyD 12:58, 28 February 2010 (UTC)

Instead of posting half a dozen "I agree" notes in the sections above, I will add a note here concurring that all of the listed issues are indeed undesirable behavior. I think all or almost all of them can be traced back to the fact that we don't capture enough information about the cover art at data entry time, so the approval logic has to make guesses about what the submitter intended. Sometimes these guesses are correct and sometimes they are not.

If we agree that this is indeed the underlying problem, then the logical solution would be to start capturing more info at the time of data entry. That could be done by making "Cover Art" a multi-field section of the data entry forms rather than a single field and implementing "Add Cover Artist" and "Add Cover Art" buttons whose behavior would be similar to the "Add Author" and "Add Title" functionality that we have in the Content sections of the Edit Pub form. Following the logic currently used by the Content section, the Cover Art fields would be grayed out if the Cover Art Title record is shared by 2+ pubs. Of course, this means that we will also need to implement the ability to add/remove Cover Art Titles just like we can currently add/remove regular Titles.

As far as dates go, we could add a Date field to the Cover Art section of the Edit Pub page to make it easier to change dates. We could also add this field to the New Pub page to make it easy to enter the date of the cover art record if it predates the date of the current printing's publication. If the date field is left blank (as I expect it will be in 90%+ of all cases), then the Cover Art Title date will default to the Pub date.

For Cover Art titles, I don't think we will need to have a separate field in the New/Edit Pub forms. If the Cover Art record is used by one and only one Pub record, then the Edit Pub approval logic can automatically change the associated Cover Art title at approval time. If the Cover Art record is shared by multiple pubs, then editors won't be able to edit them anyway, but we'll have to figure out whether the submitting editor will have to use "Add Cover Art" to add a new Cover Art record or whether the approval logic can create a new record automatically.

Overall, I think/hope that this solution will not increase the amount of work when entering/editing simple pubs yet it will be flexible enough to accommodate more complex cases. Ahasuerus 17:08, 28 February 2010 (UTC)

I wanted to make a list of problems before talking about solutions, so we can compare anything we talk about to the problem list. That said, I have a strong minimalist streak. Not yet having tried to work through all of the details, my impression of this problem is that it stems from trying to provide a too-simplistic interface ("Artist" as a simple value) on the Pub screen to capable-but-complex underlying functionality (titles). Ignoring the very important issue of simplicity, my other thought is that "Content" title handling in the Pub screen would deal with just about all of the problems -- it handles shared/not shared, would distinguish one piece of artwork from multiple pieces, shows dates and title text, etc. So my inclination is to try to reuse that mechanism while finding a way to provide some level of simple interface, too. --MartyD 12:23, 1 March 2010 (UTC)

Enhanced support for other languages

Here is our current (well, it's been "current" for about 3 years) devious plan to enhance support for other languages:

1. Create a new table, "languages". The MARC-21 standard maintained by the Library of Congress has a nice list of languages, but other options can be considered as well.
2. Add a new field, "Language", to the Title record and update the data entry forms to accommodate this new field.
3. Change Help to tell users to enter foreign language translations as Titles (rather than as Pubs under the canonical Title) and then create VTs.
4. Add a User Preference page which will let each user decide which languages he wants to see. "Show all" may be an option and at least one moderator wants a "Suppress all foreign language originals" check box, but that may be harder to do.
5. Make sure that any users who are not logged in still see "Originals+English translations", i.e. preserve the current behavior for casual browsers.
6. For any logged in user who has selected his preferred language(s), show the original title and any VTs whose language matches his preferred language(s).
7. Create an option to show all VTs in all languages for a given author or title -- this will accommodate "single author collectors" without forcing them to wade through lots of irrelevant content.
8. Display something like "Log in to see other languages" at the top of each page or in the navbar to encourage more users to log in.
9. Display the language code for foreign language VTs

Additional functionality that we will get out of this:

1. Adds the ability to search for foreign language titles
2. Adds support for translations of short fiction titles
3. Lets us easily distinguish between, say, the English and the Polish versions of Solaris.

Known limitations:

1. The thorny issue of translators will be ignored for now -- pseudonymous translators would be a pain to implement and will require more thought. I'd love to implement it at the same time as the language codes, but it would likely overwhelm the project.
2. Latin-2 (and especially Cyrillic) languages will still require transliteration.
3. Multilingual books will not be well supported.

Things to consider:

1. What do we do about foreign language collections/omnibuses that have no analogs (no pun intended) in the original language? There is any number of foreign language Heinlein/Asimov/etc collections which collect their stories more or less randomly and do not have a matching canonical title in English.
2. How many languages do we want to support? The MARC-21 standard support a lot of them, from Sanskrit to Klingon, but do we really need all of them? And if not, how do we decide what languages would be safe to drop? I guess we could start with all European and major non-European languages and add more languages as needed.

Anything that I am missing? Ahasuerus 22:40, 5 October 2009 (UTC)

I don't understand why a translator field raises particular problems. Wouldn't this use the same logic as an author or editor field, simply being one or more names from the author (person) table? I'm sure I am missing some aspect which makes it harder than i am thinking, but could you expand of the issue a bit? -DES ^Talk 23:20, 5 October 2009 (UTC)

You are quite right that translators will simply go into the Author table. Some of them are fiction writers/editors in their own right, e.g. Brian Stableford, and their Summary page will need to have a separate section for "Translations". That's not particularly difficult to do, though.

The complexity lies in the fact that there is only one type of relationship between Titles and Authors at this time, something that is deeply embedded in our software, and we will need to add another relationship between Titles and Authors to support translators. Think of all the complexities that are associated with handling Authors, e.g. all those nooks and crannies where we need to display things like:

Trudno Byt' Bogom (1964) with Boris Strugatsky  only appeared as:
   * Variant Title: Hard to Be a God (1973) [as by Boris Strugatski and Arkadi Strugatski ] 

and then visualize adding another layer of complexity on top of that:

Trudno Byt' Bogom (1964) with Boris Strugatsky  only appeared as:
   * Variant Title: Hard to Be a God (1973) [as by Boris Strugatski and Arkadi Strugatski ] [tr. by Joe Schmoe and John Doe [as by John Q. Public]]

In addition, individual translations are effectively "derivative titles" and, in theory, each one needs a VT. So what do we do when dealing with, say, the Italian translations of Edwin A. Abbott's Flatland? There are translations by Masolino D'Amico, Marisa Nascimbeni, Marisa Nascimbeni, Guido Marè and possibly others. Do they all get a separate VT since they are all separate texts? Or will it make the Summary page look so busy that it will become hard to navigate? On the other hand, if we set up only one Title record per language, how do we know which translation appeared where and how do we create "translator bibliography" sections on the Summary page?

Another issue to consider is that there are other "person roles" that we would like to add to the software, e.g. "Editor of a single author book" and "Screenplay writer". If we decide to implement translators, we should probably do it as a step on the way to full support of "roles", which imposes additional constraints. Of course, "roles" may be associated with Titles as well as with Publications, e.g. "cover design by", so that's something else to consider. And so on and so forth.

None of it is insurmountable, but it will take quite a bit of time to design, program and test. Ahasuerus 00:40, 6 October 2009 (UTC)

Ah, I see. Yes that is a significant bite. I can understand better now why you want to delay translator support, or at least consider doing so. -DES ^Talk 00:58, 6 October 2009 (UTC)

Actually, there is a 2nd existing relationship between Titles and Authors: interviews (for the interviewee). Book reviews may be a different relationship also, I'm not sure. But I don't advise a similar special case for translators, or probably not. -DES ^Talk 01:01, 6 October 2009 (UTC)

We have a table for "title relationships", which includes fields for reviews, series and translations, but only the review column is currently used. We also have a column in the (horribly misnamed) "canonical author" table, which specifies whether a title-author relationship is for a regular title, a review or an interview. It's quite messy, really, and at least parts of it will need to be redone if translator support is implemented. (We also have an old "translation" field in the Title table which stores a list of translations for the title, but it's wholly inadequate, sparsely populated and no longer displayed.)

As an aside, one of the problems that we face is that some parts of the code have reached the point where the software has become "fragile". The code works, but some parts are so convoluted that if a "naive" programmer tries to add more complexity to them, he will likely break something. I have rewritten, commented and simplified a few sections, but it's very much a work in progress and I am not exactly the world's most knowledgeable programmer. Then there are areas that are just inherently fairly complex, e.g. there are some sections where the Python codes generates JavaScript code which then drives all those buttons that magically appear when you click on "Add Title" or "Add Author". Oh well, we'll muddle through. We always do :) Ahasuerus 02:24, 6 October 2009 (UTC)

(Unindent) As you said elsewhere - "Oh well, one step at a time...". I think we can add the data capture fairly easily. Default all new pub entries to English and no translator, but allow other choices. If any are entered that aren't English or have been translated, let's have that data NOW. It will be easier to work with later. Yes, I'm particularly looking at Ernestoveg - don't make him reenter everything, or make Mike Hutchins re-approve everything. BLongley 22:45, 6 October 2009 (UTC)

Well, yes - why do you think foreign language support has suddenly moved to the top of the list of priorities? ;-) Ahasuerus 22:50, 6 October 2009 (UTC)

Substantively, I expect that we will roll most of the new features out one at a time, but some will have to be bundled since they won't work in isolation. Ahasuerus 22:57, 6 October 2009 (UTC)

Hi, I would have the following comments / proposals to this topic:

• Additional functionality:
  • we could support also a new stat to query the titles translated to the most languages (nice to have) GaborLajos 12:08, 1 June 2010 (UTC)

That's a good point, I haven't thought of that. Ahasuerus 00:26, 2 June 2010 (UTC)

• Known limitations:
  • 1) I would add the translators to the existing author table and make a new join table to maintain the relationship (n:m) between the titles and the translators. The join table should maintain the language of the given translation using the language code (based on the language code stored in the table you created already). Although this new join table is the most complex part of this enhancement but I think we cannot skip it. GaborLajos 12:08, 1 June 2010 (UTC)

I am rather torn re: this one. Doing it right undoubtedly involves adding support for translators, but it would also make the project noticeably more complex. Ahasuerus 00:26, 2 June 2010 (UTC)

Sorry. I was wrong. The new join table mentioned above is NOT needed at all. I have already a working prototype for translation without adding any new table. The only database changes I made: 1) add a new author_language field to the authors table, 2) add a new title_language field to the titles table, 3) using a new constant value (translator=4) in the canonical_author table for the ca_status. GaborLajos 09:37, 6 June 2010 (UTC)

• • Multilingual books should be handled as special omnibus editions. GaborLajos 12:08, 1 June 2010 (UTC)

Given how few truly multilingual books we have in the field, I am not sure the overhead is worth it. It brings up another issue, though: Titles will obviously have language codes associated with them, but what about pubs? I know of a number of books, mostly for students of foreign languages, where the introduction is written in a different language. Ahasuerus 00:26, 2 June 2010 (UTC)

• Things to consider:
  • I would like to support the foreign language omnibuses as well, because without them the db will not be complete. I agree we should solve the problem that they are mixed now with the original language omnibuses - I would exclude them from the default omnibus list displayed on the author page GaborLajos 12:08, 1 June 2010 (UTC)

This can be solved by assigning languages to authors and suppressing translated omnibuses unless the current user chooses to see the language of the translation. Ahasuerus 00:26, 2 June 2010 (UTC)

• • I would propose to support only limited scope of the possible languages, but I would implement the architecture which is capable of handle all the languages GaborLajos 12:08, 1 June 2010 (UTC)

You may want to take a look at the current list, which was based on my review of the OCLC and LOC data. It's not guaranteed to be 100% complete, but it's a start and it probably covers over 99.99% of all genre books. Ahasuerus 00:26, 2 June 2010 (UTC)

Yes. I know this list and for me seems to be ok - Hungarian is included :). I tried out how this works if I add a simple drop-down menu with all these languages to the author table and the size of the list is still user-friendly, but if we would extend the languages table content to include all the ISO language codes, then we would need a more complicated, complex UI to allow the user to easily choose the language. GaborLajos 09:39, 4 June 2010 (UTC)

• Further comments:
  • We should not use the old translation field in the title table anymore
  • We could extend relationship types in the canonical author table with the "4=translator" relationship (nice to have)
  • We could add a new optional language code field to the author table to indicate the native language of the author (nice to have)

BR, GaborLajos 12:08, 1 June 2010 (UTC)

As indicated above, assigning languages to authors can help address other problems as well. One word of caution re: "native languages", though. Some authors, especially in places like Quebec and Europe, use multiple "working languages", e.g. Jean-Louis Trudel and Vladimir Savchenko. We may have to allow multiple "native languages". Ahasuerus 00:26, 2 June 2010 (UTC)

You are right, but I would not implement n:m relationship between the author and languages - in practise the maximum number of the "working languages" could be 2 or 3. And in many cases you can select a "native language" from the "working languages", e.g.: in case of Milan Kundera the working languages are the czech and french, but the native language is czech. To solve this issue I would propose to introduce the original language field for the titles: in the mentioned example "Milan Kundera"'s language would be Czech, his early books will have "Czech" as original language and his newer books would have "French" as original language. Supporting more than one native languages for the author table could be an enhancement option for the future, but I would not say that this is prio1 and would not implement in the phase1. FYI: this week I downloaded and installed a local isfdb server to play around with the enhancement options of this translation / translator support. GaborLajos 09:39, 4 June 2010 (UTC)

Keep in mind that the primary reason why we want to capture the author's "working language(s)" is to help distinguish between original and translated "container titles", i.e. collections/omnibuses/chapterbooks/anthologies. As long as the translated container title has a matching canonical title in the author's working language, we will enter it as a Variant Title (VT) and the standard rules for suppressing the display of VTs that the current user is not interested in will apply. It's only when you run into a Hungarian, French, Russian, Japanese, etc omnibus/collection/chapterbook of, say, Heinlein's works that does not map on to an English omnibus/collection/chapterbook that you have no way to suppress its display unless there is an indication that English was Heinlein's working language.

This is not a problem for translated novels since there is always (well, almost always) a matching "original language" Novel title, so the translated title will be entered as a Variant Title and the standard rules will apply. Ahasuerus 18:13, 6 June 2010 (UTC)

(unindent) As an aside, we may want to create a separate sub-section for User Interface (UI) issues. The two screen snapshots above look like a good start, but we need to figure out how to show magazine appearances that are also translations, e.g. see George O. Smith's Lost in Space. Ahasuerus 18:29, 6 June 2010 (UTC)

Yes. I know the issue with the collections/omnibuses/anthologies. I think we have options to solve this:

1. We add one more language field to the authors table. And the collections / omnibuses / anthologies can be filtered based on the language code of the author and the title.
2. We should correct the authors of these collections/omnibuses/anthologies. I mean in many cases - and specially in case of foreign language omnibuses/anthologies/... the author of the title should not be the author(s) of the novels/stories collected in these titles, but they should have "editors" instead of "authors". Let us check for example the following omnibus: Neiromant, I am sure that William Gibson had nothing to do with this Russian edition, this had a Russian editor, or maybe it was printed without mentioning the editor, but this does not mean that Gibson played any role in addition to being the author of the novels in the book - which role is already maintained and mentioned elsewhere and not in the author field of the omnibus title.
3. There is one more option: we do not need author language(s) if we assume that the foreign language omnibus/collection/anthology contains translations, because after implementing this enhancement for the support of the translations, we can easily doublecheck if an omnibus/collection/... contains translations or original titles. If it contains translations instead of original titles we could exclude it from the author's title list.

So I think the root cause of the problem is the incorrect database content... these titles were not added with the right author. Please note that all these options require addition database adjustments: in the first case all the language fields has to be set manually for these titles, in the second case the authors should be corrected manually, in the third case the translations has to be added as "translations" and not as variant titles. To implement automatic solution for these database migrations would be very risky. I would prefer the second option, if this is not ok, then the third one... the first one seems to be the ugliest workaround to solve the issue, that author/editor field of the existing omnibuses/collections/... is not maintained correctly. (btw: if you create a new omnibus, then the author field appears as "editor" on the Add Omnibus form, but later the same field displayed as "author" on the title.cgi.

Well, you could argue that any posthumous collection (translated or otherwise) is almost invariably 100% the work of the editor, but that's a very different approach to attributing books than the one that we are currently using and the one that other bibliographies and catalogs use. Ideally, we would have a separate field for "single author book" editors so that we could capture both the author and the editor for the same Title, something that is supported by library catalogs (see the 7XX fields in MARC21) and has been repeatedly requested. Ahasuerus 22:26, 6 June 2010 (UTC)

I could not find the "single author book editor" in the referenced link, but I could find many other relator codes and in general we could easily support them by using the ca_status field of the canonical_author table. Why do not we support them? (aut, aqt, aft, aui, bjd, clb, col, com, ctb, cph, cov, edt, ill, ive, ivr, pht, trl) GaborLajos 11:34, 7 June 2010 (UTC)

There is an outstanding Feature Request (see [https://sourceforge.net/tracker/?func=detail&aid=2811856&group_id=199503&atid=969757 FR 2811856) to implement "roles" and there is broad support for it, but it would require a fairly substantial reworking of the current display logic, which is rather convoluted in places. Keep in mind that we have very few active developers on the project and their ISFDB time is quite limited, so bigger tasks like "roles" and "foreign language support" are very resource-intensive for us. Ahasuerus 02:41, 8 June 2010 (UTC)

If you don't mind and this could help in the resource problems I could join to the project as developer and take over the translation topic. Of course I would not mixed this with the enhancement request for the generic "role concept". GaborLajos 15:38, 9 June 2010 (UTC)

Oh, a volunteer! Grab him before he has a chance to change his mind! :-)

More developers would certainly be a very welcome, well, development, but translations are a big area and it would be hard to start with something that complex. Do you think you could pick something smaller as a test project that would let you get used to the application, the development process and the underlying database? For example, there is an outstanding request to split the data that is currently stored in the "storylen" field of the Titles table into smaller fields. Perhaps you could add a field for "nvz" and a field for "jvn" for starters? If you are feeling adventurous, you could also add a field for omnibus contents, but that would require making changes to more scripts, including the Publication Edit script. Ahasuerus 22:54, 9 June 2010 (UTC)

I read through the request you proposed but I am not convinced that the solution proposal described in the mention request is really the right way and I do not think that this request would have higher prio than the translation handling or the introduction of new roles. I would leave this request to somebody who has no doubt that it makes sense to implement it in this form. BTW: I understood that you do not want to take the risk that my translation implementation is not working properly, therefore I propose that I document the proposed changes and then it is up to you if you merge them into the source or not. Of course, additional testing would be helpful and could reduce the risk you are afraid of. GaborLajos 08:22, 16 June 2010 (UTC)

Creating detailed requirements and a detailed design outline will certainly lower the risk, but the biggest problem is the scope of the proposed change. We are talking about something that will touch every ISFDB script that handles titles -- title merges, summary bibliography, the works. It would be very hard for a developer who has never worked with the application to pull off something of that magnitude as his first project. We all started small here -- a minor change here and a minor change there -- and then we slowly graduated to bigger and more ambitious projects, a path which mitigated against the risks that you mentioned. For example, there were major high profile flaws in the application when I started working on it, but I didn't tackle them until after I had a few dozen relatively small changes under my belt.

Having said that, we have a CVS repository, so the exposure will be limited. Whichever project you choose as your first one, your changes will be uploaded to the repository, then downloaded to the test systems maintained by other developers/testers and tested there. If testing finds problems with the changes, then they will never be installed on the live server and no harm will be done. Still, it will take a significant amount of time to review the code and to test everything given the proposed scope. It will also complicate other development efforts while there are different versions of the code in the CVS repository.

Another approach that we may want to consider is simply zipping up your version of the source code after you implement your changes locally and sending them to a tester for review. We do have a volunteer reviewer/tester in this case, so if he is willing to work with you on testing/debugging, the risk drops to almost zero: a win-win situation :-) There may be some extra work associated with reconciling the completed changes into CVS, but nothing insurmountable. Ahasuerus 04:29, 18 June 2010 (UTC)

This idea sounds good. I'll create a zip version and send to you... I am working on the translations support for "publications" now... translations for titles are already working. Thanks. BR. GaborLajos 12:03, 18 June 2010 (UTC)

I think that for the purposes of this discussion, adding a new "working language" field to the Author table should be adequate, although it may have to allow multiple semicolon-delimited values similar to some other fields in that table. Ahasuerus 22:26, 6 June 2010 (UTC)

And what's your opinion about the third option I mentioned? I think this would be a better workaround than using the author's language field(s). GaborLajos 10:43, 7 June 2010 (UTC)

If I understand it correctly, the idea is to dynamically retrieve all Title records associated with each omnibus/collection/etc, examine their language field and then decide whether to display the omnibus/collection based on this information. I don't think it would work, in part because of the added performance overhead and in part because omnibuses/collections are not directly associated with their constituent Title record. They are associated with Publication records, which are in turn associated with Content Title records, so it's not a direct relationship. Ahasuerus 17:06, 7 June 2010 (UTC)

BTW how many language fields do you intent to support on the author's level? I would propose max two or three fields. Do you know any example for authors working in 4 or more languages?

I try to summarize the databases changes here:

ALTER TABLE titles ADD title_language VARCHAR(3) DEFAULT NULL;
ALTER TABLE authors ADD author_language VARCHAR(3) DEFAULT NULL;  #not used at the moment in the program logic
ALTER TABLE authors ADD author_language2 VARCHAR(3) DEFAULT NULL; #if needed

Please note that I used the lang_code instead of the lang_id as foreign key, because it is unique and much more admin friendly... and default null is used for sake of backward compatibility with the current solution and database content. GaborLajos 10:43, 7 June 2010 (UTC)

Realistically, two languages should be sufficient for our purposes, but creating multiple fields for the same type of data is generally asking for trouble. As far using "lang_code" instead of "lang_id" goes, it's possible that we may want to change language codes in the future -- after all, the Library of Congress did it at one point. Besides, primary keys should be generally used as foreign keys unless there is a very good reason not to. Ahasuerus 02:53, 8 June 2010 (UTC)

A quick comment on the "key" aspect of this: I agree with the above -- permanent unique IDs should be used for foreign keys, not anything that could possibly change. --MartyD 10:44, 8 June 2010 (UTC)

Although I cannot see and high risk here - considering that ISO code changes do not belong to the daily work of the Library of Congress either, and migrating the ISO code once a year or once in 5 years is not a big deal - I can accept that you are on a more conservative standpoint and I'll change the title_language and author_language types from VARCHAR(3) to INT(11). GaborLajos 15:38, 9 June 2010 (UTC)

There is a variety of ISO and ANSI standards that apply to the world of bibliography, but in this case I was referring to the MARC21 formats, which are maintained by the Library of Congress. If you check their list of language codes, you will see that quite a few language codes have been changed over the years. Ahasuerus 22:23, 9 June 2010 (UTC)

However, the important thing to keep in mind is the principle of using immutable unique IDs for primary keys and then using primary keys as foreign keys. That's a basic tenet of database design and we can't make any exceptions to this rule. Ahasuerus 22:23, 9 June 2010 (UTC)

Well, we break that rule already with the Verification Sources. I wouldn't encourage more breaking of that rule, but fixing that exception doesn't look impossible (just a little risky). And Publication Tags as well as Pub IDs is a bit of a no-no. But as DES pointed out, we may be stuck with those. BLongley 23:57, 9 June 2010 (UTC)

Indeed, Verification Sources are not as clean as the rest of the underlying tables, but we certainly won't be repeating that mistake again and may even fix what we have wrought in the fullness of time. In addition, at least one table is horribly misnamed, but that would be somewhat harder to fix. Ahasuerus 01:23, 10 June 2010 (UTC)

BTW: Is there any architecture guide, where we could document the outcome of this discussion as design decisions for the implementation? and maybe store the UI mockups, database schemas related to this topic? BR. GaborLajos 20:46, 6 June 2010 (UTC)

There is ISFDB Design Documentation, which may be slightly out of date, but should be generally adequate. Ahasuerus 22:26, 6 June 2010 (UTC)

I created a new "Translation" requirement section in the design documentation and began to summarize the outcome of this discussion and the results of my prototyping. Please review and extend it. Note that I had no time to include everything... maybe in one week it will be more complete. GaborLajos 08:00, 13 June 2010 (UTC)

I have updated the "current situation" section and will try to update the requirements tomorrow. Ahasuerus 05:18, 19 June 2010 (UTC)

Foreign language support - design

Starting a new section to discuss the specifics of the proposed approach. Once we agree on what the new functionality will look like, we will update the Requirements page.

General principles

All titles will be entered as Title records regardless of the language. The practice of adding translations to Titles as publication records will be discontinued. All titles will have an optional title field associated with them. This will let the application support translations of short fiction, essays, poems, etc.

The only languages that ISFDB will support are the one that users can select in Language Preferences. If additional languages are identified as required for ISFDB purposes, the list can be expanded by ISFDB developers programmatically. All supported languages will be selectable from drop-down lists in edit options.

Users who are not signed in will see canonical titles (see below for the way translated collections and omnibuses with no original language analog will be handled) and English translations only. They will not see translations to other languages.

Users who are signed in will see canonical titles, English translations and translations to the languages that they selected in Language Preferences.

Authors will have a new role, "translator". Any number of translators (0 to N) can be associated with a given Variant Title. Author bibliographies will include a "Translations" section for titles that were translated by the author. Title pages will list the translator(s) of the currently displayed title if one is defined.

Pseudonymous translators will be supported in a way similar to the way pseudonymous authors and editors are currently supported -- more details required.

The pseudonmous translators are pseudonymous authors, there is no change in this sense in the db architecture or the concept. The translation support should not have any impact on this. But if you could describe me an example, I will test it. GaborLajos 19:18, 20 June 2010 (UTC)

The "pseudonyms" table and its associated scripts (Make/Remove Pseudonym etc) will not be affected, that's true. However, pseudonym assignment also happens at the title level using the VT mechanism. It's a little unwieldy, but it's necessary to accommodate joint pseudonyms, house names, host writers, etc.

To use the book that I mentioned on your Talk page as an example, there will be three Title records when all is said and done:

• Sargasso of Space, the canonical English title
• Sargassy v Kosmose, a VT whose language code will be set to "Russian" and whose translators will be set to "S. Berezhkov" and "S. Vitin" (in canonical_author)
• Sargassy v Kosmose, another title record whose language code will be set to "Russian" and whose translators will be set to "Arkady Strugatsky" and "Boris Strugatsky"

The question then is how do we link these three titles so that the final result looks like this:

Sargasso of Space (1955) also appeared as:
  * Variant Title: Sargassy v Kosmose (1969) [tr. by Boris Strugatsky and Arkady Strugatsky ] [as by S. Berezhkov and S. Vitin]

I can't think of a way to do this with just "titles" and "canonical_author". We may have to leverage "title_relationships", but then it can get complicated and hard to maintain. Oh well, food for thought. Ahasuerus 22:55, 20 June 2010 (UTC)

Tested with the following result:

   *  Solar Queen
         o The Solar Queen (2003) [O/1,2]
         o 1 Sargasso of Space (1955) [also as by Andrew North ]
               + Russian Translation: Sargassy v Kosmose (1969) translated by S. Vitin and S. Berezhkov 
         o 2 Plague Ship (1956) also appeared as:
               + Variant Title: Plague Ship (1956) [as by Andrew North ]
               + Variant Title: Plague Ship (2006) [as by Andre Alice Norton ] 
               + Magazine Appearances:
               + Plague Ship (Complete Novel) (1957) [as by Andrew North ] 
         o 3 Voodoo Planet (1959) [SF] [also as by Andrew North ]
         o 4 Postmarked the Stars (1969)
         o 5 Redline the Stars (1993) with P. M. Griffin
         o 6 Derelict for Trade (1997) with Sherwood Smith
         o 7 A Mind For Trade (1997) with Sherwood Smith

Here you can click on the S. Vitin and S.Berezhkov names and then you will get the pseudoname info referring / forwarding to the Stugackij brothers. GaborLajos 21:30, 22 June 2010 (UTC)

This approach works as long as the pseudonym is used by just one person. It doesn't work for house names (see, e.g., Alexander Blade) or ghost writers, which is why we had to add support for pseudonyms at the VT level in the first place. Ahasuerus 23:51, 22 June 2010 (UTC)

Hmm... maybe there is some misunderstanding here. The solution I described is working exactly the same way as all the existing pseudonyms are working in the isfdb right now. For example: Randall Garrett page which refers the pseudonym mentioned in your example, will use the pseudonym the same way as my example does. GaborLajos 08:40, 23 June 2010 (UTC)

Since pseudonyms like "S. Berezhkov" can be shared by multiple people (separately or jointly), it's not enough to identify the pseudonym, we have to identify the person behind the pseudonym as well. Therefore the desired behavior in this case is as follows:

 Sargasso of Space (1955) also appeared as:
  * Variant Title: Sargassy v Kosmose (1969) [tr. by Boris Strugatsky and Arkady Strugatsky ] [as by S. Berezhkov and S. Vitin]

This can only be accomplished by identifying pseudonyms on a title by title basis -- the Author level that the current approach is using is insufficient. It should be doable as yet another tweak to the database, we just need to think of the easiest and least intrusive way to do it. Ahasuerus 13:00, 23 June 2010 (UTC)

Is this a completely new requirement for the pseudonames or are we talking about an existing feature for VTs? In the latter case I can apply the same logic for translations as we had for the VTs before, just give me an example where the desired behaviour you described is already working for VTs. I mean I need an example where the VT has two authors and at least one of them used pseudonym -> if I select this VT in the author page of one author, then the other author should be listed with both pseudoname and canonical name after the VT as you described. If this is a completely new requirement and was never implemented for VTs either I would separate it from the translation enhancement. GaborLajos 18:58, 23 June 2010 (UTC)

I think this feature was never working for VTs, either. I made a short test:

(unindent) I created a new pseudoname for Bruce Sterling as Paul French. This pseudoname was already used by Isaac Asimov as well. Now I changed The Difference Machine assuming first that they released also under pseudoname:

Here you have no information in the VT line who Paul French is. Maybe you can find out from the canonical name, but if I would change William Gibson's name in the VT to a pseudoname, then you would not have any info who is who. In second test I assumed that The Difference Machine was released only under pseudoname of Bruce Sterling. In this case you have no info at all, if Paul is Isaac Asimov or Bruce Sterling.

Clicking on the name does not help either:

So I think this feauture is not something I missed during the translation enhancement, but something which is totally missing from the current isfdb database structure. On the other hand I do not think that this is a very serious issue, because the mentioned test cases are very extreme ones, I could not find any real example, only create one manually. I think this complex scenario might affect the less than 1% of the translations (translation with pseudoname where the same pseudoname is used by more than one author, not forgetting that translator usually do not use pseudonames at all...). Considering that this is an exception and not the normal case In these cases I would propose to use the well-known "Note" field and to document the real, canonical name of the translator in there. GaborLajos 19:26, 23 June 2010 (UTC)

I don't think we are on the same page yet. Perhaps the confusion comes from the fact that I used two translators in my example even though this problem can happen with any number of translators. To simplify, let's drop one of them:

 Sargasso of Space (1955) also appeared as:
  * Variant Title: Sargassy v Kosmose (1969) [tr. by Arkady Strugatsky ] [as by S. Vitin]

Using the current VT approach to capturing variants of the same title, we would have to have 3 Title records:

• Title: "Sargasso of Space"; author: Andre Norton; translator: [none] <-- canonical title
• Title: "Sargassy v Kosmose"; author: Andre Norton; translator: S. Vitin <-- VT
• Title: "Sargassy v Kosmose"; author: Andre Norton; translator: Arkady Strugatsky <-- VT

If we had only one VT, the one that uses "S. Vitin", and if "S. Vitin" was shared by multiple people, then there would be no way of telling who is behind the pseudonym. On the other hand, if we use the two VTs listed above, I can't think of a way to link them in a way that the software could understand.

Granted, we don't have to use a separate VT to capture translators. We could create a new table or add fields to an existing table, we just need to come up with a way to do it without messing up everything else or making the code so complicated that it is no longer maintainable. Ahasuerus 04:20, 24 June 2010 (UTC)

New data elements and values

Agreed upon data elements:

• Author table: 2 new fields for "working languages"
• Title table: 1 new field for the language of the title
• Title-author table (the increasingly misnamed "canonical_author" table): Add a new value for translators

Data elements under discussion:

• Publication table: do we need to capture the language of the publication as opposed to the language of its title(s)?

I think we do not need "language" field on the publication level. The solution proposed on the title level is generic enough to support all the possible use cases: we are able to store more than one different translation with the same language, - either with different translators or with the same translators but different title (foreign language VT) GaborLajos 19:11, 20 June 2010 (UTC)

Maybe one more comment to clarify this issue: the Language field appears for example on the New Pub screen (e.g. if you create a new omnibus edition), but the content of this field will be stored in the title level and not on the pub level. GaborLajos 19:35, 20 June 2010 (UTC)

Sure, that's how NONGENRE is currently handled in NewPub. Ahasuerus 23:29, 20 June 2010 (UTC)

Arguments for:

• This will facilitate limiting the display of publications on the Title page to the languages that the user is interested in. Ahasuerus 15:50, 19 June 2010 (UTC)

Not needed. This feature can be easily implemented without publication level "language" codes. GaborLajos 19:11, 20 June 2010 (UTC)

Well, it can be done by retrieving the "reference title" of each publication and using it to decide whether to display the pub or not, but some publications do not have a reference title. Not a big deal though. Ahasuerus 23:29, 20 June 2010 (UTC)

Arguments against:

• In most cases we are only interested in the language of the "reference title" in the publication. Maintaining another language field will increase the amount of work that editors will have to do and increase the likelihood of pubs and titles getting out of sync. Ahasuerus 15:50, 19 June 2010 (UTC)

Display behavior - Author bibliographies

The desired behavior of the Summary and Chronological pages is as follows.

If the user is not signed in or if the user's Language Preferences are not set up, then the page will only display canonical titles, VTs whose language code is set to "English" and VTs whose language is not defined. For Collections and Omnibuses, canonical titles are only displayed if one of the following conditions applies:

• The language code of the canonical title matches one of the language codes for the author's "working languages"
• The author's "working languages" codes are not defined
• The canonical title's language code is not defined

If the user is signed in and has defined Language Preferences:

• Canonical titles for title types other than Collections and Omnibuses are always displayed regardless of user preferences.
• Canonical titles for Collections and Omnibuses are only displayed if one of the following conditions applies:
  • The language code of the canonical title matches one of the language codes for the author's "working languages"
  • The author's "working languages" codes are not defined
  • The canonical title's language code is not defined
• If the language code is not defined for a VT, then the VT will be displayed
• If the user's Language Preferences are set to display all languages, then all VTs will be displayed regardless of the VT's language
• If the current user's Language Preferences are defined to display a set of preferred languages, then only the VTs whose language codes match one of the preferred languages will be displayed

The Alphabetical page, which currently displays canonical titles only, will continue to ignore VTs, at least for now.

The Awards page will not be affected and continue to display all awards for the author.

Ahasuerus 15:50, 19 June 2010 (UTC)

Display behavior - Title page

The desired behavior of the Title page is as follows.

• All titles, canonical as well as VTs, will display the title's language if it is defined.
• If the title is a VT and the language of the canonical title differs from the language of the VT, then the language of the canonical title will be displayed in a separate field called "original language"
• If the title is a canonical title and has VTs whose language codes do not match the canonical title's language code, then the VTs will be displayed in a separate section called "Translations"
• If the title is a VT and has translator(s) defined, then the translators will be displayed in a separate field

Ahasuerus 16:18, 19 June 2010 (UTC)

Outstanding issues:

• If a VT is both a serialization and a translation, should it be displayed in the "Magazine appearances" section or in the "Translations" section?

I would propose to use a dedicated "Foreign language magazine appearances" section. GaborLajos 19:41, 20 June 2010 (UTC)

OK, sounds reasonable. Ahasuerus 23:26, 20 June 2010 (UTC)

• Should the Title page display Publications whose languages do not match the current user's preferred languages (see the data element discussion at the beginning of this section)

This issue is not relevant if we define the "Language" on title level only. GaborLajos 19:43, 20 June 2010 (UTC)

Ahasuerus 16:18, 19 June 2010 (UTC)

Edit Behavior

A few thoughts:

• The option "Add a Variant Title or Pseudonymous Work to This Title" is rarely used. There is no harm in extending it to support translations, but the option that is usually used to add data to an existing title is "Add Publication to This Title", so we will presumably need to beef it up or clone it to support adding translated pubs. Ahasuerus 00:06, 21 June 2010 (UTC)

I would not like to mix the "Add Translation" and "Add Variant Title" options in one menu item because of the following reason: the user should clearly know if it is about a variant title or a translation, but from the cgi script we cannot decide if he/she wants to add a translation or variant title if we do not separate them in the menu. The result would be that we should allow the user to edit both the author fields / original title fields and the translator / translated title fields. If the user changes both the title and the translated title (or both the author and the translator) then we should create VT and a translation parallel to clean up this situation (or stop with some error message), but this would result a much more complex program logic and user interface / guidelines. GaborLajos 13:41, 21 June 2010 (UTC)

I am not sure we are talking about the same scenario. Let me take a step back and describe the common ISFDB use cases and how they will be affected by the addition of translations:

• Use case #1. An editor wants to enter a publication. He uses one of the "Add New ..." options in the navigation bar on the left and enters publication-specific data. When the submission is approved, a new Title record and a new Publication record are created. If the user entered additional titles in the Content, then corresponding Title records were created. When foreign language support is implemented, we will need to add a new field to the New Pub form to capture the language of the main Title.

• Use case #2. An editor realizes that Title A is a variant of Title B. He uses "Make This Title a Variant Title or Pseudonymous Work" to link the two titles. When foreign language support is implemented, this functionality will continue to work "as is" and allow editors to link translations to their canonical titles.

• Use case #3. An editor wants to add a publication to an existing title and the author(s)/title of the publication are exactly the same as the author(s)/title of the title. He uses the "Add Publication to This Title" option and enters publication level data. This option will also continue to work "as is".

• Use case #4. An editor wants to enter a publication to an existing title, but the pub's author(s)/title are different than the canonical and all existing variant titles that ISFDB already has on file. In a case like that the editor has to follow one of a two-step approaches. He can either:
  • enter the pub via New Pub, wait for the approval and then VT the newly created Title record to the pre-existing canonical Title record
  • use "Add a Variant Title or Pseudonymous Work to This Title", wait for the approval and then use "Add Pub" to add publication level data to the newly created Title record

Use case #4 is unwieldy and I have been thinking about converting it to a one-step process for some time. Once foreign language support is added, the proposed option "Add a Variant Title or Pseudonymous Work to This Title" will parallel the behavior "Add a Variant Title or Pseudonymous Work to This Title" except for the added language-related features and deactivating some fields. I expect that the discrepancy between what editors really want to do, i.e. add publication level data, and what they can do, i.e. add title level data, will become noticeably more irksome since foreign language translations will no longer go under canonical titles and will probably become more common. I am thinking that we may want to improve this area as part of the original roll-out of foreign language support, but, unlike the issue of pseudonymous translators, it's not critical. Ahasuerus 02:55, 22 June 2010 (UTC)

I would propose to implement first the original 2 step scenario for translations: 1) Add Translation to This Title (wait approval) and 2) Add Publication to the new created translation title record. I was also thinking about combining more than one step into a single one as I added the translation support to the "Add New ..." option (use case #1), but this really overcomplicates the translation support feature. GaborLajos 21:12, 22 June 2010 (UTC)

• In ISFDB, uneditable fields are gray rather than yellow. Yellow is reserved for container Titles. Ahasuerus 00:06, 21 June 2010 (UTC)

OK, GaborLajos 13:41, 21 June 2010 (UTC)

Code organization: maintenance and security

By my count, there are currently 197 CGI scripts and numerous modules (.py that are not renamed to .cgi) that are also copied (sometimes redundantly) into the CGI web space. That seems like far too many CGI scripts and the modules do not need to be in the CGI space at all (they are not accessed by the web server in any way) and certainly do not need to be redundantly copied around to different directories. For security reasons, I would like to see the modules moved out of the web space into a Python isfdb site package and many of the CGI scripts merged to reduce the count. Technically, the entire application could in theory be a single script and the script could just do different things based on the URL provided to it. Currently the CGI scripts seem to use some parts of the provided URL but not others. The GET query string (mostly in a raw format accessed via CGI argv but sometimes in key-value paired HTML form data accessed via CGI QUERY_STRING environment variable with the Python cgi library) and POST data (almost elusively in key-value paired HTML form data accessed via CGI stdin with the Python cgi library) are used but I have not seen any code that makes use of the extra path (CGI PATH_INFO). Methinks the functionality can be broken out into more modules (as part of a package) and the CGI scripts merged based on URL extra path or HTTP method (GET vs. POSTed form data, etc.). This would make the code more organized which is good for both maintenance and security. Such scripts could also be generalized to use a CGI shim to WSGI so that porting the application to other environments would be easier (e.g., running it as a WGSI application in a FCGI or SCGI application server with nginx web server, etc.).

I thought I would solicit feedback before submitting a feature request (although this sort of thing could perhaps be considered a bug). Uzume 16:40, 2 March 2011 (UTC)

I think you've managed to confuse enough people that you won't get any feedback as we're frightened to comment on things so technical that even someone like me with over 30 years in IT can't usefully comment on most of it. :-( If you've actually been working on this, please post some examples of what you mean. Like Bluesman, the other Bill, I can learn a lot from examples and pictures. BLongley 12:39, 22 January 2013 (UTC)

I shall have to work on that (some sort of pictures or perhaps a mock-up). There is some related discussion below #Improving the installation setup. Uzume 16:15, 3 January 2015 (UTC)

Author Directory

There is a known Bug 3128545 "Author directory doesn't handle apostrophes in author names" - which I've sort-of fixed by counting a second-letter apostrophe as a 27th possible "letter" for the grid e.g.

It also needed a bit of work to allow apostrophes in the drill-down: e.g.

However, there seems to be another existing bug - when I tested with an author Z'Zum it didn't "take" in the author directory, and it seems that an author of Zzum doesn't "take" in the current one - so I think there's something odd about the very last cell in the grid. Can anyone tell me what this problem may be? BLongley 17:40, 7 July 2011 (UTC)

Never mind, seems I must have been testing with double-capitals or something. I'll submit this and wait for complaints about the other things it doesn't fix. BLongley 22:38, 8 July 2011 (UTC)

I think we've finally fixed this. Does anybody disagree? BLongley 12:30, 22 January 2013 (UTC)

Unmerge Foreign Title(s)

I'm working on a new menu option, provisionally placed like this:
that will allow you to select one or more publications to be unmerged and placed under a variant title of the original, with a selected language - like this:

So far it's been quite easy but it might need a few more tweaks: e.g. check that the unmerged title is a different language from the original, or you're probably using the wrong unmerge. If all the unmerged titles are identical, create just one variant title with all the pubs under it. (In the example above, I suspect that two titles are better than one, is this how people usually deal with split volumes?) Make sure that this doesn't lead to chains of variants - or are such desirable? (Some might consider a translation of "Harry Potter and the Philosopher's Stone" different from a translation of "Harry Potter and the Sorcerer's Stone".) Should we try and build in some checks matching ISBN ranges with languages? Perhaps we should restrict this option to people that have actually set their language preferences? Do we need to make "My Languages" effective on displays first? Should Ahasuerus and I set up Paypal "tips jars" so that people can persuade us to prioritise this through sheer bribery? Is the lack of Klingon language support a show-stopper? How do I stop asking so many questions? Why am I up this late? Answers to any or all of the above welcome. BLongley 00:15, 17 July 2011 (UTC)

My initial reaction was that this should be combined into the existing unmerge, without the need for a second one, but I'll have to think about it. The more choices that are available, the greater the likelihood that someone will pick the wrong one.... As an aside, I like your idea about ISBN consistency checking -- perhaps we could get a warning (like the new publisher / new author warnings) on the mod screen(s)? --MartyD 10:04, 17 July 2011 (UTC)

We could overload the existing one, but as the existing one normally needs a follow-up edit (or several) and this one shouldn't, I chose to make it separate. But I see your point - this feature could be used to create same-language variants in one step, in which case we wouldn't need a warning about same language. If we want that then it's probably got the wrong name - just "Unmerge to Variant" and document the two different use-cases? BLongley 17:39, 17 July 2011 (UTC)

Since my programming skills are virtually non-existent, I can't comment from that side. From the user oriented view, the "unmerge foreign title" function looks very useful for people like Hervé (Hauck) and Christian (Stonecreek). I have held back on entering Dutch translations, so there are probably less than 100 of those in the database (all easily traceable). On the bright side, I don’t care much about the absence of Klingon. The languages I will need are all there (including Frisian and Esperanto) --Willem H. 13:29, 17 July 2011 (UTC)

Yes, I did have certain editors in mind when I coded this - feel free to spread the word to other Editors. I didn't highlight it to Hervé as he's on holiday, but if this is still outstanding when he returns I'd like his views too. BLongley 17:45, 17 July 2011 (UTC)

Back from rainy holidays ;-(. It's quite exactly what I have in mind. I just wish to add that I'm not overly fond of the idea of "language preferences" (my library is quite splitted in half between french and english titles). Two potential problems : 1) the cluttering of the "main" page of authors with vts. Just think about the sheer number of those, even in the same language(this should perhaps be solved by setting a display option); 2) the books with exactly the same title in different languages (Let's say _Dune_) : is _Dune_ (en français dans le texte) a vt of _Dune_ (in english) or barely just another instance. Hauck 16:05, 25 July 2011 (UTC)

I believe "My Languages", when activated, will happily allow you to indicate that you're interested in both French and English titles. Setting such should make the summary bibliographies manageable, unless you're masochistic enough to choose "Show translations in all languages". Ahasuerus is holding back changes like this until we deal with the display issues - I don't know how much needs to be done before he'll allow this in. But setting a "Working Language" for our more prolific authors does seem to be another prerequisite, so we can default the summaries to a manageable level even if you haven't set "My Languages" - I've been working on that too: see User_talk:Ahasuerus#Working_Languages. At present, I believe that the final goal is that even identical titles in different languages will be variants - although we might need "Translator" support before some editors will start the rework. (I've started on that too, but put it on hold till we fix some of the prerequisites.) At present, I have the time but not the talent: Ahasuerus has the talent but not the time. But we're beginning to get there, and the more comments and discussions we get started now the clearer the path becomes. BLongley 17:32, 25 July 2011 (UTC)

After a year or so, I think I'm ready to continue work on this. Does anyone have further comments to add before I (re)start? BLongley 12:49, 22 January 2013 (UTC)

ARCs

The various ARCs-in-or-out discussions spurred this thought: How about a new publication type, ARC, akin to a CHAPTERBOOK? It could have all of the information a regular publication would have: ISBN, date, binding, etc., and we would make it like a one-work collection (so it would contain the actual NOVEL title, along with its ARC title record). On the title summary page, we could relegate ARC appearances to their own section. The non-"All" form of titles searching could avoid the ARC title records. I'm sure there's more to it than that, but hopefully you get the idea. --MartyD 11:44, 24 October 2012 (UTC)

I'm not sure that would work even if the policy were to be changed. If we add this type without changing the policy, it would only encourage editors to enter them. Currently the policy stands and ARCs are out, so my objections are moot.

But here goes: Having a different type would require a separate section for ARCs on an author's summary page. Displaying them separately from the published works of the same title, I feel, will lead to user confusion. Title page display of both types seems to be OK, but I'm not sure how you would be able to do this without a way of "binding" together a title record of one type with a title record of another type without using the tired and overused variant function. But as I said above, none of this matters until the policy is changed. And I, personally, would have a strong objection to entering nonexceptional ARCs into the database. I think the de facto standard of entering exceptional ones on a case-by-case basis is the best approach to this. Mhhutchins 16:00, 24 October 2012 (UTC)

Yeah, what we'd really(?) want is allowing an ARC publication directly associated with the appropriate title so there'd be no ARCs section in the author bibliography. Or maybe have it be CHAPTERBOOOK-like but not display ARC container titles in author bibliographies. A benefit of having a specific ARC type, even with current practice of exceptional-ARCs-only, would be it could be labeled very explicitly (e.g., "Special ARC" or some such), and there could be very and detailed help for it with regard to what's in and what's out. Plus, for those cases where the work was truly published but one or more ARCs are "in" due to unique ISBNs, it would separate ARC appearances from what we consider to be the actual publishing history. --MartyD 13:01, 25 October 2012 (UTC)

More overload wanted?

I'm finding it quite difficult to maintain conversations now - two email accounts, two LiveJournal accounts, using the Wiki here, etc. But while last updating LJ I noticed they offer contacts by (deep breath): Facebook, Twitter, AOL IM, ICQ UIN, Yahoo! ID, Windows LiveID, Jabber, Google Talk, Skype and Gizmo Project. Does anybody fancy having any of these options (or othes I've haven't noticed) available as part of their user profile? BLongley 13:19, 22 January 2013 (UTC)

P.S. I won't be taking up any of them myself, but thought I'd offer the code change. Be warned, you may end up as official 'Speaker-to-Facebook" or suchlike, like I became 'Speaker-to-LJ!). BLongley 13:19, 22 January 2013 (UTC)

How about a "Speaker-to-ISFDB-Users"? Wouldn't it better to have a way for db users (not editors) to communicate with us without having to create an account? I can only imagine how frustrating it would be for a user of the ISFDB to have no other way of contacting the maintainers of the website. Mhhutchins 15:47, 22 January 2013 (UTC)

I thought we already had a generic email address that would get to all mods that agreed to participate? Although I must admit I can't recall what it is, and have rarely (OK, never) had my email flooded with inquiries. If we do, then obviously we need to publicise it better. And if not, probably create it, and maybe a few others so that users who don't want to use the wiki can contact developers directly, for instance, rather than mods who aren't developers. Or is email too old-fashioned now? I'd code for Twitter and Facebook if pushed, but have avoided ever using them! BLongley 20:57, 22 January 2013 (UTC)

Well, if there is such an email address, I've never heard of it. If it exists, someone must have access to the inbox. But they wouldn't have any worries about answering any inquiries, because the address isn't posted anywhere on the website! If email is old-fashioned then there must be another form of communication that db users should have to contact us. I think we sometimes forget that we don't work in a vacuum here. 22:19, 22 January 2013 (UTC)

I agree that it would be very beneficial if occasional users could leave comments without having to register. Instead of email, I'd favor a solution like on the Metal Archives: On each page near the top right there is a button labeled "report an error". Clicking it opens a web form where the user can enter corrections, additional information etc. He is also asked to provide a source. I suppose that these reports are then checked by an editor. Darkday 00:06, 23 January 2013 (UTC)

Looks pretty good to me. I'd even volunteer to be handle the submissions. Mhhutchins 00:36, 23 January 2013 (UTC)

Ooh, a volunteer, and an almost complete FR or two! Ironically, NOW would be a really good time to use one of the more instant types of communication to nail down what we exactly need. (I'm up late, and must be matching effective time-zones with some of our more Western contributors.) BLongley 01:24, 23 January 2013 (UTC)

A few weeks ago an anonymous user created an unrelated FR and mentioned that we should probably allow people to contact moderators without having to create a Wiki/ISFDB account first. There is a very easy way to make this happen -- simply allow anonymous contributions to the Wiki, which can be done with a single configuration change -- but then our mostly dormant spambot problem may come back. Ahasuerus 04:09, 23 January 2013 (UTC)

Well, if it's that simple a change it presumably can be just as easily be reversed if it causes more problems than it solves? BLongley 08:18, 23 January 2013 (UTC)

As far as the proposed "report an error" (or perhaps "leave feedback") link goes, it may not be hard to implement, but we will have no way of contacting the submitter to ask follow-up questions, which, as we all know, are frequently required. Ahasuerus 04:09, 23 January 2013 (UTC)

That's dead easy to implement - a "mailto" link with Michael's email address on would suffice! :-) Presumably he'd then be able to respond to them directly on whatever email address THEY used. BLongley 08:22, 23 January 2013 (UTC)

Improving the installation setup

The current installation process for ISFDB consists of a number of makefiles, python scriplets and makefile fragments. Overall it is not easy to get one's head around, as I have discovered over the last few weeks. :-). It is not robust -- e.g. if a script is removed from one of subdirectories, but is still listed in the corresponding TARGETS file, installation fails to complete even if make is instructed to ignore errors. And it does things in a rather complicated manner -- e.g. in creating intermediate local copies of scripts before copying them into final places, or in repeatedly processing the same files.

Suggestion:

1. Replace the whole setup with a single script (python, perl, php -- as long as it starts with a 'p' :-)). I guess python would be preferred in this context to keep down the number of languages involved.

2. Eliminate the need for separate TARGETS lists by holding scripts which wind up with the .cgi suffix as having .cgi suffix in the first place. The list of so suffixed files in a given subdirectory then replaces TARGETS.

I would also suggest two more points that may be worth thinking about:

3. Consider ways of avoiding creating duplicate copies of the same scripts in the final installation. This can be achieved either by using links (symbolic links at least do work under Cygwin -- not sure about hard links), or explicitly by tweaking script contents in the installation process.

4. I am uneasy about the localdefs.py being placed in cgi-bin, since it contains all the information for accessing the database. As a general principle, it is better to keep such information outside the document root tree.

MLA (Mike Arnautov) (This comment added on 2014-11-14, 11:29-11:31.)

Some further thoughts and a specific proposal...

The separate existence of the common/localdefs.py and INSTALLDIRS was initially puzzling. The difference between them is, of course, that symbols defined in INSTALLDIRS are replaced by their values as a part of the installation process, whereas symbols defined in localdefs.py are untouched by that process. This perhaps made sense originally -- allowing e.g. server addresses to be modified without re-installation by modifying the relevant values in a single script. However, presumably for reasons of efficiency, localdefs.py is replicated several times by the installation process and hence there is no longer a single place defining these values.

Generally, such replication of files makes maintainability more difficult. The main difficulty I found in coming to grips with the current installation process was not so much in understanding how it works as in understanding its effects. E.g. having discovered that some files were being replicated in several places, it was entirely unclear (a) what file were being so duplicated and (b) whether any files bearing the same names were in fact duplicates.

How about a modified schema, which avoids all these issues?

1. Replace all symbols other than USERNAME and PASSWORD defined in INSTALLDIRS and localdefs.py at installations time. Effectively this means moving everything but USERNAME and PASSWORD out of localdefs.py into INSTALLDIRS (or its equivalent).

2. The installation process reads this INSTALLDIRS replacement and constructs an associative array (Python dictionary, Perl hash. PHP array) of symbols defined therein and replaces all of those symbols in all scripts it is copying.

3. Keep just one copy of the localdefs.py equivalent -- scripts which need its values access it via relative pathnames. While this may (or may not!) be slightly less efficient than having a copy of that file in the same directory as the calling script, this is counter-balanced by the savings on no longer having to use localdefs.py to retrieve values of other symbols.

4. Hold the original (non-installed) version of isfdb software in exactly the same structure as the installed version. The only difference between the two being that the non-installed version has symbolic embedded in its scripts, whereas the installed version replaces these symbolic names with specific values as defined in the INSTALLDIRS equivalent. This permits the installation script to be independent of the actual structure of the site. It just needs to traverse the HTML and the CGI trees and replicate (with substitutions) their content.

5. The installation script could (but does not need to) check date/time stamps on non-installed and installed versions of files and only process those which don't exist or are out of date.

For maintainability, I would suggest that in implementing the above, any other scripts which currently get replicated across the installed tree, should also be held in a single place and accessed by relative pathnames. The efficiency impact would be minimal (if any) and in any case counter-balanced by any efficiency savings as per (3) above.

NB: Some scripts not only get variable values from localdefs.py, but then use the %s notation to insert these values into textual strings. These substitutions could/should be also eliminated.

MLA 14:02, 17 November 2014 (UTC)

I like these ideas. I agree with you on all of your major points. :) One further thought I have is that anything replicated strongly points to the need for another central/shared location. It would be better to avoid directory links so that we're not tied to O/S capabilities, but I think a little restructuring to move these into a (possibly new) location of their own and have everything refer to them there would work out well. --MartyD 13:06, 18 November 2014 (UTC)

I agree with some of your points and disagree with others—that said I at least agree there is a need on all the points you made (the proposed solutions I do not always agree on). Your multiple lists make it hard to refer to an address each in turn but let me try. First your initial list of suggestions:

1. I have wanted to move away from make/Makefiles to a more modern dependance build system. There are a few options but waf comes to mind. I would like to leverage Python's setup/distutils (perhaps even using setuptools).
2. I would have to say this is a bad idea. Frankly, I would rather move towards a solution that does not require but still allows CGI, e.g., one where we install Python packages into library path (perhaps using virtualenv/PEP 405 or something similar to allow multiple installs on the same machine at different paths) and the Python parses the URL sub-paths and routes to the correct module(s). This would likely necessitate changing our URL scheme and there would be need to be a period of backwards compatibility with several legacy CGI stubs for redirection.
3. The server runs on Linux not Windows to there is no Cygwin. I am also against this and would rather remove the redundancies moving to Python packages within the library path as mentioned above.
4. I do not like localdefs.py in the cgi-bin but then again I am against the large number of CGI entry points we have to begin with—that is too large of an attack surface and it is too easy to miss/mess-up permissions needed to protect you.

To address your final points, I see no efficiency in localdefs.py replication. I believe that was just what worked and was used. I would prefer to substitute the least amount possible as it is a maintenance issue. I find it far better to just have all the parameters defined in a single place and have a script generate a Python module for values used at run-time in the code as necessary (some values are only needed at build time such as filesystem folder structure). I do not think the installed structure needs to match the code/repository structure. Likewise I do not think URLs paths need adhere to either installed structure or the code/repository structure. On the other hand it should be organized and logical. I hope that addresses most of your points and perhaps given you something to think about. Thank you for your interest/feedback. Uzume 01:50, 19 November 2014 (UTC)

MartyD: Thanks! :-) I only mentioned links in case it was felt necessary to preserve the file structure of the installed directories exactly as it is at present.

Uzume: I understood that the system is supposed to work on Windows/Cygwin as well as on Linux. Did I get it wrong?

I fully share your dislike of the reliance on CGI, for just the same reasons. However, thigs are as they are. Getting 'there' from 'here' would require some considerable effort. My aim is a far more modest one: eliminate maintainability issues in the current arrangement with no changes to functionality of the existing isfdb software and without affecting any longer-term considerations such as you have in mind. Since this can be done simply and easily it is, to my mind, worth doing.

The current installation process creates a directory structure and copies files into that directory, with some straightforward token substitution substitution to file contents. This is too simple to require configuration maintenance tools. The simplest way of defining the target directory structure is by starting with that same directory structure -- the payout is that the installation process becomes completely insensitive to the details of that structure as well as trivially simple (25 lines of Python).

So the only work actually required does not amount to very much:

• Construct the uninstalled tree. This is easily done from the installed tree via reverse substitutions.
• (Optional) In individual scripts, eliminate the no longer required dynamic substitution of symbols formerly kept in localdefs.py -- not difficult.
• Eliminating script replications. This is the only bit requiring some though and effort, but it a with some ad hoc scripting that too should not be *too* hard.

MLA 17:31, 19 November 2014 (UTC)

Yes, you did get it wrong—at least partially. We do support the software running on Windows but I do not believe Cygwin is anywhere a part of that. None of Python, MySQL, or Apache use Cygwin.

I agree such changes require some work and time to get to the end goal but I would rather not do a bunch of work to rearrange things that will likely just be useless moving forward anyway. I prefer to do the least amount of intermediate work possible. With that in mind, I propose to try to make the bigger jumps by identifying which do not depend on others and can be done separately. The way I see it, there are a few different items:

1. Move to WSGI (perhaps using a framework to help us with such) and away from dependence on CGI (WSGI can be made to work through CGI but let's have a single CGI entry point for that option). This allows us go restructure the code (so we can use Python's package and import system instead of relying on folders in CGI) and remove redundancies like copied localdefs.py and unneeded copying during install but causes a change in our URL structure (because the Python would parse the sub-URL paths instead of the web server/CGI). I already did some previous work in identifying our public interface and what would need to be supported (via HTTP redirects) during an interim period: Talk:Development#"Public" URLs.
2. Move to a newer/better build system (i.e., away from make/Makefiles)—preferably towards something Python based (we already have this dependency; getting rid of one older one like make is good).
3. Move to a newer/better revision control system. I would prefer a distributed one. My personal preference is to move to Mercurial like Python development does but there are other good choices too (git, Bazaar, Fossil, etc.).

I have many other ideas too (like a revamped web API and submission/editing system that uses the same API) but they do seem to fit into this discussion. For my related discussion see #Code organization: maintenance and security Uzume 15:54, 3 January 2015 (UTC)