Err no, it's still grossly unfair. I cannot imagine how you're not getting why judging doesn't work that way. Especially stories, we're not grading math tests where there are right and wrong answers.

And wut? I've read and I know a lot of others read almost all the entries in every month's competition. I don't see why the judges cannot do the same.

 

I think that's partly because judging and reading something are two different, though related, activities. While I've not really participated much in DLP contests, but having done so otherwise, I can say that judging requires at least two read-throughs (as long as the story is complex enough) - the first to parse, and the second to affix the structure, theme and other metrics, and assign them scores.

 

A set of judges, semi-permanent (as in, substituted if they submit something), who judge on separate criteria. Utilise the differing personalities and tastes that comprise the community. For example, certain users would place higher credence on world-building, atmosphere, and continuity therein. Others would place higher credence on grammar and technical writing ability. Others still would value plot and pace. Others would value characterisation, dialogue, and authenticity, more perhaps than others might.

No judging can be truly objective. I'll value things in stories that others won't, and be indifferent to other readers' dealbreakers. One set of monthly judges is going to react completely differently to another set of judges for the same submissions. By creating a panel of Official Judges and their substitutes, scoring stories via an amalgamation of their individual ratings, we could potentially capture the essence of DLP's tastes and avoid the rigidity of individual judges scoring out of ten for categories they value differently to each other.

I'm sure you can think of certain members in these roles - the duty of specifically considering individual aspects of each work, the aspects they value most highly. Warlocke for grammar, for an obvious example.

There's such an eclectic soup of tastes and preferences on the boards. Though historically they've best served as entertaining arguments in review and common room threads, it's pretty clear that certain members value certain components of individual stories more or less than others. Whether people like or dislike a story is essentially irrelevant - no matter how strongly the majority or minority feel about a story, others will feel differently. Why not accept that and promote the works that have had their individual components positively evaluated by the most critical proponents of each category that DLP has? Why not, instead of promoting a piece as that particular group of judges' favourite, go about presenting a winning piece that, like or dislike, is the closest to technically and aesthetically perfect that DLP can create?

TL;DR: If you're going to quantify this shit, do it properly. Otherwise, just have the entire community be able to rate submissions and thus provide the winner. The results would probably be remarkably similar.

 

Perhaps consider having a similar rating system as a site like Rotton Tomatoes. Have a panel of judges give an overall rating. On the other hand, let the community rate the stories. So there would be two categories: the most popular story among the DLP community and the story that the judges deem to be the "best."

 

Aka exactly what we have/had.

 

A possible solution to the judging problems:

For example, say thirteen people sign up to be judges, and there are six stories to be judged.

Over a month's time (30 days), you set a deadline for each story to be judged on a 100 point scale.

That's 5 days for each story. A judge would be expected to read that story for those five days. Now, the rest of the stories have been posted for everyone to read, BUT THE THREADS ARE LOCKED so no one can post on that story until its that stories' turn to be graded. There should be no reason that you can't easily read a 30,000 word story in 5 days' time.

Then, say 10 of the 13 judges had time to read my story. Out of the 10 people, you take each score out of 100, and average it out. I'd assume this is how we did it last time, but I can't remember correctly, as I wasn't a part of either the writing or the reviewing.

That way, everyone gets a fair result, even if 8 people judged someone elses' story during that time frame. Reviewing the story after the 5 day period is not allowed, for continuity purposes. That means every story would most likely be read once by each judge, and we'd all have an equal (as equal as it gets, that is) chance to be judged properly.

Anyone agree?

 

There is no point whatsoever in a percentage system, or whatever it is. You might as well rate out of five, to two decimal places. Just like the Library ratings average out.

The point of my TL;DR was to try and generate ideas that veer away from quantifying what can't actually be quantified. 100 is 'I love this story more than any other' and 0 is 'this is the worst story in the fandom imo' - either way you're completely subjective.

If we're going to take a proper peer-review approach to the judging, make the criteria what's graded, independently, not our vague individual feelings towards the whole work.

We already have a system that does that. It's called the fucking Library.

 

What about having the community of DLP be the judges?

As in, have someone post all of the stories in a board at the end of the contest, and open it up to the members of DLP. Open up a poll for a week and have the members select the one they liked the best.

We have a pretty educated populous and all of us have different likes and dislikes, so it is quite fair.

Plus, we would never have to worry about the judges not responding to the submissions.

 

The problem with having the whole board do it, is, well, honestly, some of us suck.

What's wrong with a point system based on criteria, G.L? Say 10 points for originality, 10 points for obeying the prompt, 10 points for writing ability, etc. Somebody might get 10/10 for writing a completely original story, but maybe 5/10 for obeying the prompt. By the end you have a concrete value and you won't have to worry about 4/5's from everybody for the same story.

 

I still go with a set panel of judges reading each story and judging. Unless of course the judging is opened to the whole DLP community, with the condition that after reading, it is mandatory to provide a short review and a rating in a poll. That way there won't be 1000+ people reading an entry and only 50 rating it.

 

The fact that each of the criteria don't contribute ~10% of the value to the story.

 

We can't have a month for judging for reasons already specified. We have every judge do every story, and whoever doesn't get their scoring in on time gets dropped. That's it. By far the easiest way to do it.

 

I really like this. Especially if you have different categories, where the winner of the contest is the one whose story gets the best average score of aggregate of the categories.

It'd be nice to have categories like

Best Technical Writing--things like really artful diction (choice of words), great syntax (phrasing), timing, flow, etc . . . The plot might be shit, the characterization might make you wanna puke, but holy shit you just can't stop reading because it's just written so damn well! We have quite a number of stories in Almost Recommended that are explicitly there for this very reason: story itself sucked but goddamn was it well written crap!

Best Charcterization--This one's a little hard. Some people's panties get in a twist because "that's not X, that's an OC with X's name!" whilst others wouldn't care, so long as the character's actions and speech make sense in context, and other's may just be looking for a well written character--faithfulness to canon be damned.

Best Plot--pretty self explanatory.

and so on . . .

I'm in full concurrence.

 

I agree with Lindsey

Open the judging to community. Every reader can rank the stories in an order by giving them ranks 1,2,3 etc...

Why are we trying to stick to the 100 point scale?

 

Because it's the easiest and has no real drawbacks?

 

Have the judges place their score out of a hundred based on some relatively set criteria(one of which ought to be personal issues and preferences), but allowing the judges leeway on how much weight to pu on each criteria.
I‘m not fond of the idea of opening the judging to the whole community. That‘s just asking for problems and/or delays.
Aditionally, if the given timeframe is extended, thusly allowing for bigger stories, the given time for judging should be extended, also.

 

Okay, can anyone explain to me why exactly we need a new system for the judging? As far as I can tell the old system worked perfectly fine for a long time. What was wrong with it?

 

What was wrong with it is that some of the judges took a fucking long time to get their results in. Which is why redundancy is a good thing. Otherwise the old system was perfect.

 

I'm a n00b and a lurker and all, but after finishing my first novel-length fic a couple weeks ago, my muse is having trouble staying focused on one story. So I was cruising DLP for a challenge I could use as a distraction and found this thread. Since I wasn't around (at least, not writing) the last time the official challenges took place, I want this to happen sooner rather than later. So, for what it's worth (very little, I imagine, as this is my first post):

1) How about a trial run to gauge interest from writers and potential judges and try out some of these new ideas? Put up a notice that a new contest is going to take place in a week or two, then somebody (I nominate the OP) just go ahead and dictate the terms.

2) Community judging is fine -- even useful -- as a separate effort, but it's no substitute for a group of trusted, knowledgeable members. With the former, there's almost no chance that everybody who rates the entries has actually read them all. That's been my experience with other creative communities, at least. There's the question of who decides what trusted and knowledgeable actually means, but once again I think the organizer just needs to make the call.

3) Require the organizer to write a postmortem: what went right, what went wrong, other observations and suggestions for improvement. Sort of like the original post is doing, but perhaps something more formal to ensure the subsequent contests can build upon lessons learned from this one.

Well, that's my pitch to get things moving along. Re-reading the post it seems I'm starting to let my day job seep in...I already narrowly avoided using the word "sustainability" but I still ended up using "lessons learned" anyway. So I feel like I should work in the phrases "leveraging operational synergy" and "aligning core competencies" and stuff. Either way I'm looking forward to what comes out of this.

 

Lots of arguments on the score front, but none that would convince me to move away from scores out of 100.

The only change I see so far on the judges front is that we would chose more judges and set a hard deadline. If you don't make the deadline, you'll be ineligible for judging in the following contests.

A star/mark/banner/title or whatever will be awarded to the winner. Second place is first loser, but maybe there'll be a gold/silver/bronze dealy. Might depend on the number of entries, or the fact that DLP doesn't accept anything less than true excellence.

Something more interesting for debate: What about topics/themes? While plot bunnies seem unsuitable for a longer contest, General or Dark!Harry may be a bit too general, while Humour or Romance might keep many from competing.
So, thoughts and suggestions on that front please.