DOING THE MATH OF JURY RANKING SYSTEMS? - the numbers never lie.
June 30, 2010
There are many different ranking systems that jurors can use to score work for selection during a jury system.
A ranking system for a single juror can be as simple as
YES, NO, or MAYBE.
When more than one juror is involved or there's a large number of entries, a numerical ranking system is the most efficient way to select the best works.
The Professional Guidelines recommend a ranking system of 1 – 7 (i.e. 1, 2, 3, 4, 5, 6, 7), as the most practical and useful ranking system for the judging process and effectively differentiates results.
Jurors should consider “1” as the lowest score and “7” as the highest score in the ranking system. Jurors should be encouraged to use the full range of the 1-7 ranking system when evaluating the images (or work) submitted for the jurying.
Unfortunately, some people may suggest a scoring system that removes the middle number from a 1 – 5 system (e.g. eliminating the “3”) and uses only 1, 2, 4, 5 to supposedly “force” a selection outside of average. I took note in the previous post that the jurors in that experience were advised to use numbers 1-5 to score work and to avoid, if possible, the number 3. Jurors tend to use the middle range of numbers when scoring work because each piece viewed is rarely the best or worst that the juror has ever seen, i.e.close to "average". )
However, a scoring system that removes the middle number (e.g. eliminating the “3” from a 1-5 system and using only 1, 2, 4, 5) actually increases the likelihood of ties (regardless of the number of jurors). Mathematically, there is no difference between 1, 2, 3, 4 and 1, 2, 4, 5 since the number of sums (outcomes) is identical. Using fewer score choices simply increases the possibility of ties.
In other words, by eliminating a scoring choice, there are fewer possible total scores.
Honestly, I am not really a math person, but I live with two super math people. My husband and my son! They have exposed me to more math than I really care to know and helped prepare the Comparison of Jury Ranking Systems in the Professional Guidelines.
This document is only on my website! If you want to read the explanation of the math behind jury ranking systems, please read the whole document. If you need help understanding the document ask a math person....but start using 1-7 to review all jury situations. Numbers don't lie.
No matter what ranking system is used, the jurors should be encouraged to use the full range of numbers to rank the images.
Tell everyone you know!!!!!
Every artist can be an instrument of change. Make sure that every organization that you know adopts 1-7 as a jury ranking system.
Please consider sharing your experiences and comments about jury ranking systems.
P.S. Tomorrow's post from Guest Author Alison Antelman will offer advice about how to improve your chances of being accepted in a jury situation.
This post was updated on January 19, 2022.