In the previous post, I described two challenges using CaFE from the Juror's perspective. There were more challenges for the jurors.
From my experience as a juror, CaFE offers a sub quality jury review. Jurors and artists need to know. CaFE needs to change!
Reading this post, voicing your opinion, and the power of the marketplace can be a voice of change.
Juror Challenge #3.
Image review is a nightmare.
Image review was possible in three different methods - all bad! When reviewing images in the "Preview Slideshow" mode, "Thumbnail Scorecard" mode, or the "Slideshow Scorecard" the juror could only go forward. The juror could only advance the images on the monitor to the next images.
The impact was that you looked at the images submitted early (the ones at the beginning of the review) over and over, and over. The images at the end of the slide review were seen fewer times.
Eventually, almost by accident, I finally realized after the jurying was over that the left and right arrows on my keyboard could be used to go back and forth while in the "Slideshow" mode. But in the "Thumbnail" mode, you have to use the BACK button. Not even the same button. There were no explanations or indications for these options on the screen.
The other juror had the same problem. Neither of us figured out the keyboard options for reviewing the previous images. There are no clues or cues on the screen for how to look at a prior image. For comparison, Flickr and Facebook albums have arrows, next or back, buttons on screen near the images.
Juror Challenge #4.
Scorecard was confusing.
The review for the scorecard was effectively encrypted, i.e. hard to decipher and confusing. Images were described by a 6 digit number. There were no options for sorting the jury ranking of the submitted images. Also, the scorecard did not include a thumbnail of each image to remind the juror which six digit number was which image.
The juror is presented with a scorecard with columns of 6 digit numbers (above right image). How would you feel as the juror? How can you possibly remember an image by a six digit number? Talk about burning the brains of visually oriented people.
I went to the trouble of transferring the scorecard data to an Excel spreadsheet, re-ranked the images by the score column and reviewed my results. It probably took me longer to review my preliminary selection than it took for initially ranking the work. Bad, bad, bad news for jurors and artists.
Juror Challenge #5.
The artist's statement on the application form is for one text statement only, and it is not associated or tied to the images. Thus a statement for each image was not available.
The impact was that the artists' statements seemed generic and not specific to the images. The artist statements were not informative. I will tell you my recommendations for getting around this in future posts.
The juror review took hours and hours; much longer than necessary due to ineffective and inefficient handling of the content. I reviewed work on successive nights from beginning to end, looking at every image at least once each night so that they would be fresh in my mind.
In the big picture, it is the greatest of honors to be asked to make juror decisions. I respect the process and take the responsibility very seriously. In this event, I tried very hard to provide my best decisions, but I also came to realize that this CaFE system has room for improvement, to put it mildly. The lack of intuitive interaction caused uneven review which offends my sense of justice. I am enthusiastic about using the Internet, but the current structure and interface of this online service is a hindrance to the arts and crafts community.
If we don't talk about this out loud, it will never be fixed. I hope I am not black listed forever for revealing this disturbing little secret.
CaFE needs to improve their site, or I recommend finding another method for reviewing images.