Exploring the decisions made on each candidate
Exploring the decisions made on each candidate
Here is how to explore more deeply the judgements made on a candidates script
It is common for judges and coordinators to want to find out why a particular candidate achieved a particular score - there may perhaps be concerns that the score was too low or too high. To explore this more deeply, we can look at the decisions made for that candidate, who they were compared against, and the probability that the decision was 'correct'.
After the judging is completed, go to the Check results section and then highlight the candidate you want to look at by clicking on the box next to them.
Then click on the Decisions for Candidate button:
You will then see this page:
Here are the important aspects of this Candidate Decisions page:
Name of the candidate being looked at: at the top, you will see the candidate's name and you can click on the name to view their script;
Scaled Score: also at the top, the resulting Scaled Score for this candidate;
Judge: which judge made the given judgement;
Opponent: who the candidate was being compared against for this judgement. You can view the opponent's script by clicking on their name;
Opponent Score: the resulting Scaled Score for the opponent. The colour of this indicates whether, in this judgement, the candidate concerned won against or lost against this opponent;
Time Taken: the time taken for this one judgement;
Probability: based on the resulting candidate's Scaled Score and the opponent's Scaled Score, we can calculate the probability that this judgement was 'correct' based on other judges' decisions on the task. Or rather, the higher this probability, the more likely the decision was in agreement with the other judges.
We can order the judgements by clicking any of the headings of the Decisions table.
In the above example, we can see that Samuel Strawberry lost against Gilbert Grape and Fiona Fig, as we might have expected based on their Scaled Scores. Samuel also won one decision against Belinda Berry which again we might have expected (the probability being above 50% in these three cases). However, Samuel lost a judgement against Belinda Berry which was unexpected given their Scaled Scores (and the probability of this decision being correct was below 50% at 39.1%). Samuel also unexpectedly won a judgement against Neil Nectarine (the probability of this being correct was only 37.7%).
By looking at who the candidate won or lost against, we can get a feeling of why they achieved their given Scaled Score. We can also highlight decisions that we are concerned about, and the judges who made these decisions. We can use this information to get a better understanding of the reasons behind the scores for the given task.
Here is how to explore more deeply the judgements made on a candidates script
It is common for judges and coordinators to want to find out why a particular candidate achieved a particular score - there may perhaps be concerns that the score was too low or too high. To explore this more deeply, we can look at the decisions made for that candidate, who they were compared against, and the probability that the decision was 'correct'.
After the judging is completed, go to the Check results section and then highlight the candidate you want to look at by clicking on the box next to them.
Then click on the Decisions for Candidate button:
You will then see this page:
Here are the important aspects of this Candidate Decisions page:
Name of the candidate being looked at: at the top, you will see the candidate's name and you can click on the name to view their script;
Scaled Score: also at the top, the resulting Scaled Score for this candidate;
Judge: which judge made the given judgement;
Opponent: who the candidate was being compared against for this judgement. You can view the opponent's script by clicking on their name;
Opponent Score: the resulting Scaled Score for the opponent. The colour of this indicates whether, in this judgement, the candidate concerned won against or lost against this opponent;
Time Taken: the time taken for this one judgement;
Probability: based on the resulting candidate's Scaled Score and the opponent's Scaled Score, we can calculate the probability that this judgement was 'correct' based on other judges' decisions on the task. Or rather, the higher this probability, the more likely the decision was in agreement with the other judges.
We can order the judgements by clicking any of the headings of the Decisions table.
In the above example, we can see that Samuel Strawberry lost against Gilbert Grape and Fiona Fig, as we might have expected based on their Scaled Scores. Samuel also won one decision against Belinda Berry which again we might have expected (the probability being above 50% in these three cases). However, Samuel lost a judgement against Belinda Berry which was unexpected given their Scaled Scores (and the probability of this decision being correct was below 50% at 39.1%). Samuel also unexpectedly won a judgement against Neil Nectarine (the probability of this being correct was only 37.7%).
By looking at who the candidate won or lost against, we can get a feeling of why they achieved their given Scaled Score. We can also highlight decisions that we are concerned about, and the judges who made these decisions. We can use this information to get a better understanding of the reasons behind the scores for the given task.
Updated on: 04/09/2024
Thank you!