• Setting your human judging quota with AI tasks

  Make sure all your scans for your candidates are uploaded. Go to the **Run judging session **page and make sure the expected candidates in the judges panel is as you expect (see image below). Use the slider to set the ratio of AI judgements that you want on the task. The default will be 90% AI judging. Now set the expected human judges but set tFew readers

• Setting up judges

  Choosing the number of judges for the task The more judgements that are made by judges, the more reliable (precise) your results will be. As a general rule, we would suggest carrying out 10 judgements per candidate, spread across all of your judges. ||| For national tasks (as opposed to in-house tasks), you cannot change the Candidate Judgements value from 10, and you cannot change the Expected Candidates value. You can only change the Expected Judges value. For example, iPopular

• The judging interface

  When you are given a judging link, you will be directed to this page. If audio feedback is being collected for your task, you can click the "Start Recording" button to test the microphone of your device. Fill out the Email Address field then click "Submit" to start judging. Here is a good look at the page once signed in successfully as a judge. ![](https://storage.crisp.chat/useFew readers

• How to update judge quotas

  || During the judging process, there may be situations when you would need to adjust judging quotas or re-assign judgments from one judge to another. Here's how to do it: Log in to your NMM account. Open the relevant task. Click Run judging session to access the Judges page as shown on the screenshot below: Tick the tiny box before a judge's email address to acFew readers

• Reporting a safeguarding issue

  To report an issue with a script you encounter while judging, please click the tab between the decisions button, specify if it's the script on the left or right side, and type a comment- Note: you can type freely for yourFew readers

• Feedback on judges

  Judging Feedback After you have set your judges off judging, you will want to look at how they getting along in their judgements. This article explains how to review their progress. At any point during the judging process, for a given task, select the Run judging session option, and then, just to make sure the judging statistics are up to date, press the 'Refresh Scores' button: ![](https://storage.crisp.chat/users/helpdesk/website/2d5d0b5095261a00/screenshot-2024-09-02-at-74546_vougjFew readers

• Local judgements and moderation judgements

  When you take part in tasks that are moderated across other tasks and schools, a judge will be making moderation judgements as well as judgements on their particular task as part of their allocation. For example, in our national assessments such as Assessing Primary Writing (APW) or Improving Secondary Writing (ISW), a judge will judge scripts from other schools as well as from their own school. In APW, every 5th judgement is a moderation judgement of scripts from other schools; for ISW it is eFew readers

• Judge Infit

  | If a task has been judged mainly by AI then the infit statistic is not reliable. A judge needs to complete at least 20 decisions before the infit becomes a reliable guide to consistency. If you are using AI judges then we recommend you consult the AI agreement percentages instead. Judge infit The infit for a judge is a measure of how far a judge agrees with the other judges in how they make their dSome readers

• Exploring the decisions made by each judge

  Exploring the judgements made by a given judge Here is how to explore more deeply the judgements made by a particular judge To explore the judgements that were made by a particular judge more deeply, we can look at the decisions made by that judge, whose scripts they were comparing, and the probability that the decision was 'correct'. After the judging is completed, go to the Run judging session section and then highlight the judge you want to look at by clicking on the box next toFew readers