EU AI Act Compliance & Governance Guide: AI-Judgement Scores

Overview & Regulatory Scope

Under the EU AI Act and No More Marking's (NMM) Information Security Policy, AI systems deployed in educational settings to assess student work or evaluate learning outcomes require rigorous risk governance, transparency, and human oversight. No More Marking operates a comprehensive AI Governance Framework designed to ensure full compliance with the EU AI Act and global data protection standards.

Key Pillars of EU AI Act Compliance

1. Mandatory Human-in-the-Loop Oversight

• Advisory Role & No Fully Automated Decision-Making: NMM does not perform automated profiling or unmonitored automated decision-making. AI-generated judgement scores serve as advisory assessment inputs.
• Guaranteed Double Human Review: NMM implements a recommended workflow of 90% AI judgement and 10% human judgement. This structure ensures that every single piece of student writing is evaluated twice by a human teacher—providing greater human oversight than traditional single-marker assessment.
• School Override & Fallback Mechanism: Educators review all scores and feedback before final publication. If a school has any concern regarding an assessment outcome, it can remove AI judgements entirely and re-judge the task using 100% human-only Comparative Judgement.

2. Accuracy, Reliability & Continuous Monitoring

• Psychometric Calibration: Comparative Judgement is grounded in psychometrically validated pairwise comparisons.
• Monitored Agreement Rates: NMM continuously benchmarks AI performance against human markers. Research demonstrates an 80–85% agreement rate between AI and human judges—identical to human-to-human agreement baselines.
• Discrepancy Control: Major disagreements occur in less than 0.3% of judgements (often 0.2–0.3%) and are primarily driven by minor human input errors (e.g., misclicks), which are rapidly identified and corrected through comparison with human judgements.

3. AI Risk Management & Governance Framework

• Formal Risk Modeling: In accordance with NMM's Information Security Policy, AI processing is subject to structured risk assessments evaluating technical vulnerabilities, ethical implications (fairness and bias mitigation), privacy impacts, and security threats (such as prompt injection or data leakage).
• Transparency & Explainability: System error margins, performance metrics, and operational guidelines are made fully transparent to participating schools and data protection officers.

4. Data Governance & Model Safety

• Anonymisation at Ingestion: Identifying student details (names, dates of birth, UPNs) printed on script headers are separated from writing content via Google Cloud Vision (in-memory within the EU) before judging or processing takes place.
• Strict Prohibition on Model Training: Student writing and submission data are contractually barred from being used to train, retrain, or fine-tune underlying foundation or sub-processor AI models.
• Secure Cloud Infrastructure: Persistent student data and databases are stored on AWS S3 and MongoDB Atlas in Ireland (EEA), with transient API processing governed by strict 30-day retention limits.

Document Reference: NMM-HELP-EUAI-001
Owner: No More Marking Compliance & Security

Updated on: 25/09/2026