Ehdotuksen tiedot

This document is aimed at those organisations wanting to create, fair, valid and reliable assessments in any context. The word high-stakes is often used, as opposed to low-stakes exams which are used in a formative context. The document takes a user through the process that they need to go through to create assessments and also identifies key roles within the assessment process. This Standard provides comprehensive guidelines on the application of information technology (IT) in delivering assessments. It outlines the scope of IT usage across various types of assessments, detailing the life cycle stages, security protocols, and privacy considerations specific to IT-enabled assessments.

 

The standard covers a broad range of assessment types, each tailored to specific contexts and objectives. These types of assessments ensure that the standard can be applied across various domains, from education to professional certification, and each has distinct characteristics and requirements. The key aspect of this is that the exams are high-stakes and have some specific use case, rather than for formative assessment, which typically has less rigorous requirements for creation and delivery. The primary types of assessment covered in the standard: a) K-12 Assessments b) Academic Assessments c) Employment Testing d) Professional Certification

 

The assessment lifecycle is a comprehensive process that encompasses the various stages involved in the creation, implementation, and evaluation of assessments. Although assessment procedures vary, the typical life cycle of assessment consists of the following steps: a) identification of need to assess b) design of outcomes/rubrics/assessment methodology c) creation, preparation and calibration of content d) preparation of assessment content (includes computer adaptive test engine/model) e) registration (may include payment) f) distribution g) authentication (includes identification) h) delivery i) response return (and/or submission) j) irregularities k) scoring, result determination and/or feedback l) data return m) analysis n) appeals and o) credentialing (certification, licensure, degree, badge, micro-credential, etc.) The design and instruments for computer adaptive testing and assessment methodology are not covered by this Standard. However, when IT is used for delivery, the relevant clauses of this Standard should be considered during the lifecycle.

 

The emphasis throughout this Standard is on the additional and distinct measures required with the integration of modern IT in assessment systems. While measures that are common to both traditional paper-based and IT-delivered assessments are mentioned only in general terms, this Standard addresses the novel challenges and opportunities introduced by digital technologies. This edition expands on the previous version by incorporating advanced technologies that were not previously covered. These include, but are not limited to, AI-assisted scoring, automatic feedback systems, adaptive testing, and remote proctoring solutions. The use of IT now supports a more comprehensive range of assessment functions: AI-Based Automated Scoring and Feedback: Modern systems employ machine learning algorithms to provide instant scoring and multi-dimensional feedback. Although the quality and validity of the assessment content remain paramount, IT is now capable of offering detailed, rubric-based evaluations that can be reviewed and validated by human markers. Adaptive Testing and Dynamic Item Banking: The introduction of computerized adaptive testing (CAT) and robust item banking systems allows assessments to adjust in real time to the test-taker’s ability, ensuring a more personalized and efficient measurement of competency. IT supports the maintenance and continuous updating of these item banks, including automatic generation and statistical calibration of test items. Remote Proctoring and Data Security: The transition to online assessments has necessitated the incorporation of remote proctoring technologies that combine AI monitoring with human oversight to mitigate fraud while safeguarding student privacy. This Standard specifies requirements for secure data transmission, storage, and compliance with international data protection regulations such as GDPR and ISO/IEC 27001. Learning Analytics and Data-Driven Decision Making: IT now plays a significant role in collecting and analyzing assessment data, enabling educators to leverage learning analytics for real-time feedback, formative assessment, and instructional adjustments. The interface between assessment content and IT delivery is enhanced to support a data-driven approach to both scoring and pedagogical decision-making. It is important to note that this Standard does not address the purely pedagogical aspects of assessment design or the intrinsic quality of assessment content. Rather, it focuses on the IT mechanisms that implement pedagogical decisions, facilitate the efficient transmission and secure handling of assessment data, and enable innovative evaluation methods. Compliance with this Standard does not imply that an assessment is pedagogically robust; rather, it ensures that the IT infrastructure supporting the assessment adheres to best practices in security, transparency, and reliability. The scope includes: IT scoring systems that are designed for subsequent human review; The IT-facilitated transmission and delivery of assessments, where responses may be scored either automatically or by human markers; The provision of automated feedback and immediate result reporting, while result-determination processes requiring human judgment remain outside this scope. By addressing these enhanced IT aspects, this Standard aims to support the evolution of assessment practices in a digital age, ensuring that technological innovations are harnessed effectively while maintaining the validity and fairness of the assessment process.

 

The aim of this Standard is to articulate overarching principles and best practices without prescribing specific technical implementations. The recommendations can be achieved through a variety of technological or procedural approaches, reflecting the expanded range of modern IT applications—including AI-assisted scoring, adaptive testing, and remote proctoring—without being tied to any specific hardware or software platform. In many instances, the principles outlined herein may be supplemented by additional regulations set by assessment providers, ensuring that while the underlying IT-enabled processes evolve, the core values of validity, fairness, and security remain paramount. 1.6 Compliance Assessment sponsors, assessment distributors and assessment centres may claim compliance with this Standard if they comply with all the clauses or subclauses applicable to their role (see table below). Notes to the clauses indicate the role(s) to which each clause or subclause is applicable. This Standard is applicable to both high-stakes and low-stakes assessments, but some clauses or subclauses are applicable only to high-stakes assessments; this is indicated in Table 1. The scenarios given in Annex A illustrate how different types of organisation might need to comply with different clauses of this Standard.