Overview
ISO/IEC TS 25058:2024 - “Systems and software engineering - Systems and software Quality Requirements and Evaluation (SQuaRE) - Guidance for quality evaluation of artificial intelligence (AI) systems” - provides a structured guidance for evaluating AI system quality using an AI system quality model. Published as a Technical Specification by ISO/IEC JTC 1/SC 42, it is applicable to all types of organizations involved in the development, deployment, or use of AI. The document aligns AI quality evaluation with the SQuaRE approach to systems and software quality.
Key Topics
The specification defines an AI-focused quality evaluation methodology and addresses quality characteristics familiar to SQuaRE, adapted for AI systems. Major topics include:
- Quality evaluation methodology for AI systems and use of an AI system quality model.
- Functional suitability aspects such as completeness, correctness, appropriateness, and adaptability.
- Performance efficiency covering time behaviour, resource utilization, and capacity.
- Compatibility including co-existence and interoperability with other systems.
- Usability dimensions like recognizability, learnability, operability, accessibility, user controllability and transparency.
- Reliability attributes: maturity, availability, fault tolerance, recoverability and robustness.
- Security elements: confidentiality, integrity, non-repudiation, accountability, authenticity and intervenability.
- Maintainability (modularity, reusability, analysability, modifiability, testability) and portability (adaptability, installability, replaceability).
- User-centered measures: effectiveness, efficiency, satisfaction (trust, usefulness, pleasure, comfort, transparency).
- Freedom from risk and risk mitigation across economic, health & safety, environmental, societal and ethical dimensions.
- Context coverage and the extent to which an AI system meets required operational contexts.
Applications
ISO/IEC TS 25058:2024 is practical for:
- Defining quality requirements and acceptance criteria for AI products and services.
- Designing evaluation plans, test cases and benchmarks for AI system performance and safety.
- Supporting procurement specifications, vendor assessment, and contract clauses for AI systems.
- Performing internal QA audits, independent conformity assessment, and post-deployment monitoring.
- Guiding risk assessment and mitigation strategies that include ethical, safety and environmental considerations.
Who should use it
- AI developers, system architects and QA teams
- Product managers and procurement officers
- Compliance, audit and risk-management professionals
- Regulators, certification bodies and researchers focused on AI quality and safety
Related standards
- SQuaRE family (ISO/IEC 25000 series) and other ISO/IEC JTC 1 and SC 42 AI standards provide complementary guidance for quality models, requirements and conformity assessment.
Keywords: ISO/IEC TS 25058:2024, AI quality evaluation, SQuaRE, AI system quality model, AI system assessment, AI risk mitigation, AI transparency.
Open preview (PDF)Systems and software engineering — Systems and software Quality Requirements and Evaluation (SQuaRE) — Guidance for quality evaluation of artificial intelligence (AI) systems
