European AI confidence and assurance framework
=================================================================================
In the rapidly evolving world of AI, safety concerns have emerged as a critical issue, particularly as AI technologies become increasingly integrated into daily life. However, unlike other critical sectors, AI lacks the safety guarantees that are expected. This is where independent third-party testing comes into play.
Academic research indicates that self-assessments deliver lower safety and security standards compared to accredited third-party or governmental audits. Recognising this, the focus has shifted towards mandatory independent third-party testing for advanced AI products and systems.
The European Union (EU) is leading the charge with regulatory frameworks like the EU AI Act, effective from August 2, 2025. This Act establishes stringent requirements for high-risk AI systems, mandating conformity assessments before these systems can be placed on the market. These assessments can involve internal controls but may also require independent third-party assessments by notified bodies to verify compliance with safety, transparency, and risk mitigation standards.
Key features of these third-party assessments under the EU AI Act include verification of training and testing data sourcing and preparation, validation of system robustness, accuracy, and risk controls, documentation of governance measures, and enforcement through CE marking and mandatory registration of AI systems in an EU database.
Post-market monitoring is also mandated, requiring providers to continuously collect and analyze data on AI system performance, with obligations to report serious incidents and cooperate with authorities throughout the lifecycle of the AI product.
The EU's approach extends traditional quality assurance mechanisms to AI systems through the European Quality Infrastructure (EQI). This integration includes embedding rigorous conformity assessment frameworks, creating a clear regulatory infrastructure, and promoting a culture of professionalisation in AI assurance.
Regulatory sandboxes and testing, experimentation facilities can offer a controlled testing environment for AI models. The independence and competence of conformity assessment bodies in the EQI ecosystem is ensured by accreditation.
While regional and national governments have made commitments to some external scrutiny and testing for the most advanced AI products, these are not yet mandated. Adversarial testing by independent experts can help uncover potentially dangerous features in AI models. Independent assessments and testing should include audits or evaluations of data quality, model robustness, accuracy, and bias.
Establishing an assessment ecosystem for AI models will take time and significant resources. However, it is a necessary step to minimise potential harms of AI technologies and ensure safety standards are maintained.
Science, technology, and finance intersect in the development of AI regulatory standards. The European Union (EU) is pioneering this intersection with regulatory frameworks like the EU AI Act, which establishes requirements for third-party testing and assessment bodies accredited by finance organizations to ensure AI systems conform to safety, transparency, and risk mitigation standards. This financial support will bolster the development and operation of testing facilities, such as regulatory sandboxes, to provide controlled environments for AI model testing and fostering a culture of professionalisation in AI assurance.
