The European Central Bank ("ECB") has published its opinion on the EU's proposed regulation of artificial intelligence, with a particular focus on how the AI Act could affect credit institutions. The Basel Committee on Banking Supervision (BCBS) has also published a newsletter on AI and machine learning.
These latest publications follow recent commentary from other financial regulators on the use of AI and Machine Learning in the financial services sector (which we wrote about here).
What is the EU Artificial Intelligence Act?
The EU AI Act is one of the first legal frameworks proposed by a major regulator concerning AI. It aims to address the risks stemming from the various uses of AI systems and to promote innovation in the field of AI.
Is the EU AI Act relevant to businesses established outside the EU?
The AI Act applies to providers of AI systems who are based outside the EU but place AI systems on the market or put AI Systems into service in the EU, irrespective of whether a provider is established in the EU or a third country (i.e. the Act would apply to a UK business that places AI systems in the EU).
What is the scope of the EU AI Act?
The definition of AI systems under the proposed Act is very broad, covering software that is developed to generate outputs such as content, predictions, recommendations, or decisions, using statistical and machine learning approaches, as well as search and optimisation methods.
The Act proposes a risk-based framework, tackling unacceptable-risk, high-risk, limited-risk and minimal-risk systems. AI system providers would have to implement differing thresholds of safety measures according to the risk levels, with some AI systems being completely prohibited under draft proposals (e.g. the use of "real-time" remote biometric identification systems in publicly accessible spaces for the purpose of law enforcement, unless the use is strictly necessary for certain specified objective).
What did the ECB say in its opinion?
The ECB commented on various matters with particular reference to credit institutions, including the extent to which credit-scoring AI systems should be categorised as high risk, conformity assessments of AI systems, and the need to update the list of high-risk AI systems proposed in the AI Act.
More generally, the ECB noted that certain requirements for high-risk AI systems needed to be clearer and more specific. The design and development requirements for high-risk AI systems under the AI Act, referred to by the ECB, include that such AI systems should be created:
(1) on the basis of training, validation and testing data sets that meet certain quality criteria where they make use of techniques involving the training of models with data;
(2) with capabilities enabling the automatic recording of events (‘logs’) to ensure a level of traceability of the system’s functioning throughout its lifecycle that is appropriate for its intended purpose;
(3) in a way that ensures that they are effectively overseen by natural persons, including human-machine interface tools, in order to prevent or minimise the risks to health, safety or fundamental rights that may emerge when a high-risk AI system is used; and
(4) for the purpose of achieving an appropriate level of accuracy, robustness and cybersecurity.
The key to achieving the desired level of transparency and explainability will ultimately lie in more detailed implementing provisions.
Similar themes highlighted by the BCBS
The BCBS notes that banks are maintaining a level of transparency in model design, operation, and interpretability of model outcomes commensurate with the risk of the banking activity being supported. It also calls out the challenges around data governance in ensuring data quality, relevance, security and confidentiality. From a practical perspective, the BCBS points out that banks should have appropriately skilled staff to deal with such systems, which may include model developers, model validators, model users and independent auditors.
Next steps
We await greater clarity on the matters discussed above as the EU AI Act proceeds through the legislative process.