RBI mandates kill switch for AI models as banks face sweeping new AI risk management rules

RBI mandates kill switch for AI models at banks, introduces comprehensive model risk framework

The Reserve Bank of India is implementing stringent rules for banks and financial entities using Artificial Intelligence. A new draft framework mandates 'kill switches' for AI models, ensuring immediate shutdown if errors occur. Banks must also ensure human oversight, disclose AI use to customers, and manage risks associated with third-party AI providers. Board-level accountability for AI governance is a key focus, with a risk-based approach to model oversight. Mumbai: The Reserve Bank of India has mandated that banks and all other regulated entities must have the ability to instantly override, suspend or deactivate any artificial intelligence model deployed in their operations -- including a kill switch arrangement -- as part of a sweeping draft framework on Model Risk Management. The draft guidelines, released for public consultation, calls for requirement for robust human oversight of all AI-driven decision-making. Banks must establish override, suspension and deactivation mechanisms -- including kill-switch arrangements -- to ensure that no AI model can operate without the ability to be immediately shut down if it produces harmful or erroneous outputs. Also read: RBI expands digital fraud protection, introduces compensation for small-value scam victims The RBI has also flagged the risk of automation bias -- the tendency of bank employees to over-rely on AI outputs without applying their own judgment. For customer-facing AI systems, banks must disclose to customers that they are interacting with an AI system and must provide them with the option to switch to a human at any point. The framework also introduces a risk-based tiering structure that requires regulated entities to classify all models -- from simple spreadsheet-based calculators to complex frontier AI systems -- by their risk level, and apply proportionate oversight, validation and controls accordingly. The risk tier of a model must be reviewed at least annually. High-risk models require approval from the Risk Management Committee of the Board before deployment -- they cannot be cleared by the technology or risk team alone. Board-Level AccountabilityFor the first time, the RBI is placing AI and model governance squarely at the board level. Every regulated entity must have a Board-approved Model Risk Management Framework covering all models -- whether built internally, sourced from vendors, or a combination. The board is responsible for approving the entity's risk appetite for model risk, setting policies for model risk tiering, and ensuring these are forward-looking and informed by stress testing and scenario analysis. Also read: RBI to raise large exposure limit for upper layer NBFC-IFCs to 45% from 35% of eligible capital base Third-Party and AI Vendor ModelsThe draft guidelines take a particularly firm line on third-party models -- including AI platforms and models sourced from fintech and technology vendors. A regulated entity remains fully accountable for the outcomes of any model it uses, regardless of whether it built the model itself or bought it from outside. The RBI has specifically flagged supply chain risk -- the risk arising from over-dependence on a limited number of AI model providers -- as a concern that banks must actively manage. This is a pointed reference to the growing concentration of AI capabilities in a handful of global technology companies. Explainability, Bias and FairnessFor AI and ML models, the RBI has introduced a set of specific requirements that go beyond conventional model validation. Banks must define explainability thresholds for all AI models -- the ability to explain, in understandable terms, why a model produced a particular output.

Lowdown: RBI wants banks to add sill Switches, human oversight for AI models - MEDIANAMA

Banks and other regulated entities (REs) may soon have to ensure every artificial intelligence (AI) system they deploy can be overridden, suspended or deactivated through "kill-switch arrangements", under the Reserve Bank of India's (RBI) draft Guidance on Regulatory Principles for Model Risk Management, 2026. Released for public consultation on Wednesday, the draft proposes a broad framework governing how banks, NBFCs and other financial institutions develop, deploy and oversee models, including AI systems. Stakeholders can submit comments until July 24, 2026. The guidance would apply to commercial banks, small finance banks, payments banks, local area banks, co-operative banks, regional rural banks, NBFCs, all-India financial institutions, asset reconstruction companies and credit information companies. The guidelines also clarify that "further requirements, if any, applicable to AI models may be issued later". Why is the RBI issuing these guidelines? The RBI said REs are increasingly using models to improve customer service, automate business processes, strengthen risk management and defend against cyber attacks. It attributed this to the growing scale and complexity of financial activities, digitalisation of financial services, advances in analytical and computational capabilities, AI and machine learning (ML), and greater reliance on third-party providers. The draft defines a "model" broadly. It includes any internally developed or third-party system that uses data and statistical, mathematical, financial or AI/ML techniques to generate outputs used for business operations or decision-making. It also covers algorithms, analytics, applications, decision-based rules and other computational tools that materially affect business decisions, regardless of whether an RE formally classifies them as models. The AI-specific provisions apply to AI and ML models, including foundational AI models and frontier AI models. The RBI said REs should define their scope and implement additional controls based on their potential impact on customers, business operations and financial outcomes. Board oversight and governance: The RBI proposes that every RE establish a Board-approved Model Risk Management Framework (MRMF) covering governance, model tiering, inventory, documentation, validation, approvals, monitoring, change management, business continuity and decommissioning. The Board would oversee the framework, while the Risk Management Committee of the Board would review high-risk models, monitor third-party and AI models, review model tiering at least annually and oversee breaches or other material concerns. The RBI also makes clear that REs remain accountable for the outcomes of every model they use, whether developed internally, sourced from third parties or a combination of both. AI should only be used where risks can be managed: The RBI said REs should assess whether risks arising from AI models can be adequately identified, measured, monitored and managed before deploying them. AI models should only be used in business processes and use cases where those risks can be effectively managed. The draft also requires REs to classify every model according to risk. For AI models, this assessment should additionally consider "the extent of reliance and the level of autonomy placed on the model outputs for decision-making." Models with greater autonomy or greater reliance on their outputs could therefore attract higher risk classifications. For material third-party AI models, datasets and dependencies, REs should also consider risks arising from dependence on a limited number of providers, supply chain risks, limitations in independently validating models and behavioural changes resulting from provider-driven updates. AI must remain under human control: REs would have to establish robust human oversight arrangements for AI models, including automated decision-making systems. These arrangements should include: * Human-in-command mechanisms, such as human-in-the-loop or human-on-the-loop oversight. * Override, suspension or deactivation mechanisms, including kill-switch arrangements. * Periodic human review of model outputs and AI-driven decisions to identify anomalies. The oversight mechanism should also account for automation bias, over-reliance on AI outputs and decision fatigue. Additionally, personnel responsible for overseeing AI systems should possess sufficient expertise to "effectively challenge, override, or escalate issues/concerns in model outputs where required." REs should periodically review human interventions, overrides, incidents and near misses to strengthen oversight arrangements. How RBI wants AI models to behave: The draft requires REs to define explainability and transparency thresholds for every AI model. Models relied upon for material decision-making or having significant impact on customers or operations should meet higher explainability standards. Where full explainability cannot be achieved, the RBI said REs should implement enhanced validation and testing mechanisms to verify and corroborate model outputs before use, more frequent validation and monitoring, usage restrictions and other compensating controls. The draft also directs REs to establish "appropriate control boundaries" to mitigate hallucinations, particularly in generative AI models and use cases where AI outputs directly or indirectly influence customer interactions or decision-making. Additionally, REs should identify risks of bias and discriminatory outputs, particularly where AI could unfairly treat certain customer groups. They should conduct fairness assessments and implement mitigants, including recalibration or redesign where necessary. The RBI further said REs should ensure AI models: * are not overfitted to training data and can generalise to real-world conditions; * do not rely on spurious correlations or unintended relationships; * do not produce excessive or unexplained variation under similar inputs, and where such variability, stochastic behaviour or model uncertainty exists, it should be managed through measures such as confidence scores and probability outputs; and * continuously monitor and address data quality issues, non-representative or incomplete datasets, intellectual property risks, data drift and concept drift. How RBI wants AI models tested: * Independently validate every model, including third-party models, regardless of any validation, certification or assurance provided by the service provider. Validation should assess the model's inputs, assumptions, conceptual soundness, design, performance and alignment with its intended use. * Validate models throughout their lifecycle, including before deployment, after deployment, following modifications, on internal or external triggers and periodically under the MRMF. * Test AI models under stressed conditions to ensure vulnerabilities do not emerge through edge cases, abnormal inputs, manipulations and adversarial conditions. * Carry out structured challenge processes, including red-teaming or equivalent testing, particularly for models involving customer interaction or generative AI capabilities. * Apply additional controls to models that receive dynamic or automatic updates, including defining what can be updated automatically, justifying automatic updates, conducting enhanced data quality checks, and monitoring such models more frequently. * Maintain enhanced documentation for AI models to ensure traceability, reproducibility, and auditability, reflecting their complexity, self-adapting nature, and reliance on training data. Stronger controls for third-party AI: Before acquiring or using a third-party model, REs should carry out due diligence covering the credibility of the service provider, the model's methodology and limitations, and the suitability and quality of data used. Where a third-party provider does not disclose sufficient information about an AI model, REs should identify risks arising from those constraints and implement mitigants, including limiting the model's use. The RBI also proposes that contracts with third-party providers should provide REs with access to sufficient technical documentation to understand, validate and audit models, while also covering audit rights, continuity arrangements and exit mechanisms. Deployment controls: REs should ensure model outputs are replicated and stable in the production environment before deployment. AI models should also not introduce vulnerabilities into either the model itself or the RE's production environment. REs should implement access controls, cybersecurity safeguards and controls covering APIs, external interfaces and third-party integrations. For customer-facing AI systems, including generative AI models, REs should implement additional cybersecurity controls against prompt injection, adversarial inputs, persistent sessions and anomalous usage patterns. They should also clearly disclose that users are interacting with an AI or ML-based system, explain its limitations and provide customers with the option to switch to human assistance whenever requested.

RBI proposes guidelines for banks to manage AI risks

MUMBAI, June 24 (Reuters) - India's central bank has proposed rules requiring banks to strengthen oversight of risks tied to AI and machine-learning models, mandating board-approved policies, stronger controls and model inventories. The Reserve Bank of India said banks must put in place a board-approved risk management framework covering all models, including those for AI and machine-learning. Regulated entities must assess risk at both the individual model level and across the enterprise on an ongoing basis, the RBI said. If risks are found to be excessive, lenders should take timely corrective steps, including enhanced controls, restrictions on use, remediation or decommissioning of the model, and submit a report to the board's risk management committee, the bank added. The RBI also said banks should ensure all models, including third-party models, are subject to independent validation. Banks should establish human oversight for AI models used in automated decision-making, the draft guidelines said. For generative AI models that interface with customers or external users, additional cybersecurity controls should be implemented. RBI has invited feedback on the draft guidelines by July 24. (Reporting by Gopika Gopakumar; Editing by Shailesh Kuber)