Building Trustworthy AI: What Capital Market Participants Should Be Doing Today

Introduction

The regulatory conversation around artificial intelligence in capital markets is moving from identifying risks to considering how those risks can be supervised. IOSCO’s 2026 Supervisory Toolkit for AI Use in Capital Markets is an important development in that progression. The Toolkit is intended to support securities regulators in supervising AI systems used by regulated entities, including through practical supervisory tools covering areas such as governance and risk management, third-party and outsourcing arrangements, disclosure, and recordkeeping and reporting.

For market participants, however, the significance of these developments extends beyond the question of what a regulator may eventually ask during an inspection. The more immediate question is whether firms can explain, document and govern the AI systems already being used across their businesses.

This does not necessarily require a separate AI compliance regime. It requires firms to understand where AI is being deployed, what decisions it influences, who remains accountable for those decisions, and whether existing governance, technology-risk and operational-resilience frameworks can address the associated risks.

This is particularly relevant as AI becomes embedded in business-critical technology environments. An AI inventory can provide not only visibility into where AI is being used, but also help firms identify which AI-enabled processes support critical or time-sensitive operations and what arrangements exist if those systems become unavailable.

Start With Visibility: Know Where AI Is Being Used

The first governance challenge may be more basic than model validation or explainability: knowing where AI is actually being used.

AI adoption does not necessarily occur through a single centrally approved project. Different business, technology, compliance and operations teams may adopt tools independently, including externally hosted generative AI applications and AI-enabled products embedded within existing software.

That creates a practical governance question: can an institution identify the AI systems being used across its front, middle and back offices?

An effective internal AI inventory can provide that visibility. It could identify the purpose of each system, the business function in which it is deployed, the data it accesses, whether it influences decisions affecting clients or markets, the relevant technology or service provider, and the controls applied to its use.

This is particularly important because not every AI application presents the same level of risk. A tool used to summarise publicly available documents may require a different control environment from a system that contributes to investment decisions, executes trades, assesses clients or supports market surveillance.

The objective should therefore be risk-based visibility, rather than treating every AI application identically.

IOSCO’s 2025 report notes that firms are developing governance and risk-management frameworks covering areas including AI strategy, model and data risks, third-party risks, accountability and human oversight.

Treat AI as a Lifecycle, Not a One-Time Approval

A second consideration is that AI governance cannot end when a system is approved for deployment.

AI systems can change through retraining, changes in data, model updates, modifications by vendors or changes in the way employees use them. Consequently, the relevant question is not simply whether a model was appropriate when it was introduced, but whether it remains appropriate for its intended use.

A lifecycle-based approach can therefore be more effective than a one-time approval process. Before deployment, firms can assess the intended purpose, data sources, performance characteristics and potential impact of the system. During deployment, monitoring can focus on accuracy, reliability, unexpected outputs and changes in performance. Lifecycle governance should also address material model or vendor changes, incidents, system unavailability and the measures required to intervene, suspend an AI-enabled process or revert to an alternative process where necessary.

This becomes particularly important for generative AI. Outputs may appear plausible while containing errors, creating a different type of operational risk from conventional deterministic software.

For capital market participants, this reinforces the importance of maintaining appropriate human oversight for material decisions, alongside defined escalation, intervention and fallback mechanisms. IOSCO identifies human oversight as an important governance principle, noting that AI should augment rather than replace human decision-making and judgement. The objective is not to eliminate human involvement from every AI-enabled process. It is to determine where human judgement remains essential and how that judgement should operate alongside the system, including when the system produces an unexpected output or becomes unavailable.

The objective is not to eliminate human involvement from every AI-enabled process. It is to determine where human judgement remains essential and how that judgement should operate alongside the system.

Third-Party AI Requires More Than Vendor Due Diligence

AI governance becomes more complicated when firms do not develop the underlying technology themselves.

Capital market participants may depend on cloud infrastructure, foundation models, specialist AI providers or AI functionality embedded within third-party applications. This can create dependencies that extend beyond traditional outsourcing arrangements and can introduce questions of concentration, substitutability and operational resilience. Where multiple critical processes depend on the same technology provider, cloud infrastructure or underlying model, a disruption or material change at that level may affect several functions simultaneously.

The IMF has highlighted the possibility of increasing concentration of AI systems and models among key providers and market participants. It also notes that accelerated technology adoption can create supervisory challenges where authorities need to understand dependencies and potential concentration risks.

For firms, the practical implication is that procurement should not be the end of the assessment. Contracts, data arrangements, access controls, information-security obligations, incident notification, business continuity and rights to obtain relevant information may all become important considerations depending on the use case.

The critical question is:

“If an AI system becomes unavailable, produces a materially incorrect output, changes materially following a vendor update or becomes compromised, can the firm detect the problem, intervene, switch to an alternative process and remain operational”

These questions connect AI governance directly with existing frameworks for third-party risk, outsourcing, cybersecurity, business continuity and operational resilience.

Documentation and Accountability Will Become Increasingly Important

As AI becomes integrated into business-critical processes, the ability to demonstrate how it is being used may become almost as important as the technology itself.

A firm should be able to establish, at an appropriate level of detail, what an AI system is intended to do, who approved its use, what data it relies upon, what controls apply to it, how its performance is monitored, what material changes have occurred and who is responsible for addressing failures or deciding whether an AI-enabled process should continue, be suspended or revert to an alternative process.

This does not mean that every AI system requires extensive technical documentation accessible to every employee. Rather, documentation should be proportionate to the significance of the application and sufficient to support oversight, audit and accountability.

IOSCO’s 2025 analysis identifies accountability, transparency, reliability, security, risk management and human oversight among the principles reflected in governance approaches being adopted by larger financial firms.

This is also where AI governance intersects with existing legal responsibilities. The introduction of an AI system does not, by itself, transfer responsibility from the regulated entity to the technology provider or the model.

The more AI influences consequential decisions, the more important it becomes for firms to maintain a clear chain of accountability.

What Should Indian Market Participants Take from This?

India does not currently have a single, dedicated framework governing all uses of AI across securities markets. That, however, does not place AI governance outside the existing regulatory architecture. Rather, AI-related considerations are emerging alongside broader requirements concerning technology governance, cybersecurity, operational resilience and business continuity within the securities-market ecosystem.

SEBI’s May 2026 advisory on advanced AI tools for vulnerability detection illustrates one aspect of this emerging approach. The advisory highlighted the potential use of AI for vulnerability detection and autonomous or agentic mitigation, while also drawing attention to the risks posed by AI-accelerated cyber threats and the need for continuous vulnerability management. This technology-risk focus has subsequently been accompanied by further measures concerning the resilience of market infrastructure.

In August 2026, SEBI introduced an IT Resilience Index for Market Infrastructure Institutions (MIIs), adding a more structured dimension to the assessment of technology resilience. SEBI has also separately addressed the alignment of its cyber incident reporting framework with the FIRE format. In September 2026, SEBI further proposed measures to strengthen Business Continuity Plans and Disaster Recovery arrangements for MIIs. These developments are not AI-specific rules, but they are relevant to AI governance because AI-enabled processes increasingly operate within the same technology infrastructure and dependency chains that firms must keep resilient.

More broadly, SEBI has addressed the use and reporting of AI and machine-learning applications and systems by Market Infrastructure Institutions since 2019. Viewed together, these developments suggest that AI governance for Indian market participants cannot be considered in isolation from the wider technology, cyber-risk and resilience environment in which AI systems operate.

These developments should not be read as evidence of what a future comprehensive AI framework in India will contain. They do, however, demonstrate that technology governance is not an entirely new consideration for Indian securities-market participants.

For firms preparing for the next phase of AI adoption, the practical priority is therefore not to predict the precise form of future AI regulation. It is to establish whether existing governance structures can answer basic questions about AI inventory, accountability, data, third-party dependencies, human oversight, monitoring, incident response, business continuity and recovery. The ability to demonstrate how an AI-enabled process can be controlled during normal operations, and how the firm can intervene or continue critical activities when that process fails or becomes unavailable, may become increasingly important as supervisory expectations around technology resilience develop.

Conclusion

Responsible AI adoption in capital markets is increasingly becoming a question of institutional capability. As AI moves from experimentation into critical business processes, firms need sufficient visibility, governance, accountability and resilience to understand how these systems operate, how their dependencies are managed and how risks will be addressed when circumstances change.

The emerging international approach does not suggest that AI should be treated as inherently incompatible with financial markets. Instead, it points towards a model in which innovation is accompanied by appropriate governance, risk management, human oversight, transparency and resilience. IOSCO’s latest supervisory work reinforces this direction by giving regulators practical tools to assess how regulated entities govern and oversee their use of AI. At the same time, developments in securities-market technology and operational resilience demonstrate that AI governance is increasingly part of a broader conversation about how financial institutions manage critical technology and third-party dependencies.

For market participants, preparing for this environment does not necessarily mean waiting for a dedicated AI rulebook. It means building the institutional ability to demonstrate that AI is being used deliberately, proportionately and accountably, while ensuring that critical AI-enabled processes can be monitored, challenged, interrupted and, where necessary, supported by appropriate fallback and recovery arrangements. The next question is therefore not simply how firms should govern AI, but how emerging AI supervisory expectations may become integrated with the broader regulatory architecture for technology, cybersecurity and operational resilience in capital markets.

Authored by: Anshu Bansal, Partner, and Rashmita Das, Legal Researcher.