Wall Street Giants Warn of New AI Risks: Hallucinations, Cybercrime, and Employee Morale

Wall Street's Growing Concerns Over AI Risks

Major Wall Street firms, including Goldman Sachs, Citigroup, and JPMorgan Chase, are sounding the alarm on new risks associated with the increasing use of artificial intelligence (AI) in the financial sector. These concerns, highlighted in their annual reports, encompass a range of issues from AI hallucinations to cybercrime and workforce challenges 1

1

2

2

.

AI Hallucinations and Data Quality

One of the primary concerns raised by banks is the risk of AI hallucinations – instances where AI systems produce inaccurate or unreliable outputs. Citigroup, for example, warned about the potential for "ineffective, inadequate or faulty" results from generative AI systems being rolled out in select parts of the bank 3

3

. The quality and accuracy of data used to train AI models is a critical issue, with JPMorgan emphasizing the importance of maintaining the highest level of "data quality" in their AI models 3

3

.

Cybersecurity Threats

The adoption of AI technologies has also opened up new avenues for cybercriminals. Banks are increasingly concerned about sophisticated AI-powered cyberattacks. According to a survey by Accenture, 80% of cybersecurity executives in the banking industry believe that generative AI is empowering criminals faster than banks can respond 3

3

. This highlights the urgent need for robust cybersecurity measures to keep pace with AI advancements.

Workforce Displacement and Morale

JPMorgan's annual report pointed out that AI could lead to "workforce displacement," potentially affecting staff morale and retention 1

1

. This shift in the employment landscape could also intensify competition for employees with specialized technological skills 2

2

. The integration of AI into various job functions is forcing banks to reconsider their hiring strategies and employee development programs.

Regulatory Challenges

The rapidly evolving regulatory environment surrounding AI presents another significant challenge for financial institutions. The implementation of the EU Artificial Intelligence Act in 2024 has established new rules for AI systems in the region, affecting many U.S. banks with European operations 3

3

. However, the regulatory outlook in the U.S. remains less clear, adding to the complexity of compliance efforts.

Competitive Pressures

While AI offers numerous benefits, it also introduces competitive pressures. Goldman Sachs noted that failing to integrate AI technologies in a timely manner could affect their ability to attract and retain customers 3

3

. Banks are now in a race to leverage AI for improved productivity, cost reduction, and enhanced customer services.

Balancing Innovation and Risk

Financial institutions are walking a tightrope between embracing AI innovations and managing associated risks. As Marco Argenti, Chief Information Officer at Goldman Sachs, stated, "It's so important to take a responsible approach and really be applying controls so that you protect yourself from potential inaccuracies and hallucinations" 3

3

.

The Way Forward

To address these challenges, banks are implementing governance mechanisms and controls to ensure safe, fair, and secure deployment of AI. As Ben Shorten from Accenture emphasized, "This is not a plug-and-play technology" 2

2

. The financial sector's approach to AI will need to evolve continuously, balancing innovation with robust risk management strategies.

1

1

: https://www.bloomberg.com/news/articles/2025-03-12/wall-street-flags-new-ai-risks-from-hallucinations-criminal-use 2

2

: https://www.pymnts.com/artificial-intelligence-2/2025/banking-giants-warn-ai-can-bolster-cybercrime-and-lower-morale/ 3

3

: https://www.seattletimes.com/business/wall-street-warns-of-new-ai-risks-from-hallucinations-criminal-use/