Recent AI failures: lessons learned point the way forward for AI-enhanced ITSM

Recently, the tech and business press have been focused on “failed” AI projects. While it may be unfair to consider these implementations outright failures – in the cases cited here, the earlier approaches were ultimately refined and positive outcomes delivered – it’s worth considering what initially went wrong, and how ITSM-focused organisations can avoid the pitfalls and ultimately find success.

Let’s consider three high-profile cases:

IBM replaced 8,000 employees with their "AskHR" chatbot, only to find that the system struggled with complex tasks, especially interactions requiring judgment or empathy. In the face of a significant hit to customer service levels IBM rehired many of the original workers.

Klarna deployed chatbots to reduce customer service headcount by 700. Customers reported that bots failed to address their inquiries, lacked conversational flow – and would begin to misinterpret the initial inquiry over time. Ultimately, Klarna reduced the scope of their rollout to only the most routine inquiry types and hired back customer service agents for more complex tasks.

In late 2024 Duolingo decreased their staff of human linguists, and rapidly added 148 new courses created by Generative AI. This sparked a significant backlash among Duolingo’s typically loyal user base. Users reported that content seemed “artificial” and “soulless” and responded with a raft of public cancellations.

Three main narratives emerge from the recent coverage:

🔄 AI rollouts that focus on rapid cuts in headcount often result in later rounds of rehiring

🤖 The technology can struggle with empathy, judgment, and complex problem-solving

📉 Customer satisfaction often drops with impersonal AI interactions

Does this apply to AI in ITSM?

It's important to bear in mind that that these apparent failures emerged from use cases that weren’t purely ITSM. AI has a good track record in delivering positive outcomes around ITSM workloads like summarising ticket descriptions, improving communication with customers, and the use of predictive models for request categorisation and proactive problem management.

Still, there is a cautionary tale for us here. Interactions with customers and complex problem solving are key elements in ITSM, and these are the common denominator in the less-than-optimal AI projects cited above. It’s worth asking – why?

Basic research is often overlooked in the marketing material, but it may provide some answers here. Recent findings suggest that:

• customers have lower levels of trust in AI-created content, particularly when making important decisions, and

  
  

• they view AI bots negatively, believing that the motivation behind the technology is more about profit and cost-cutting than improving service

• Apple’s recent paper - still the subject of fierce debate at time of writing – suggests that current AI models may struggle with reasoning tasks commonly performed by humans.

  
  

The research suggests that the success of AI critically depends not just on how it is rolled out, but also the problems that we expect AI to solve. ITSM is a rewarding space to work in because it combines many disciplines, such as organisational psychology, business analysis, technology, and customer service. Businesses that succeed in ITSM do so by innovating in each of these spaces – so it is worth considering how AI might contribute to each domain.

So what does work for ITSM?

We’ve arrived at three main insights that point to how ITSM practitioners should be approaching AI technology:

✅ Best results come from augmenting humans, not replacing them

🎯 The focus should be on repetitive, standardisable tasks

❌ Automation without a clear purpose (that is, AI for AI’s sake) often fails

In our experience the following approach works best.

Start small

Identify a part of the business that would benefit from AI automation - ideally with a minimum of organisational change or human training.

Consider workloads that require manual, repetitive tasks that are downstream from activities that provide value for the business

Use this both to demonstrate value, and to learn how to roll out the technology at your organisation more generally

Measure the right outcomes

It's critical that you have clear outcomes, and your metrics should reflect these.

This should go beyond simple cost savings -

consider broader efficiencies like mean-time-to-resolve, staff morale due to less repetitive work, and customer satisfaction.

Use you pilot to begin the process of developing your metrics.

Focus on unique learning last

Generative AI models are pre-trained on massive bodies of generic data. Out of the box they can be applied to general tasks, like summarisation, categorisation, and co-piloting. These can provide quick wins up front.

However, your most valuable data is that which is unique to your business: ticket records, CAB minutes, how-to guides, all found within internal documents stores and knowledge bases.

Successfully extending AI to this content is wholly dependent on the quality of available data - mature pre-existing ITSM and knowledge management processes are critical.

Concluding remarks

We feel that ITSM-focused organisations are uniquely placed to benefit from AI technologies. This isn’t despite the recent examples of sub-optimal AI rollouts, but rather because of them. The lessons we can learn allow us to form an evidence-based view of AI based on real world outcomes, rather than just taking statements about AI’s potential at face value.

In its current state AI is not ready to replace humans in high-value customer-facing tasks, or those requiring complex and creative reasoning. Large-language models are designed to find patterns in noisy data. Models trained on past customer service interactions will, as per design, deliver very average customer service.

However, AI truly excels at categorisation, pattern matching, search and language tasks like summarising content. For ITSM the potential is enormous – for organisations facing bottlenecks like manual ticket categorisation and routing, proactive problem management, and estimating planned change impacts across an array of interconnected Config Items, the benefits are significant.

Automated agents, too, have their place – when deployed appropriately. If portal users can be quickly assisted by a bot for simple tasks, and then smoothly handed off for more complex issues, customers will feel a direct benefit.

It is critical to identify what areas of the business AI is appropriate for and start there – then proceed with a rational approach based on business objectives. At Lida we have found success by working with several organisations using this approach, and we're ready to get you started as well.