Too Smart To Care? Study Finds Emotion AI Can Backfire In Customer Service

New Study Explores AI’s Role in Emotional Customer Care

A new study in the journal Management Science by Yifan Yu, a Texas McCombs assistant professor of information, risk, and operations management, offers companies guidance on how to balance the promise and perils of AI for customer care.

With McCombs postdoctoral researcher Wendao Xue, he analyzes AI systems that detect human emotions, known as emotion AI, and how companies might deploy them in various scenarios.

"Firms can refine how they use AI to ensure fairer, more effective decision-making," says Yu. "Our study provides a practical framework for businesses to navigate this balance, particularly in customer care, where emotional communication plays a crucial role."

Game Theory Reveals How Emotion AI Affects Interactions

Yu and Xue, with co-authors Lina Jia of the Beijing Institute of Technology and Yong Tan of the University of Washington, used game theory to model interactions among customers, employees, and companies. Variables included a customer's level of emotional intensity, how much recompense an employee can offer to satisfy a customer, and costs and benefits to the company.

Overall, the analysis showed emotion AI works best for customer service when it's integrated with human employees. Some kinds of scenarios were handled better by AI and others by people. Yu shares some principles for both.

Integrating Emotion AI into Customer Support Systems

"Many companies already use AI to handle basic customer inquiries," says Yu. "Adding emotion AI could help these systems better gauge frustration, confusion, or urgency.

"Instead of providing one-size-fits-all responses, the chatbot could tailor its approach based on the detected emotions, offering quicker solutions or escalating the case to another agent when needed."

AI as the First Line of Customer Response

Emotion AI can decrease emotional toll and employee turnover by serving as a first point of contact with irate customers. Humans can step in when more nuance is required or when customers demand more.

Choosing the Right Channel for Emotion AI

In public channels such as social media, where other users might be watching, human customer service may handle customer complaints with more sensitivity. Private channels such as customer phone calls might be a better use case for emotion AI.

When a Weaker AI Performs Better

Noise in the emotion AI system, random or irrelevant data, may make it harder to game the system and discourage customers from trying. Therefore, a weaker AI, with higher levels of noise in recognizing emotions, may sometimes better regulate gaming behaviors and increase the system's social benefits.

"Normally, companies assume that better emotion recognition leads to better decisions," Yu says. "But we found that when AI is too strong, customers are more likely to game the system by exaggerating their emotions, creating a 'rat race' of emotional escalation. This leads to misallocated resources and an overall loss in efficiency."

Beyond Complaints: Broader Applications of Emotion AI

For businesses, emotion AI could handle more than customer complaints, he says. It could help screen job candidates and monitor employees. For any of those uses, though, he recommends keeping a human component.

Keeping Humans in the Loop for Emotional Intelligence

"AI has made remarkable strides in reasoning and problem-solving, often surpassing human capabilities in these areas," says Yu. "But its ability to understand and respond to human emotions is still in its early stages."