NLP-Powered Call Transcription and Sentiment Analysis for a Help Desk Software Product

About Our Client

The Client is a North American provider of help desk software solutions.

Opportunity to Enhance Help Desk Software with an NLP-Powered Customer Service Module

As of 2023, the Client has been partnering with ScienceSoft on software refactoring and DevOps projects for over two years. In an informal conversation, the Client and one of ScienceSoft’s senior software developers happened to discuss the latest advancements in neural networks, and our expert provided a few examples of how the technology could work for the Customer’s help desk software.

Natural language processing (NLP) models could automate call transcription and sentiment analysis, which would reduce manual work for support agents and provide new insights into service quality and client preferences. Our developer also pointed out that such functionality could be implemented with the help of open-source AI models, which won’t require licensing or usage fees. Inspired by the potential of these NLP capabilities, the Client hired ScienceSoft to implement them.

Building a Natural Language Processing Module MVP

The Client and ScienceSoft’s developer held a series of sessions during which they documented the exact AI features to be implemented: audio transcription, text classification and summarization, and client sentiment analysis.

As the Client wanted to test the feasibility of the proposed features first, it requested ScienceSoft’s developer to assemble the minimum viable NLP module without a full-scale UI or back-end integrations with the Client’s help desk software. If the MVP proved successful, the Customer would enhance the module with the needed components.

Since the Client already had an AWS-based cloud infrastructure, our developer suggested using AWS technologies to implement the module. He developed an MVP comprising five neural network models (see below), each of which is hosted in an individual Docker container. The containers are deployed on EC2 instances within AWS Elastic Container Service (ECS). The ecosystem is automatically managed by AWS Fargate technology. Thanks to its dockerized nature and automated container management, the module easily scales the cloud resources up and down based on the load for each container and optimizes resource utilization.

The system is available via a secure public endpoint governed by AWS Application Load Balancer. The balancer uses path-based routing algorithms to distribute requests across Docker containers.

Call Summarization and Sentiment Analysis to Streamline Customer Service

ScienceSoft developer delivered the following capabilities for the NLP module:

Instant audio transcription

The module transforms client calls into text. The Client is planning to use the transcriptions for analytics (e.g., client sentiment analysis, identification of trends in the requests and issues) and to train support agents.

This feature is powered by two AI models: Pyannote/speaker-diarization-3.0 to identify different speakers in a recording and Openai/whisper-base to enable speech recognition in 99 languages. Our developer chose these models because of their high accuracy and excellent context understanding.

Text classification and summarization

The module creates summaries of call transcripts. With this functionality, support agents don’t have to take notes manually, and help desk managers can quickly get an idea of service quality level.

Two models enable these capabilities. The Facebook/bart-large-cnn model can summarize extensive texts, understand different types of context, and retract salient information.

The MoritzLaurer/DeBERTa-v3-large-mnli-fever-anli-ling-wanli model demonstrates high performance on new data sets and can work with multiple topics and languages. The Customer also successfully applies this model to analyze emails that come to its main address and automatically redirect them to relevant department addresses.

Client sentiment analysis

Thanks to the SamLowe/roberta-base-go_emotions model, the app can classify texts (e.g., transcribed calls, chat messages) into one or several of 28 emotions (e.g., surprise, joy, anger). ScienceSoft’s developer went for this model due to its diversity of emotions and fine-tuning flexibility, which is essential for future analytics improvement.

Scalable AI Module MVP Ready in 3 Months

In just three months, the Client received an MVP of a natural language processing module that relies on five neural network models to enable call transcription, text classification and summarization, and client sentiment analysis. The module will be used in the Client’s help desk software product to facilitate customer service management and analytics. Thanks to its dockerized nature, the AWS-based module is highly scalable. Since all five NN models are open-source, the Client will be able to integrate them into help desk software without restrictive licensing or usage limitations.

Satisfied with MVP performance, the Client is planning to introduce the module in an upcoming update of its help desk product. The Client also plans to engage ScienceSoft to deliver more AI features in the future (e.g., chatbots for agents, speech synthesis, video recognition, comparative client-agent sentiment monitoring).

Technologies and Tools

Python, AWS Elastic Container Service, AWS Application Load Balancer, AWS Fargate.