LLM-Supported Smart Search for Mobile Banking App Users

About Our Client

The Client is a European commercial bank with over $1.6 billion in assets. The bank is recognized among the top high-tech financial institutions in its domestic market due to its commitment to superior digital customer services and continuous banking innovation.

Basic Search No Longer Met Mobile Banking User Expectations

The bank’s client-side iOS and Android banking apps were equipped with basic search features that relied on exact-match keywords. The search was limited to isolated databases, each connected to specific functional parts of the app (e.g., the search of deposit options was available only in the deposit module, and of loan terms – only in the credit module). Such restrictive functionality forced mobile banking users to either guess the precise search terms or manually navigate multiple app sections to find specific banking information. Also, the old search engine failed to accurately interpret the context of users' queries, so search results often appeared irrelevant or incomplete. Over time, this friction led to lower user engagement with the apps and increased demand for direct customer support.

The bank wanted to upgrade its mobile banking apps with more intuitive search functionality. As modern consumers got accustomed to intelligent search assistants that understand natural language and context, the bank considered using generative AI for instant user communication and higher-quality search results.

The bank already had a 10-year-long story of fruitful cooperation with ScienceSoft. It first commissioned our team to develop its mobile banking apps in 2014, and ScienceSoft’s engineers have been maintaining the apps since their launch. Trusting our mobile banking development skills and decades-long experience in data analytics and AI, the bank decided to involve ScienceSoft in the implementation of the new search functionality.

Technical Design and Implementation Plan for LLM-Powered Smart Search

ScienceSoft assigned a project manager, a business analyst, and a data science engineer with experience in banking IT to the project. The business analyst conducted several interview sessions with the bank’s stakeholders to identify their pains related to the current search engine and elicit requirements for the new solution. Additionally, she surveyed the bank’s customer support specialists to gain deeper insights into the issues mobile banking customers typically faced with the old search features. Following this analysis, our expert assisted the bank in formulating and refining the solution concept and translated it into a detailed functional specification.

Next, ScienceSoft’s data science engineer suggested the optimal technical realization for the smart search solution. He designed the solution to enable hybrid search supported by large language models (LLM). This would allow instant synthesis of data from isolated databases, search across unstructured data like the bank’s multi-format knowledge base, and the delivery of highly relevant and nuanced responses.

Our expert introduced a high-level architecture for the advanced hybrid search solution, reflecting its key functional components:

• The data preprocessing module where a banking customer’s natural language query is automatically cleansed, enriched, and tokenized for further search.
• The lexicographic search engine runs the initial query against the data from the bank’s databases. ScienceSoft’s expert suggested applying BM25 ranking algorithms for enhanced search speed and higher output relevance.
• The semantic search engine employs machine learning (ML) algorithms for natural language processing (NLP) to interpret the context and intent behind the user’s query and retrieve semantically equivalent data from the bank’s databases.
• The vector database stores the chunked and vectorized multi-format data from the bank’s disparate databases. Vectorizing data first is required for efficient semantic search. ScienceSoft’s engineer picked high-performing commercial vector databases and mapped pragmatic ways to automate data vectorization pipelines.
• The search result fusion engine combines the data retrieved during lexicographic and semantic search to create a comprehensive response context.
• The reranking engine analyzes the retrieved data pieces, sorts them by relevance, and composes the optimal response.
• The optional LLM module can further enhance the hybrid search by analyzing the context of textual and voice queries and communicating directly with mobile banking users. To optimize solution costs, ScienceSoft’s expert suggested employing pre-trained LLMs and enhancing them with bank-specific knowledge using a cost-effective retrieval augmented generation (RAG) technique.

The data science engineer composed a tailored set of KPIs so that the bank could control the quality of smart search results. The set covered specialized metrics for data retrieval performance (Precision@k, mAP, MRR, NDCG) and LLM evaluation (F1 score, BLEU, ROUGE, perplexity).

The proposed approach to implementing smart search in mobile banking apps assumed the development of a complex, innovative solution. ScienceSoft suggested starting with a proof of concept (PoC). This way, the team could test the performance and accuracy of various search methods in a controlled environment and identify potential improvements early on.

ScienceSoft’s team recommended that the search algorithms should be trained and tested on the data shown to consumers in the mobile banking app, such as data about the bank’s products, services, exchange rates, service points, promotions, and new offerings. This would let our engineers evaluate the quality of client-side search results more precisely.

After the bank approved the smart search solution’s technical design and PoC plan, ScienceSoft’s project manager scoped the tasks for the PoC stage and determined the required resources. He also estimated the cost and time to complete PoC development and mapped the steps to mitigate potential technology, operational, and financial risks.

Consulting on Cost-Effective IT Infrastructure for Smart Search

The bank’s strict information security policies didn’t allow deploying the smart search solution in the cloud. As such, the bank needed to set up a dedicated on-premises infrastructure that would accommodate complex hybrid search operations, AI model training, and storing large data volumes. ScienceSoft’s engineer checked the capabilities of the bank’s as-is IT infrastructure, determined its hardware gaps, and advised the Client on the optimal CPU, GPU, RAM, and SSD modules to add to ensure sufficient solution performance and scalability.

Our expert explained that while GPUs are generally more efficient for vector search, high-end CPUs with a high core count can still smoothly handle smaller workloads and data sets. He suggested implementing less expensive CPUs instead of GPUs at the initial project stages, which could help the bank reduce early investments.

PoC-Ready Smart Search Solution Concept Mapped in 4 Weeks

In just 4 weeks, the bank received a functional concept, a technical design, and an implementation plan for its customer-side smart search solution. ScienceSoft’s advice on the cost-effective solution tech stack and hardware ensured optimized project investments. Starting with a PoC allowed the bank minimize project risks and stay confident about the technical and economic feasibility of its solution prior to full-scale developments.

As of December 2024, ScienceSoft has rolled out a test environment and is progressing with database integration.

Based on our estimates, bringing LLM-supported search in mobile banking apps at scale will give our Client the possibility to boost CSAT by 7%+, cut the servicing teams’ workload by over 60%, and increase financial product cross-selling potential.

Technologies and Tools

• Programming languages: Python.
• Vector databases: Faiss, Qdrant, Milvus.
• Pretrained LLMs: Llama3.1, BERT.
• NLP and LLM libraries: spaCy, NLTK, Hugging Face Transformers.
• ML frameworks: PyTorch, TensorFlow.
• DevOps tools: Docker, Kubernetes.