Duplicating Zepto's Multilingual Query Resolution System: A Step-by-Step Guide
Zepto, a leading technology company, has introduced a cutting-edge multilingual query resolution system that significantly improves search accuracy and user satisfaction. This innovative system leverages a combination of Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG) to correct misspellings, handle multiple languages, and provide a seamless user experience.
The system's primary function begins with RAG, which retrieves the top-k most relevant product or brand names that closely match the user's noisy or misspelled query. This is achieved by using FAISS, Facebook's similarity-search engine, to return the closest brand and product names in the high-dimensional vector space.
These retrieved terms are then fed, along with the original noisy query, into an LLM. This powerful language model interprets, disambiguates, and corrects the misspellings and phonetic variations, generating a clean, corrected query. The BAAI/bge-small-en-v1.5 model is used for creating product embeddings, while Meta's Llama3-8B, hosted on Databricks, is employed for cost control and performance.
The corrected query is then used to improve search results, ensuring that users find what they are looking for with minimal misunderstandings and retrieval errors caused by noisy inputs. Zepto's system demonstrates a robust, scalable architecture that provides a clear path to significantly improving user experience and search conversion rates.
Moreover, the system's components are chained together using LangChain Expression Language (LCEL), creating a seamless flow from query to final result. User feedback is also utilised to improve the system, with new few-shot examples, synonyms, and bug fixes added as necessary.
In summary, Zepto's multilingual query resolution system effectively addresses query noise in a multilingual setting, ensuring a high-quality search experience for users worldwide. The system's ability to correct misspellings and slang with high accuracy, provide structured, auditable outputs, and disambiguate queries using retrieved context, sets it apart as a game-changer in the field of search technology.
Data science plays a crucial role in the development of Zepto's multilingual query resolution system, as it relies on advanced Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG) for its functionality. Such technology, leveraged within home-and-garden or data-and-cloud-computing industries, could potentially revolutionize search engine performance, providing users with artificial-intelligence-powered search experiences. In the future, it might even be possible to apply this technology to lifestyle domains, enhancing various online search experiences.
