Elevating Unified Observability with Ved: A Deep Dive into Our GenAI Copilot

Introduction

Ever been in one of those fire-fighting situations where you’re clicking through a dozen dashboards, trying to figure out what’s gone wrong this time? It’s like every incident turns into a detective story but without the fun of solving a mystery. Site Reliability Engineers, Customer Support Teams, and Business Leaders know this all too well. You’re juggling between screens, trying to connect the dots across different applications, and let’s be honest, it’s a headache.

Now, picture this:

What if you could just ask, “List the top 3 Payment Gateways that are showing higher failure rates compared to their daily averages and a sharp drop in the incoming requests in the last 15 mins?” or “Compare the error rate of transactions between the first and last week of November?” and get your answer straight away?

That’s where Ved comes in. It’s like having an assistant (like Jeeves from the PG Wodehouse novels) who happens to know everything about your system’s health and performance. No more screen hopping to infer the behavior or degradation. With Ved, you get to the heart of issues instantly, using plain old conversation. It’s about making your life easier and letting you focus on what really matters—solving problems and making decisions, not playing hide and seek with data.

While the dashboards still provide a unified view of your critical business and operational metrics, Ved provides a frictionless interface to quickly get to the root of certain queries. We have designed Ved as a powerful feature that would supercharge your observability journey with vuSmartMaps.

Unveiling Ved’s Capabilities

Ved stands out with its remarkable ability to interpret user inquiries across four critical domains:

• Transaction Analytics:  Gain deep visibility and actionable insights into complex transaction data from Instant Payments, Internet or Mobile Banking, and other financial systems. Ved’s advanced analytics capabilities allow you to seamlessly analyze transaction flows, identify patterns, and uncover valuable intelligence, empowering you to make data-driven decisions.
• Alert Analytics: You can leverage Ved’s proficiency to understand and analyze system alerts. This enables you to easily comprehend the overall health and performance of your systems.
• vuSmartMaps Documentation: Effortlessly navigate and comprehend the extensive vuSmartMaps documentation through natural language interactions with Ved. Whether you’re a seasoned user or new to the platform, Ved ensures you can quickly find the information you need, seamlessly guiding you through configuration steps, feature explanations, and best practices.
• Enterprise Knowledge: Harness the power of Ved to unlock insights from any text data you possess. VuNet provides access to generally available and custom knowledge while allowing customers to seamlessly integrate their enterprise-specific data for personalized experiences.

Whether you’re exploring the nuances of instant payment transactions, delving into alert analytics for system health monitoring, or seeking guidance on vuSmartMaps configurations, Ved is designed to simplify your footprint. Through simple conversational interactions, Ved streamlines complex analyses, making observability more accessible and intuitive than ever before.

Ved’s Architecture

Ved’s effectiveness lies in its robust architectural design. We have meticulously engineered Ved to handle a diverse array of user queries with precision, overcoming the common uncertainties posed by large language models (LLMs). This system seamlessly identifies the underlying intent behind each user query, leveraging advanced intent classification techniques. Ved then queries relational or vector databases, efficiently retrieving the necessary information to address the user’s needs. Finally, Ved synthesizes the retrieved data into comprehensive responses with charts where necessary, delivering actionable insights with clarity and accuracy.

The above architecture diagram is a visual representation of the complete life cycle of a user query within Ved’s system, from its initial query submission to the generation of the final response. The query’s journey commences at the Ved User Interface, where users input their inquiries. This initiates a sequence of events that traverse through various components of the system’s architecture.

Intent Classifier:

The Intent Classifier deciphers the underlying intent or purpose of the query, thereby determining the appropriate processing path – whether it necessitates interactions with relational databases or vector databases.

Relational Database Query Processing:

For structured data queries, such as those related to transaction analytics or alert analytics, the system engages with the HyperScale Data Store. The Text-to-SQL Generator seamlessly translates the user’s natural language query into a corresponding SQL query, facilitating the retrieval of relevant data from the relational database. We invested significant effort into carefully crafting the prompts to ensure that the queries generated by this system precisely capture and cater to the users’ requirements.

Vector Database Query Processing:

For queries about unstructured data, such as user guide documents or custom text knowledge, the system leverages VectorServe, the in-house developed RAG framework. In this scenario, the embedding model converts the user’s query into embeddings (array of numbers), enabling efficient vector searches within the appropriate collections of a vector database.

Response Summary Generation:

Once the requisite data is obtained from the respective databases, Ved employs specialized components like the SQL Response Generator and the RAG Response Generator to process the raw data and formulate coherent, natural-language responses. Additionally, the Chart Data Generator can generate visual representations (charts) based on the query results, thereby enhancing the user experience.

Ultimately, the final response, synthesized from the collective outputs of these various components, is presented to the user through the Ved User Interface, concluding the query’s transformative journey from inception to a tailored and accurate answer.

Design Considerations and Implementations

Large Language Models (LLMs): Currently, OpenAI’s GPT-4 serves as the backbone of Ved, offering unparalleled knowledge and analytical capabilities. This state-of-the-art large language model (LLM) provides the foundation for Ved’s remarkable performance. We plan to support more open-source models in future.

Embedding Models: To ensure fast performance and precise responses, Ved employs state-of-the-art open-source embedding models. These models excel at efficiently capturing and representing the intricate relationships within text data as vectors (arrays of numbers), enabling rapid and accurate information retrieval from vector databases.

Databases: VuNet’s commitment to robust and scalable data management is exemplified by the choice of our databases to provide seamless insights into structured and unstructured data.

Vector Database: For vector-based semantic search capabilities, Ved leverages VectorServe, a Retrieval Augmented Generation (RAG) framework developed in-house which is built on top of the Weaviate vector database. This combination ensures efficient storage and retrieval of information, enabling precise and relevant search results.

Relational Database: On the relational database front, Ved employs HyperScale Data Store, a powerful solution that offers scalability, flexibility, and a comprehensive set of features. This database choice ensures Ved leverages huge enriched data already present in the vuSmartMaps platform.

Orchestration Framework: Ved operates on our proprietary orchestration framework, Muscat, designed for high efficiency and adaptability, setting it apart from generic frameworks available in the market. Muscat achieves up to 2 times faster query processing, enabling Ved to deliver accurate responses swiftly, essential for fluid real-time interactions for various user personnas from SREs to business teams. Ved can handle up to 1000 user queries simultaneously. Its flexibility enhances our capability to incorporate new technologies rapidly, boosting Ved’s performance continuously.  This tailored, fine-tuned framework gives us complete control over optimizing Ved’s underlying technologies, ensuring a level of customization often lacking in off-the-shelf alternatives.

With this potent combination of cutting-edge technologies and models, Ved stands poised to deliver accuracy and efficiency.

Overcoming Implementation Challenges

The development of Ved was not without its fair share of challenges. From mitigating the risk of hallucinations through meticulous prompt engineering to reimagining traditional architectural frameworks for reduced latency and increased reliability, our team went through a journey of learning and overcoming challenges. This led us to develop specialized prompts that optimize and refine the expertise of GPT models like GPT-4 for our specific use cases. We call these systems as ExpertGPTs which ensure Ved’s responses are remarkably accurate.

Our team learned invaluable lessons, gaining insights into the intricacies of prompt engineering, architecture optimization, and the deployment of cutting-edge AI technologies. This hard-earned knowledge has reinforced our commitment to continuous innovation and our pursuit of excellence in the field of AI for observability.

Impact on Customers and Key Personas

We envision Ved as a guide that empowers SREs, business analysts, and other professionals to achieve their goals with greater efficiency and precision. For SREs, Ved simplifies monitoring and analytics, allowing for proactive incident management. Business analysts gain instant access to deep insights, facilitating informed decision-making. Transcending across roles, Ved stands as a catalyst for productivity, elevating operational excellence through its intuitive and powerful capabilities.

Conclusion

As mentioned earlier, Ved offers a frictionless interface to receive answers to your queries quickly at your fingertips. The Key differentiator of  Ved lies in bootstrapping LLMs with our internal knowledge base and in-house expertise in financial systems.

In our next blog, we will deep-dive into the use cases we can undertake with Ved.  With Ved’s advanced technical architecture, our customers can look forward to a future where data insights are more accessible, actionable, and impactful than ever before. Welcome to the next level of business journey observability, as we transcend from Observability to Recommendations and Actions