Docs > Configuration > Log Analytics 

Log Analytics

Introduction

Log Analytics on the vuSmarMaps platform empowers users to explore, analyze, and visualize logs across applications, including those from various banking systems. With its intuitive interface and advanced querying capabilities, Log Analytics allows users to efficiently retrieve log data, identify trends, and gain actionable insights from vast volumes of log information.

The module optimizes log ingestion by focusing on relevant logs and leveraging data compression to primarily reduce storage costs. Additionally, it provides the flexibility to drop logs by context streams, further reducing costs and enhancing system performance. Compatible with S3 API-based object storage, Log Analytics also supports flexible object storage with the option to retrieve data on demand, ensuring that users can access historical logs whenever needed. Role-based access control (RBAC) ensures secure access, allowing only authorized users to view sensitive logs at both the dashboard and Log Analytics module levels. Log masking further enhances data privacy and compliance by concealing sensitive information in logs (refer to this section for more details on data masking).

To streamline querying, Log Analytics features Vunet Query Language (VQL), enabling users to build complex queries quickly and conduct precise, effective searches. This combination of features allows for rapid root cause analysis (RCA) within seconds, driving faster issue resolution and minimizing downtime.

From interactive visualizations to powerful search and filtering options, Log Analytics on vuSmarMaps provides a robust solution for streamlined log management, empowering teams to make informed, data-driven decisions with ease.

Getting Started

Accessing Log Analytics

  1. The Log Analytics page can be accessed from the platform left navigation menu by navigating to Observability -> Log Analytics.
  2. When you click on Log Analytics, you will be directed to the following landing page.

User Interface Overview

At the landing page of log analytics, you have the provision to select Data Store and Table to start the new analysis, Hyperscale is selected as default data store.

1. Data Store allows you to choose the particular Data Store

2Table option allows you to choose from where you want to fetch the logs.

💡Note: Previously, all tables in the selected Data Store were listed on the Log Analytics page. Now, only the tables a given user has access to are displayed.  For more details, refer to the User Management Guide.

The Data Stores are database connection instances configured in the data modeling workspace. Each Data Store defines a separate connection to a database instance, either internal or external, using which data analysis can be done. VuNet’s HyperScale database instances are the only types of Data Stores supported for log analytics. The default internal HyperScale Data Store instance will be available in all systems for use in log analytics.

Once the Data Store and Table are selected, the log analytics module provides a listing of matching logs for the time selected, without any other filters.

💡Note: To ensure the table appears in the Log Analytics table listing, it must contain the following fields: timestamp, message, log_uuid, and message_lower.

  1. Conversely, you have an option to load the previously saved search.

2. Once the Data Store and Table are selected you will be landed to the following page.

3. If you want to change the selected data store and table, click on the currently selected data store at the top left.

4. You will be redirected to the page where you selected the Data Store and Table. Now, make the desired selections and click the Start a New Analysis button.

5. At the top right corner, you’ll find the global time selector, allowing you to gain insight into the logs at a specific time range selected.

6. Besides the time filter, there’s a Live data toggle button that, when enabled, provides the user with the real-time display of the most recent 500 records from the last five minutes, automatically refreshing every 10 seconds.

7. Refresh button beside the Live data toggle enables data refresh, and the User Guide tab redirects you to the Log Analytics User Guide.

Below are the action buttons, each serving a specific purpose:

  • Save: Clicking the button opens a context menu with two options. For new searches, the ‘Save’ option is disabled; you need to use ‘Save as’. For a saved search, use ‘Save’. In an already saved search, ‘Save As’ functions like a clone.
  • New/Load: This button allows you to initiate a new search or access previously saved searches.
  • More Actions: Here, users can grant permissions on saved searches.

  1. The search query field allows you to input VQL-based text queries to retrieve specific logs from the Table.
  2. Clicking the ‘VQL’ button on the search bar opens the VQL help section documentation in a new tab.

3. The Build Filter option located next to the search box allows you to add filters. By clicking ‘Add Filter,’ you can select columns and operators to filter out specific logs in the table.

4. The Columns option on the left enables you to choose which columns are displayed in the Table. By default, the ‘timestamp’ and ‘message’ fields are selected on the Table.

💡Note: Column options may vary depending on the Data Store and selected Table. A maximum of twenty five columns can be selected at a time.

On the right-hand side, the area chart and log table are displayed based on the selected Data Store, Table, and time range. This enables you to visualize trends and analyze log data precisely tailored to the chosen parameters, enhancing the effectiveness of the data exploration and analysis within the platform.

For further details on how to effectively analyze and visualize log data, refer to the ‘Exploring Log Data‘ page.

Further Reading

FAQs

To search for logs with specific keywords, use the search query field on the Log Analytics page. Enter your keywords, and the system will display matching logs. For more advanced search options, such as case-sensitive searches or composite queries, refer to the Using Text Queries section in the user guide.

To save a search query, click on the ‘More Actions’ button, select ‘Save’, and provide a name and description. You can access saved searches by clicking the ‘Open’ button. For step-by-step instructions, refer to the Saving and Reusing a Search section.

You can use the negate operator ~ to find logs that do not contain a specific keyword. For example, ~Error finds all logs not containing ‘Error’. Refer to the Negate the Operation section for more information.

Yes, use the has keyword for case-insensitive searches on specific fields. For example, log_level:has(war) finds all logs with ‘war’ in the log_level field. See the Case-insensitive Search on a Field section for more details.

Yes, you can assign access permissions to different user roles by navigating to More Actions > Permissions while saving a search. This allows you to control who can view or modify the saved searches. Detailed instructions are available in the Access Permissions of Saved Search section.

You can construct complex queries using logical operators (AND, OR) and specific field conditions. For example, error + Server finds logs containing both ‘Error’ and ‘Server’. Refer to the Searching Multiple Keywords section for examples.

The live data feature provides up-to-the-minute information, essential for monitoring critical operations in real time. This feature ensures that teams can respond swiftly to any emerging issues, maintaining system stability and performance.

By analyzing historical log data, Log Analytics can identify usage patterns and peak times, helping organizations plan for capacity needs and optimize resource allocation. This ensures that the system can handle load variations efficiently without over-provisioning.

Customizing columns allows users to focus on the most relevant data points, improving clarity and making it easier to spot trends and anomalies. This tailored view helps in conducting more precise and meaningful analyses.

Log Analytics improves the overall user experience by providing detailed insights into system performance and user behavior. This enables quicker issue resolution, better system optimization, and ultimately a smoother, more reliable experience for users on the vuSmartMaps platform.

Resources

Browse through our resources to learn how you can accelerate digital transformation within your organisation.

Quick Links