O11ySources

Welcome to the O11ySources User Guide! This guide is your essential resource for mastering Observability Sources (referred to as O11ySources hereafter) within the vuSmartMapsplatform. With O11ySources, you can effortlessly set up data sources, storyboards, and alert rules, streamlining your data pipeline and ETL process.

Prerequisites

Ensure vuSmartMaps is installed before configuring O11ySources and that you have admin access to configure O11ySources.

Getting Started with O11ySources

In today’s tech-driven world, O11ySources serves as your gateway to understanding your system’s performance, providing actionable insights into various critical aspects. O11ySources allows you to monitor a wide array of essential data sources, such as databases, network devices, applications, web servers, and more.

By collecting and analyzing data from these sources, O11ySources empowers you to optimize the performance of your systems. For example, it helps you track the response times of your applications, ensuring they run smoothly and efficiently. It also enables you to monitor server resource utilization, identifying potential bottlenecks that may impact your system’s responsiveness.

Whether you’re evaluating the health of your servers, tracking network performance, analyzing application behavior, or diving into the performance of your database queries, O11ySources provides you with the specific tools and insights needed to ensure the efficiency and effectiveness of the systems.

Data Onboarding with O11ySources

O11ySources, such as ‘Linux Monitor’ and ‘Apache’, seamlessly integrate with your existing setup. They consolidate data from multiple sources, creating a unified data flow that empowers your analysis and reporting systems. This integration streamlines data onboarding from target systems to enhance business observability with vuSmartMaps.

Data Flow Journey

Explore how data flows within the system by examining the key stages of the data journey:

  1. Data Ingestion: Agents like Healtheat, Logbeat, and Telegraf collect data from various sources using mechanisms like push or pull.
  2. Data Processing: The data undergoes transformations and enrichments, gaining context and domain knowledge. This phase of ContextStreams involves components like I/O streams and data pipelines, ensuring efficient data processing.
  3. Data Storage: The processed data finds its home in stores like Elasticsearch (ES) or HyperScale. These data stores form the basis for user functionalities such as alerts and dashboards, providing actionable insights.
  4. Data Consumption: The final stage is where the processed data is utilized through visualizations, dashboards, alerts, and reports. This is where users interact with the data, deriving meaningful insights, making informed decisions, and taking necessary actions based on real-time data analysis and historical trends.

To fully utilize the O11ySource framework, your journey starts with configuring data sources by enabling O11ySources. This guide will be your trusted companion, guiding you through these essential steps. As you delve deeper, you’ll gain a profound understanding of the vital components that keep your data flowing smoothly within your system.

Working with O11ySources

Accessing and managing O11ySources in vuSmartMaps is a simple process. Here’s how you can do it:

  1. vuSmartMaps O11ySources can be accessed by navigating from the left navigation menu (Data Ingestion > O11ySources).


  2. You will be directed to the O11ySources page, displaying a complete list of available O11ySources.


  3. Use the checkbox (EnabledDisabled) to filter the list, showing only enabled/ disabled O11ySources.


  4. By default, all O11ySources are disabled initially.
  5. Click on the O11ySource name you want to configure, such as “Apache.”
  6. Click the Enable button for the desired O11ySource. For example, we enable Apache as below:


💡Note: When O11ySources is enabled, it shows the Alert Rules and Storyboards linked to the displayed sources. If it is disabled, it hides them. Ensure that you’ve followed all the instructions in the Get Started section to enable data onboarding for seamless operations.

Linux Monitoring O11ySource – An Example

To understand the workflow more properly, as an example, we discuss the steps to configure the source for Linux Monitor in detail.

Importing O11ySources

The “Import O11ySources” feature allows users to upload new O11ySources or upgrade existing ones via a tar.gz file. This feature streamlines integrating new O11ySources or enhancing current ones, ensuring your observability framework remains up-to-date.

  1. Access the Import Feature: Navigate to the O11ySources landing page, and click the Import button.
  2. Upload the O11ySource File:
    • A dialog box will appear prompting you to upload a tar.gz file.
    • Select the tar.gz file of the O11ySource you wish to import (maximum file size: 0.29 MB).
    • The Upload button will be enabled once a valid file is selected.
  3. Import Process:
    • Click the Upload button to initiate the import.
    • If the uploaded file is a new O11ySource, it will be added in a disabled state.
    • If the file corresponds to an existing O11ySource, the current state (enabled/disabled) will be retained, and the upgrades will be applied.

💡Note: 

  1. New O11ySource: Upon import, new O11ySources will be added in a disabled state. You need to enable them manually after ensuring all configurations are correctly set.
  2. Existing O11ySource: If importing an upgrade for an existing O11ySource, it will retain its current state (enabled or disabled).
  3. Sources Configuration Changes: If the upgrade involves changes in the Source configuration, the import may fail due to potential backward incompatibilities. In such cases, contact the VuNet support team to confirm whether the upgrade can be performed using the import button. If not, the upgrade will be done through a product release update.

Each O11ySource comprises the following sections:

  1. Get Started
    This section provides specific details about the O11ySource and guides you through the process of configuring it and initiating data onboarding from your system into vuSmartMaps. Follow these instructions to kickstart the setup of your O11ySource and establish a seamless data flow.
  2. Sources
    Discover and manage data sources linked to the O11ySource. Configure various data collection methods and even connect to other O11ySources. This tab is enabled with O11ySources Diagnostics for streamlined configuration and troubleshooting, allowing you to improve data visualization and decision-making. Observe source status with color-coded indicators and diagnostics.
  3. ContextStreams
    This section offers a visual representation of data flow within an O11ySource, illustrating the journey of data through input streams, pipelines, output streams, enrichments, and connectors. It empowers users to troubleshoot issues, optimize performance, and gain insights into the data processing pipeline, enhancing overall system efficiency.
  4. Storyboards
    Explore and manage a collection of storyboards related to the O11ySource. These dashboards offer insightful visualizations for comprehensive data analysis. Filters can be applied to share dashboards across multiple O11ySources, enhancing your data analysis by tailoring the displayed information to your specific needs. This flexibility allows you to fine-tune your dashboards, providing a more contextually rich view of your data.
  5. Alert Rules
    Organize and maintain alert rules for effective real-time monitoring. This section offers a structured approach to configure and manage alerts, ensuring proactive issue resolution and system stability. Aligns with the concept of Storyboards for streamlined project management.
  6. Fields
    Access a comprehensive list of fields captured by the O11ySource, providing valuable insights into the available data and facilitating in-depth analysis and customization.

Each of these sections plays a crucial role in configuring and managing your O11ySource effectively. Follow along with this guide to harness the full potential of O11ySources and gain valuable observability into your systems.

O11ySources Diagnostics

Introduction

Welcome to the O11ySources Diagnostics feature, a powerful addition to the O11ySources framework. This feature empowers you with the ability to seamlessly configure your O11ySources integration while effortlessly diagnosing potential data-related issues. By gaining insights into the data collection process, you’ll be equipped to enhance data visualization and decision-making.

As an end user with the privilege to configure sources for O11ySources integration, the O11ySources Diagnostics feature is designed to offer you an intuitive configuration process coupled with swift troubleshooting capabilities. This empowers you to not only set up your data sources seamlessly but also to swiftly identify and resolve any data-related challenges. Ultimately, this feature enhances your control over data and enriches your user experience.

Purpose and User Benefits

The O11ySources Diagnostics feature serves the following purposes and brings corresponding benefits to users:

  1. Root Cause Identification: Understand configuration issues hindering data collection.
  2. Time-Saving: Swift diagnostics pinpoint the stage of data loss.
  3. Simplified Experience: Minimal technical knowledge required for efficient use.
  4. Visual Clarity: Simple UI showcases data transfer status at each stage.
  5. Proactive Issue Resolution: Clear insights lead to faster troubleshooting.

Performance and Scalability

The O11ySources Diagnostics feature effortlessly handles multiple diagnostics instances, ensuring minimal interference with other O11ySources. Even when dealing with an extensive number of data sources (~1000) and multiple agents, this feature continues to function robustly.

Understanding the Need for O11ySources Diagnostics

O11ySources simplifies source configuration but can pose challenges when data doesn’t appear as expected on dashboards. This complexity arises from the intricate data flow journey involving data ingestion, processing, and storage. Users often struggle to diagnose issues due to this complexity.

O11ySources Diagnostics steps in to bridge this gap, helping users understand the journey and troubleshoot effectively. This feature offers insights into the intricate data flow, making it easier to identify and resolve data-related issues. It ensures smoother operations and reliable data visualization, enhancing overall system efficiency.

The detailed instructions on working with O11ySources Diagnostics are discussed separately.

Further Reading

FAQs

O11ySources (Observability Sources) is a feature within the vuSmartMaps platform that helps you onboard various data sources like databases, applications, and servers. It provides insights into system performance, helps optimize resources, and ensures smooth operations. O11ySources is designed to handle large-scale data ingestion efficiently. It uses agents like Healtheat, Logbeat, Telegraf, etc. which ensure that data is ingested and processed seamlessly.

Use the O11ySources Diagnostics feature to identify and resolve data-related issues. This feature helps you understand the data flow journey and pinpoint where the problem lies.

The status of data reception is indicated by color-coded buttons (Green, Orange, Red) on the O11ySources page. Hover over these buttons for more details, and click if diagnostics are needed.

  • Green: Data reception is fine.
  • Orange: Data is missing from one or more sources.
  • Red: No data received at all.

Use the ContextStreams tab to see a visual representation of the data flow through input streams, pipelines, output streams, enrichments, and connectors.

Storyboards are dashboards offering insightful visualizations for data analysis, that are pre-packaged with O11ySources. You can access and manage these through the Storyboards section, which allows for filtering and sharing across multiple O11ySources.

The Fields tab provides a comprehensive list of all captured data fields, aiding in detailed analysis and customization. This shows the metrics observed with the respective O11ySource.

O11ySources is designed with a user-friendly interface, making it easy for beginners to set up data sources, configure alerts, and visualize data. The Get Started section provides step-by-step guidance, ensuring a smooth onboarding process.

Yes, by monitoring response times, server resource utilization, and application behavior, O11ySources helps you identify potential performance bottlenecks and provides actionable insights to resolve them.

Absolutely, O11ySources allows for extensive configuration of storyboards (dashboards) and alert rules. You can tailor visualizations and monitoring parameters to align with your specific business requirements.

The O11ySources Diagnostics feature will alert you if an agent stops working, showing a Red or Orange status indicator. You can click on the indicator to diagnose and resolve the issue, ensuring continuous data collection.

Check the logbeat agent configuration and ensure it is running properly. Use O11ySources Diagnostics to identify any issues in the data ingestion process. Verify connectivity to Data Storage and check for any errors in the logs.

Monitor memory usage patterns of your application over time using O11ySources. Set up alerts for abnormal memory usage spikes and visualize trends to spot potential leaks.

O11ySources is designed for flexibility and ease of use. You can quickly add a new data source through the Add Source interface, configure the necessary parameters, and start monitoring it immediately.

Resources

Browse through our resources to learn how you can accelerate digital transformation within your organisation.

Quick Links