DocsData Ingestion and Processing > O11ySources > O11ySources Workflow

O11ySources Workflow

Get Started

This section is your gateway to configuring the O11ySource effectively. When you click on an O11ySource, this is the main screen that opens, providing detailed and essential information:

  • Overview
  • Prerequisites
  • Configuration

In this section, we guide you through the process of configuring your O11ySource and initiating data onboarding from your system into vuSmartMaps™. Follow these instructions to kickstart the setup of your O11ySource and establish a seamless data flow.

Sources

In the Sources section, you’ll find a comprehensive list of data sources associated with your O11ySource. These sources encompass a wide range of data collection methods, from JMX, log observability, and health collection to SNMP, and more. Additionally, sources can include other O11ySources like Apache, MySQL, Netflow, and UPI Java App, providing versatility in data collection.

One of the key features of this tab is its integration with O11ySources Diagnostics, offering a streamlined approach to configure and troubleshoot your data sources. This powerful combination enables you to enhance data visualization and decision-making. Within this section, you can monitor source status with color-coded indicators and access diagnostic information. The color codes—Green, Amber, and Red—offer quick insights into data reception, making it easy to identify and address potential issues.

The status of each source is shown here in Red, Amber, or Green Color. Hovering over these status buttons reveals a text box popup that comprehensively details the status:

  1. Green: Data Reception working fine – Not clickable
  2. Orange: Data not received from one or more sources – Click for more info or perform diagnostics
  3. Red: No data received at all – Click for more info or perform diagnostics

The details of these are discussed in the O11ySources Diagnostics section.

Discover and manage data sources effortlessly within this section, ensuring that your O11ySource is collecting data efficiently. Explore various data collection methods and easily connect to other O11ySources, all while streamlining configuration and troubleshooting through O11ySources Diagnostics.

Add a Source

Users can add a new source by clicking on the + button at the top right corner of the table.

Enter the required details and configurations for the source.

💡Note: The source creation depends on the type of O11ySource and the process varies based on the O11ySource. The details are provided for respective O11ySource in the Get Started tab.

Provide all the required information to add the source and proceed ahead to the Download Agents & Configurations.

Export Source

Click on the Export All button to export all the sources at once.

Confirm by clicking on the Yes button. The source file will be exported and downloaded as source.xlsx.

You can use the downloaded source for external purposes.

Import Sources

Click on the Import button to export all the sources at once.

This will open a dialog box to upload the data source file. Click on the Upload button to import the data source.

Sources Actions

Edit Source

You can edit any of the existing sources by clicking on the Edit Icon in the Actions column.

You have to Download Agents & Configurations and after completing the editing of data source configurations.

Click on the Finish button after making a selection.

You can select the configuration and click on the Submit button to download the same.

Delete Source

To delete any particular data source, use the delete icon for the respective source. Upon confirmation, this particular data source will be deleted.

Clone Source

You can copy any of the sources.

More Details

It shows more detailed information about the source.

ContextStreams

This tab enhances the O11ySource details page by offering users a visual representation of the data flow through various stages within an O11ySource. It showcases the journey of data, including its path through input streams, pipelines, output streams, enrichments, and connectors. This feature is designed to provide users with a comprehensive understanding of how data is processed and transformed within the O11ySource, ultimately aiding in troubleshooting, performance optimization, and gaining insights into the data processing pipeline.

It enables users to visualize the intricate data flow within an O11ySource. By presenting a clear overview of the relationships between different data sources, transformations, and destinations, it equips users to diagnose issues and gain valuable insights into their data processing pipeline. This comprehensive information offers detailed insights into the ContextStream i.e., I/O streams, pipelines, enrichments, and connectors associated with each source in the O11ySource.

Upon accessing the ContextStreams Workflow, you will encounter a visually engaging representation of your O11ySource’s data journey. Here’s what you can expect:

  1. Input Streams: The agents send the data from the source to the Input Streams (Input Topic).
  2. Pipelines: The pipelines represent the various stages of data processing and transformation.
  3. Output Streams: The processed or transformed data is then sent to the Output Streams (Output Topic).
  4. Enrichments: Enrichments are visualized, showing how additional context and information are added to the data.
  5. Connectors: Connectors reveal how data is sent to external systems or destinations.

Each element within the ContextStreams Workflow is clickable, allowing you to navigate to respective ContextStreams pages to access more details.

Clicking on an Input Stream node will lead you to a dedicated page where you can explore in-depth information about the source of the data, its configuration, and its performance.

  1. Input Stream
    Clicking on an Input Stream node will lead you to a dedicated page where you can explore in-depth information about the input stream, its configuration, and its performance.


  2. Pipeline
    Clicking on a Pipeline node will take you to a detailed view of that pipeline, including transformation logic, data processing speed, and any potential bottlenecks.


  3. Output Stream
    Clicking on an Output Stream node allows you to examine where the processed data is sent. You can access insights about the destination, data format, and transmission speed.


  4. Enrichment
    For Enrichments, clicking on these nodes reveals the specific enhancements applied to the data, helping you understand how additional context is injected.


  5. Connector
    Lastly, Connectors provide insights into how data is relayed to external systems. Clicking on a Connector node takes you to a dedicated page outlining the connection details and performance metrics.


The ContextStreams Workflow in O11ySources is a valuable addition that empowers users to dive deep, offering insights into data flow and processing. With this, you can troubleshoot issues, optimize performance, and make informed decisions about your data processing configuration. Enjoy the benefits of visual clarity and enhanced control over your data processing journey with ContextStreams Workflow.

Storyboards

Within the Storyboards section, you’ll find a compilation of various storyboards tailored for your O11ySource. Each O11ySource comes with its dedicated set of dashboards, meticulously designed to address specific requirements. These dashboards offer optional filters, granting you the versatility to share the same dashboard across multiple O11ySources while enriching it with additional context. Delve into this assortment of storyboards to explore insightful visualizations, facilitating in-depth data analysis and aiding you in making informed decisions.

Filters enhance the flexibility of these storyboards, allowing you to adapt them to various O11ySources and their specific data sets. This feature promotes a unified and efficient approach to visualizing and analyzing data across your observability framework.

The dashboards can be accessed with the View button under the Actions column. An example of Apache Server Insights dashboard is shown below:

Alert Rules

In the Alert Rules section, you’ll find a list of alert rules designed specifically for your O11ySource, similar to how storyboards are managed. This feature introduces a structured and centralized approach to alert management, parallel to the organization and management of visual elements through storyboards.

The Alert Rules feature empowers you to define, configure, and oversee your alerting requirements effectively. This structured approach ensures proactive issue resolution and promotes system stability and reliability. Aligning with the concept of storyboards, this section streamlines project management, enabling you to maintain control over real-time monitoring and issue resolution within your observability framework. For more information related to alerts, you can refer to the alerts user guide.

Fields

In the Fields section, you’ll find an extensive list of data fields captured by your O11ySource. This repository of fields offers valuable insights into the available data, providing the foundation for in-depth analysis and customization.

Accessing this comprehensive list equips you with the knowledge required to tailor your O11ySource to meet your specific requirements and enhance its data processing capabilities.

Resources

Browse through our resources to learn how you can accelerate digital transformation within your organisation.

Quick Links