vuApp360 Deployment and Configuration

vuSmartMaps Configuration: Enabling Distributed Traces Telemetry

Traces O11ySource collects Distributed Traces telemetry from applications to provide comprehensive observability, offering deep insights into application performance through standard applications performance monitoring dashboards. 

Enabling Traces O11ySource and Configuring Data Sources

1. Navigate to O11ySources:
   • vuSmartMaps O11ySources can be accessed by navigating from the left navigation menu (Data Ingestion > O11ySources).
     
     
     
2. Locate Traces O11ySource on the landing page.
   
   
   
   
3. Enable Traces O11ySources:
   • Click the Enable button to activate Traces O11ySource.
     
     

Configuring Data Sources

1. Navigate to the Sources tab and click the + button to add new services to be monitored.
   
   
   
2. Fill up the Following Details in the Wizard:
   • Applications Group: Name of the application group.
     
   • SSL Required: If Yes, TLS will be turned ON.
   • Application Services: Specify the name of your application and service that will be displayed in the observability UI. In a microservices architecture, an application can consist of multiple services. However, in the absence of microservices, each application would typically have just one service. It is possible to have the same name for both the application and service if desired.
     • Language: The programming language used for developing the application service: Java, NodeJS, .NET, GoLang.
     • Application Name: Name of the target application.
     • Service Name: Service name in the application. Typically in monolithic applications, the service name and application name are the same. In the microservices application, there would be one application that can have one or more services.
       

3. Download the client instrumentation package for the selected language from the agent configuration wizard.

4. Follow the instructions provided in the client instrumentation package.

Following these steps, you have successfully enabled Traces O11ySource and configured the data sources for real-time application performance monitoring.

Configuring Sampling Policies

Sampling configuration allows the system to selectively decide on which traces to store. This helps in reducing the amount of data kept in the system.

There are two types of sampling policies that can be configured for traces:

• Global Policies: Apply to all types of traces coming into the system.
• Service-Specific Policies: Apply to a specific set of services.

A separate page for sampling-related configurations is introduced. These configurations are controlled from the Settings tab on the Traces O11ySource page. The Settings tab includes three sections:

1. Sampling Policy
   Defines policies for deciding which traces to store. Policies can use inline rules or refer to rules defined in the Sampling Rules section.
2. Sampling Rule
   Provides a way to define individual rules that can be referred to in Sampling Policies to form compound rules.
3. Additional Configuration
   Controls data enrichments and transformations on received traces using optional YAML configurations.

Sampling Policy

Sampling Policies configured here allow vuSmartMaps to intelligently decide on which traces to store, helping to reduce the amount of data kept in the system by selecting only important traces. This is an optional functionality. If no sampling policies are configured, vuSmartMaps will store 100% of the traces collected.

Sampling policies can use rules defined inline within them or refer to rules defined in the Sampling Rules section. Using rules from sampling rules in policies enables users to define compound policies that can refer to more than one rule. When multiple sampling policies are configured, vuSmartMaps will apply all the policies to all traces received and will take a final decision based on the following:

• The trace is dropped when there is one explicit ‘Do not sample’ decision based on an inverted match.
• The trace is stored when there is one ‘Sample’ decision.
• The trace is stored when there is one ‘Sample’ decision based on an inverted match and there are no ‘Do not sample’ decisions.
• The trace is dropped in all other cases.

To configure a new sampling policy, use the + button and provide the following details in the modal:

• Policy Name: Unique name assigned by the user.
• Scope: Specify whether the policy applies to all traces (Global) or to a specific set of services. Options include All, For These Services, For Services Except These.
• Definition Type: Choose to define the policy details inline or refer to a rule defined in the Sampling Rules Section.

• • Inline: If this option is selected, the configured policy type becomes applicable:

• • • Policy Type: Specify the type of sampling policy. vuSmartMaps offers following seven policy types:

1. 1. 1. 1. Allow All: Marks all traces received to be stored.
         2. Data Rate: Tracks spans per second rate of traces and drops traces beyond the configured threshold.

1. 1. 1. 1. • Spans per Second: Maximum number of spans that can be processed each second.

1. 1. 1. 3. Latency: Stores only those traces with latency greater than the configured threshold.

• • • • • Threshold in Milliseconds: Only traces with latency greater than the given threshold (in ms) will be allowed.

1. 1. 1. 4. Number Attribute: Stores only those traces with the value of the configured attribute falling within a specified range.

1. 1. 1. 1. • Attribute: Select the attribute key to match data received from applications (e.g., HTTP Status Code).
            • Minimum Value: Enter the minimum acceptable value (e.g., 400 for HTTP Status codes).
            • Maximum Value: Enter the maximum acceptable value (e.g., 500 for HTTP Status codes).

1. 1. 1. 5. Percentage: Stores only a specified percentage of traces received, dropping the rest.

1. 1. 1. 1. • Sampling Percentage: The percentage rate at which traces are sampled.

1. 1. 1. 6. Status: Stores traces with specified statuses, dropping the rest.

1. 1. 1. 1. • Status Type: Status value for which traces will be retained. Options: Error, OK, Unset.

1. 1. 1. 7. String Attribute: Stores only those traces with the value of a configured attribute matching the configured condition.

1. 1. 1. 1. • Attribute: Select the attribute key to match data received from applications (e.g., Application, Service Name, Host Name, HTTP Host, HTTP Target, HTTP Route, HTTP URL, DB Name, DB System, Messaging System).
            • Values: Enter the values to match against (e.g., health, metrics, etc.).
            • Regular Expression: When ‘True’, the system will match the attribute values by regular expression string.
            • Invert Match: When ‘True’, the system will reverse the match condition.

• • • Use Predefined Rule Sets: If this option is selected, choose one of the following policy types and select the relevant rules from the Sampling Rules section:

1. 1. 1. 1. AND of Multiple Rules: Build a compound rule with an AND of multiple rules defined in the Sampling Rules section.
         2. Rate Distribution: Allocate portions of the total allowed rate to a set of rules from the Sampling Rules Section. Traces matching each rule will be allocated a portion of the total rate allowed.
         3. Single Rule: Use a single rule defined in the Sampling Rules section.

Sampling Rule

Sampling rules provide a way to define individual rules that can be referred to in Sampling Policies to form compound rules. The rules defined in Sampling Rules are not used unless referred to by a policy in Sampling Policies. Using rules from Sampling Rules in policies enables users to define compound policies that can refer to more than one rule.

To configure a new sampling rule, use the + button and provide the following details in the modal:

• Rule Name: Unique name assigned by the user.
• Rule Type: Specify the type of sampling rule. vuSmartMaps offers seven predefined rule types (as explained above for inline policy types).

Additional Configuration

The Additional Configuration section can be used to control data enrichments and transformations done on traces received. Users can add optional YAML configurations. Below is an example:

  transform:
    trace_statements:
      - context: span
        statements:
          - set(attributes["instrumentation_library_name"], instrumentation_scope.name)
          - set(attributes["library"], attributes["db.system"])
          - set(attributes["library"], attributes["messaging.system"])
          - set(attributes["library"], "Express") where instrumentation_scope.name == "@opentelemetry/instrumentation-express"
          - set(attributes["library"], "MongoDB") where instrumentation_scope.name == "io.opentelemetry.mongo-3.1"
          - set(attributes["library"], "Go-kit") where instrumentation_scope.name == "go.opentelemetry.io/contrib/instrumentation/github.com/go-kit/kit/otelkit"
          - set(attributes["library"], "RabbitMQ") where instrumentation_scope.name == "io.opentelemetry.rabbitmq-2.7"
          - set(attributes["library"], "NodeJS HTTP Client / Server") where instrumentation_scope.name == "@opentelemetry/instrumentation-http"
          - set(attributes["library"], "Redis") where instrumentation_scope.name == "@opentelemetry/instrumentation-redis"
          - set(attributes["library"], "Spring MVC") where instrumentation_scope.name == "io.opentelemetry.spring-webmvc-3.1"
          - set(attributes["library"], "Tomcat") where instrumentation_scope.name == "io.opentelemetry.tomcat-7.0"
          - set(attributes["library"], "Servlet") where instrumentation_scope.name == "io.opentelemetry.servlet-3.0"
          - set(attributes["library"], attributes["db.system"]) where instrumentation_scope.name == "io.opentelemetry.jdbc"
          - set(attributes["library"], "Spring Data") where instrumentation_scope.name == "io.opentelemetry.spring-data-1.8"
          - set(attributes["library"], "Hibernate") where instrumentation_scope.name == "io.opentelemetry.hibernate-4.0"
          - set(attributes["library"], "Go HTTP Client / Server") where instrumentation_scope.name == "go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp"
          - set(attributes["library"], "Spring Scheduling") where instrumentation_scope.name == "io.opentelemetry.spring-scheduling-3.1 1.14.0-alpha"
          - set(attributes["destination"], attributes["db.system"])
          - set(attributes["destination"], attributes["messaging.system"])
          - set(attributes["destination"], attributes["net.peer.name"])
          - set(attributes["destination"], replace_match(attributes["http.url"], "https?://[^/]+/([^/]+)", "$1"))
          - set(attributes["transaction"], Concat([attributes["http.method"], attributes["http.target"]], " ")) where (attributes["http.target"] != nil and attributes["http.method"] != nil and attributes["http.target"] != "")
          - set(attributes["transaction"], Concat([attributes["http.method"], attributes["http.route"]], " ")) where (attributes["http.route"] != nil and attributes["http.method"] != nil and attributes["http.route"] != "")
          - set(attributes["site"], "default")

This configuration enriches and transforms trace data, making it more meaningful and useful for analysis.

The Settings tab for Traces O11ySource provides a centralized location for configuring sampling policies, rules, and additional configurations. This setup simplifies the process, reduces user confusion, and enhances the ability to manage and control the data stored in the system effectively. By intelligently sampling traces, users can ensure that they are storing only the most relevant and important data, optimizing both storage and analysis capabilities.

Client Side Traces Instrumentation

Host / VM

1. Download the client instrumentation package containing vunet-opentelemetry-javaagent.jar and place it in your preferred directory.
   
   💡Note: It is to be noted that the downloaded instrumentation package needs to be completely unzipped since there is another zipped folder inside the package.
   
   
2. Launch the Java application with the following configuration:
   

   java -javaagent:path/to/vunet-opentelemetry-javaagent.jar -Dotel.resource.attributes=application=<APPLICATION_NAME> -Dotel.service.name=<SERVICE_NAME> -Dotel.metrics.exporter=none -Dotel.exporter.otlp.endpoint={{data.TRACES_INGESTION_ENDPOINT}} -jar app.jar

💡Note: It is to be noted that the above command needs to be run in a single-line command in one instance.

Ensure to update the placeholders:

1. APPLICATION_NAME: Change it to match your application name.
2. SERVICE_NAME: Change it to match the service name (microservices) or application layer (monolithic).
3. TRACES_INGESTION_ENDPOINT: Change it to match the URL of vuSmartMaps.

Example:

java -javaagent:vunet-opentelemetry-javaagent.jar -Dotel.resource.attributes=application=UPI -Dotel.service.name=upi-switch -Dotel.metrics.exporter=none -Dotel.exporter.otlp.endpoint=http://10.0.0.1:4317 -jar upi.jar

Docker

If your application is running in Docker, modify the Dockerfile:

COPY ./opentelemetry-javaagent.jar /usr/local/lib/opentelemetry-javaagent.jar
ENV JAVA_TOOL_OPTIONS "-javaagent:/usr/local/lib/opentelemetry-javaagent.jar"
ENV OTEL_RESOURCE_ATTRIBUTES "application=<Application Name>"
ENV OTEL_SERVICE_NAME "<Service Name>"
ENV OTEL_EXPORTER_OTLP_ENDPOINT "{{data.TRACES_INGESTION_ENDPOINT}}"
ENV OTEL_METRICS_EXPORTER "none"
ENTRYPOINT ["java","-Djava.security.egd=file:/dev/urandom","-jar","./app.jar", "--port=80"]

Ensure to update the placeholders:

• APPLICATION_NAME: Change it to match your application name.
• SERVICE_NAME: Change it to match the service name (microservices) or application layer (monolithic).
• TRACES_INGESTION_ENDPOINT: Change it to match the URL of vuSmartMaps.

Kubernetes

1. Add the Java auto-instrumentation agent to your Java application/service.
   • Download the client instrumentation package and copy the agent jar to the “agent” directory in your Java application source code.
   • Modify the Dockerfile to add the following line:
     

     COPY ./agent/opentelemetry-javaagent.jar ./opentelemetry-javaagent.jar

     

   • Build the Docker image.
2. Enable Java auto-instrumentation in the Kubernetes manifest:
   • Add the following environment variables:
     

     env:
             - name: JAVA_TOOL_OPTIONS
               value: "-javaagent:/usr/local/lib/vunet-opentelemetry-javaagent-1.18.0.jar -Dotel.instrumentation.runtime-metrics.enabled=true"
             - name: HOST_IP
               valueFrom:
                 fieldRef:
                    fieldPath: status.hostIP
             - name: POD_NAME
               valueFrom:
                 fieldRef:
                   fieldPath: metadata.name
             - name: POD_UID
               valueFrom:
                 fieldRef:
                   fieldPath: metadata.uid
             - name: OTEL_RESOURCE_ATTRIBUTES
               value: "service.name=<SERVICE_NAME>,service.version=<SERVICE_VERSION>,application=<APPLICATION_NAME>,service.instance.id=$(POD_NAME),k8s.pod.uid=$(POD_UID)"
             - name: OTEL_TRACES_EXPORTER
               value: otlp
             - name: OTEL_METRICS_EXPORTER
               value: otlp
             - name: OTEL_LOGS_EXPORTER
               value: none
             - name: OTEL_EXPORTER_OTLP_ENDPOINT
               value: {{data.TRACES_INGESTION_ENDPOINT}}
             - name: OTEL_LOG_LEVEL
               value: debug
             - name: OTEL_METRIC_EXPORT_INTERVAL
               value: "60000"

     

   • Ensure to update the placeholders:
     1. APPLICATION_NAME: Change it to match your application name.
     2. SERVICE_NAME: Change it to match the service name (microservices) or application layer (monolithic).
     3. TRACES_INGESTION_ENDPOINT: Change it to match the URL of vuSmartMaps.
   • Deploy the pod.

Following these steps, you have configured the application to be observed with vuApp360.