vuLogAgent 1.1.0 – AIX and Solaris Readme

Introduction

vuLogAgent is a log data shipper used on systems running Solaris and AIX. Installed as an agent on your servers, vuLogAgent monitors the log directories or specific log files, tails the files, and forwards them to vuSmartMaps for further analysis.

Pre-requisites

1. vuLogAgent needs java version 1.6 or higher. Ensure that a supported java version is installed on the system and is available to the user $PATH.
2. A sudo user with elevated privilege or root user access is required to create startup scripts during installation.
3. The agent does not need any superuser privileges for running.
4. Firewalls, if any, should allow the outgoing TCP port of the vuSmartMaps Data collector/shipper from the node on which this agent is installed.

Firewall Requirements

1. The node on which the agent is installed should be allowed to connect to the vuSmartMaps data collector/shipper IP and TCP port configured for the agent.
2. The default ports on which the data is sent are
   
   1. logstash –  5043
   2. Kafka – 9092

Supported OS

• AIX – 7.1, 7.2, 7.3
• Solaris – 11.2, 11.3, 11.4

Installation

1. Extract the package and run the install script.
   
   tar -xvf vulogagent-setup.tar
   ./install
2. Choose the installation option:
   
   1. Select “1” to install the agent for the first time
   2. Select “2” to upgrade the agent. The upgrade option will only update the binaries and leave the configurations intact.
3. Enter to accept the default directory. If you want the agent to be installed elsewhere, please specify the directory path.Example,
   
   /home/$USER          – The installation will create vuLogAgent directory inside /home/$USER/
   /home/$USER/agents  – The agents directory should exist in the system.
4. The installation will ask to provide superuser / sudo access to create init.d/systemd services for the startup on system reboot.Enter [y] to accept this and provide the superuser / sudo username and password to proceed with.If you don’t have superuser / sudo access enter ‘n’ and skip the above steps. But, you have to run these steps manually with sudo access.

Enabling services at startup manually

Login with sudo user or root user and run the below commands based on your os.

Solaris:

AIX:

Note: (VULOGAGENT_HOME) is the directory where vuLogAgent is installed.

Configuration

1. After the installation, the script will ask for the configuration of the installed agent. press y to configure the agent on the go or press n to skip the configuration and you have to manually configure it by updating (VULOGAGENT_HOME)/conf.d/vulogagent.json. Refer to the sample configurations available in the path.
2. The configuration will update the following attributes:
   
   1. Shipper/Target IP  –  The remote vuSmartMaps system where the data is to be sent
   2. PROTOCOL  – The protocol to be used to communicate to the remote machine.We support below protocols:
      
      1. Kafka
      2. Logstash
   3. PORT –  The port on which the agent has to be connected on the remote machine.
   4. TIMEOUT  –  To set the max time to wait for the connection in milliseconds
   5. LOGPATH   –  The path to log, which you want to ship to vuSmartMaps
   6. LOGTYPE  –  The type of log, which will be used to identify the logs
   7. MULTILINE PATTERN  – The ‘pattern’ setting is used to detect the beginning of a new multi-line message.When a message line matches one of these expressions, vuLogAgent waits until the next instance of this same expression,and then groups all lines between them as a single message. You can specify multiple patterns by providing ‘|’ in between the patternseg1: ^INFO eg2: ^INFO|^ERROR
   8. NEGATE & WHAT  –  The ‘negate‘ and ‘what‘ settings decide the strategy to combine the consecutive logs. For example, if negate is true and ‘what’ is set to ‘next’, consecutive lines that DOES NOT match the given pattern will be appended AFTER the line that DOES match the pattern. This is a typical case of a java exception log event that spans across multiple lines.
      
      1. negate: false, what: next, pattern: ^b
         Result: Consecutive lines that match the pattern are appended to the previous line that doesn’t match.
         Example:
          _
         a   |
         b   | => abb

         b  _|

         c   |

         b   | => cbb

         b  _|

      2. negate: false, what: before, pattern: ^b
         Result: Consecutive lines that match the pattern are prepended to the next line that doesn’t match.
         Example:
         _
         b   |
         b   | => bba

         a  _|

         b   |

         b   | => bbc

         c  _|

      3. negate: true, what: next, pattern: ^b
         Result: Consecutive lines that don’t match the pattern are appended to the previous line that does match.
         Example:
         _
         b   |
         a   | => bac

         c  _|

         b   |

         d   | => bde

         e  _|

      4. negate: true, what: before, pattern: ^b
         Result: Consecutive lines that don’t match the pattern are prepended to the next line that does match.
         Example:
         _
         b   |
         a   | => acb

         c  _|

         b   |

         d   | => deb

         e  _|

   9. DEAD TIME – The ‘dead time’ setting is to ignore the log files which are not modified within the specified time. Time can be specified as ‘m‘ for minutes or ‘h‘ for hours (ex: 30m / 1h)
   10. CLOSE TIMEOUT – The ‘closeTimeout‘ setting is to determine the maximum duration for which any file will be allowed to remain open in the vuLogAgent watchmap. Time can be specified as ‘m’ for minutes or ‘h’ for hours (ex: 30m / 1h)
   11. FILTER PATTERN & NEGATE – Your use case might require only a subset of the data exported by vuLogAgent. vuLogAgent allows you to specify different filtering criteria.You can configure each input to include or exclude specific lines.Examples of pattern and negate for filtering
       
       1. pattern: ^b negate : false
          Result: The lines that are starting with ^b will be exported by vulogagent
       2. pattern: ^b negate : true
          Result: The lines that are starting with ^b will be dropped by vulogagent
   12. TOPIC – The topic name where we have to send the data in case of kafka protocol

Note: A general assumption is agents are installed with the knowledge of the exact collection method and listening port setup at the vuSmartMaps end. If this information is not yet known, contact Vunet support.

Understand the  config.json

Common Options

1. network => This setting is to specify the target protocol, target ip and port.
   
   1. type    => the target protocol
   2. servers => the target ip and port.
   3. timeout  => the network timeout in milliseconds
2. files => This block is to specify the log paths, fields, and dead time.
   
   1. paths     => A list of paths to watch. You can give a single file or a wildcard. It also supports wildcard directories.
   2. fields    => A list of fields to be annotated on events from a paths entry
   3. dead time => If this option is enabled, vuLogAgent ignores any files that were modified before the specified timespan.
   4. close timeout => To determine the maximum duration for which any file will be allowed to remain open in the vuLogagent watchmap
3. filter  => vuLogAgent allows you to specify different filtering criteria, you can configure each input to include or exclude specific lines.pattern => Specifies the regular expression pattern to match a line.negate  => Specifies the action on a match. If negate is false then vuLogAgent includes the line or vuLogAgent will drop the line that matches the pattern.

Sample Single line log monitoring Configuration

Example 1:

Example 2:

Multiline Configuration Options

Multiline => This block enables to specify the multiline log-related settings.

pattern => Specifies the regular expression pattern to match. Depending on how you configure other multiline options, lines that match the specified regular expression are considered either continuation of a previous line or the start of a new multiline event. You can set the negate option to negate the pattern.

negate  => Defines whether the pattern is negated. The default is true.

what    => Specifies how vuLogAgent combines matching lines into an event. The settings are previous or next. The behavior of these settings depends on what you specify for negate.

Sample Multiline Log Monitoring Configurations

Logging

Logs are written to (VULOGAGENT_HOME)/log/vulogagent.log by default.

Managing vuLogAgent Service

Solaris:

AIX:

Know the Version

Uninstall

The vuLogAgent package comes with an uninstall script to uninstall.

To uninstall, simply go to the extracted setup directory and run

Uninstallation will then be complete.