Logbeat 8.1.3 – Windows Readme

Introduction

Logbeat is a lightweight shipper for forwarding and centralizing log data. Installed as an agent on servers, Logbeat monitors the log files or locations that you specify, collects log events, and forwards them to the configured destination (Shipper). 

Pre-requisites

PowerShell with admin access.

Supported OS

• Windows (64-bit) – 8.1, 10, 11
• Windows Server – 2016, 2019, 2022

Installation

1. Extract the contents of the vuLogbeat_813_windows_x86_64.zip file into any folder of your choice.
2. Open a PowerShell prompt as an Administrator (right-click the PowerShell icon and select Run As Administrator).
3. From the PowerShell prompt, go to the extracted vuLogbeat_813_windows_x86_64 folder and run the following commands to install Logbeat as a Windows service.
   

   .\install-service-logbeat.ps1

   
   Note:
   If you get a script execution error while running the installer, rerun using an unrestricted execution policy:
   

   PowerShell.exe -ExecutionPolicy UnRestricted -File .\install-service-logbeat.ps1

   (OR)

   Set-ExecutionPolicy -Scope Process -ExecutionPolicy Bypass .\install-service-logbeat.ps1

4. Select an installation option:
   
   1. Fresh installation : To install the Logbeat service for the first time
   2. Upgrade an existing installation : To upgrade the agent binaries
   3. Exit : To exit the installation wizards
5. Enter the directory where you want to install the Logbeat agent. The default value is the (User Home).
   Press enter to accept the default. Else, if you want to specify the installation directory explicitly, please provide the path.
   Example,

   C:\Users\USER\ – The installation script will create Logbeat directory inside C:\Users\USER\

   C:\Users\USER\agents – The installation script will create Logbeat directory inside C:\Users\USER\agents

   If the directory exists, press enter or y to continue installation on the same directory, or else you can exit wizards by pressing n.

Configuration

How to Configure the Agent manually?

1. The configuration files controlling the functioning of the agent are

   (AGENT_HOME)\logbeat.yml

   and

   (AGENT_HOME)\inputs.d\*.yml

2. The (AGENT_HOME)\logbeat.yml file mainly contains the modules directory path from where the individual module configuration needs to be fetched and the configuration for the output method to send the collected data.
3. The logstash output can be enabled by uncommenting the “output.logstash” setting.
   Following are the configurations in the logstash output block.

   #----------------------------- Logstash output --------------------------------

   output.logstash:

   # The Logstash hosts

   hosts: [“127.0.0.1:5050”]

   # Optional SSL. By default it is off.

   # List of root certificates for HTTPS server verifications

   #ssl.certificate_authorities: [“/etc/pki/root/ca.pem”]

   # Certificate for SSL client authentication

   #ssl.certificate: “/etc/pki/client/cert.pem”

   # Client Certificate Key

   #ssl.key: “/etc/pki/client/cert.key”

4. The kafka output can be enabled by uncommenting the "output.kafka" setting. Following are the configurations in the kafka output block.

   #—————————– Kafka output ——————————–

   output.kafka:

   #initial brokers for reading cluster metadata

   hosts: [“127.0.0.1:9092”]

   ###message topic selection + partitioning

   topic: ‘logbeat-server’

   required_acks: 1

   compression: gzip

   max_message_bytes: 100000

   channel_buffer_size: 100000

5. The logging block is towards the end of the configuration where log level, log path and rotation policy can be specified.
   #================================ Logging =====================================
   # Set log level. The default log level is info.

   # Available log levels are: critical, error, warning, info, debug

   logging.level: debug

   # At debug level, you can selectively enable logging only for some components.

   # To enable all selectors use [“*”]. Examples of other selectors are “beat”,

   # “publish”, “service”.

   #logging.selectors: [“*”]

   logging.to_files: true

   logging.files:

   path: C:\Users\USER\logbeat\logs

   name: logbeat

   keepfiles: 5

   permissions: 0644

6. You can configure the inputs for Logbeat manually by editing the (AGENT_HOME)\inputs.d\log-config.yml or run the configure_logs.ps1 under (AGENT HOME):

   (AGENT_HOME)\configure_logs.ps1

   Note: If you get a script execution error while running the installer, rerun using an unrestricted execution policy:

   PowerShell.exe -ExecutionPolicy UnRestricted -File .\configure_logs.ps1

Sample Configuration

- type: log

Encryption of Agent Communication

vuSmartMaps uses 9094 for TLS and 9092 for plaintext communication from agent to broker.

To use encrypted communication with Kafka broker, the following steps should be performed.

For one-way SSL encryption, i.e. server authentication by the client, only one property needs to be added to the outputs.kafka section, i.e. certificate_authorities. This refers to the Root CA of the certificate used by the broker. In default vuSmartMaps installation, the broker is set up with a certificate signed by custom Root CA and this CA certificate is pre-packaged with the agent truststore. However, in case of a requirement to use your organization certificates, they have to be used during the install time of kafka broker while deploying vuSmartMaps and should also be added to agent truststore manually post agent installation.

ssl.certificate_authorities:["(path-to-client-truststore (.pem))"]

The certificate and key properties are required for client authentication. If client authentication is not required, no need to specify these properties.

ssl.certificate: "(path-to-client-certificate (.pem))"
ssl.key: "(path-to-client-key (.key))"

#If implementing only 1-way SSL
ssl.verification_mode: "none"

• certificate_authorities – The list of root(CA) certificates for server verifications. If certificate_authorities is empty or not set, the trusted certificate authorities of the host system are used. Simple SSL encryption requires only 1-way authentication, i.e., server authentication. This is done by the CA which has signed and trusted the server’s public key (certificate).
• certificate – The path to the certificate for SSL client authentication. If the certificate is not specified, client authentication is not available.
• key – The client certificate key used for client authentication. This option is required if a certificate is specified.

The server may be running a valid CA signed certificate or a self-signed certificate depending on the installation. In case CA signed certificate being used by the server, the client trust store must have the CA root certificate of the signing authority on its trust store. If a self-signed certificate is used by the server, the server certificate itself should be added to the trust store of the client.

Please note the agent uses PEM format for certificate_authorities.

Managing Logbeat Service

How to start, stop and get the status of the Logbeat service

1. To start the Logbeat service, use the command below

   net start logbeat

2. To stop the Logbeat service, use the command below

   net stop logbeat

3. To get the status of the Logbeat service, use the command below

   get-service logbeat

Logging

How to Run for Debugging?

(AGENT_HOME)\logbeat.exe -c "(AGENT_HOME)\logbeat.yml" -e -d "*"

Uninstall

To uninstall the Logbeat service, run uninstall-service-logbeat.ps1 script located at (AGENT_HOME)\logbeat directory as follows:

.\uninstall-service-logbeat.ps1

Note: If you get a script execution error while running the uninstaller, rerun using an unrestricted execution policy:

PowerShell.exe -ExecutionPolicy UnRestricted -File  .\uninstall-service-logbeat.ps1

Uninstallation is now complete.