Curlbeat 1.0.0 – Linux Readme

Introduction

Curlbeat or curl-heartbeat is an alternate light-weight shell-based implementation of URL heartbeat. It uses the curl command to get the availability status and various latency timers for one or more URLs. The URLs to be monitored are picked from a file (urls.config by default).

For the output, we support two methods/channels.

1. TCP
2. Kafka Topic

Pre-requisites

Java 1.6 or above is required if the output channel is Kafka.

Note: A general assumption is agents are installed with the knowledge of the exact collection method and listening port setup at the vuSmartMaps end. If this information is not yet known, contact VuNet support.

Supported OS

All Linux flavors with bash.

Installation

1. Extract the contents of the Curlbeat package tar into any folder of your choice.
2. Run the installation script from the extracted directory.Example:
   
   bash#  cd /home/vunet/curlbeat
   bash#  bash ./curl_installation.sh
3. You can choose the default directory or any directory of your choice to be used while installing the script. The default is user $HOME. Please note that the directory specified must exist.
   
   Example: If the specified directory is /home/vunet, the installation script will be in the /home/vunet/curlbeat subdirectory that will self-contain everything required to run Curlbeat.
4. Once the path is specified, the user will be prompted to configure the agent, the default is yes.

Configuration

1. The installer will optionally assist in configuring the agent. It is recommended to use the option for creating the config files automatically. Else, the configuration needs to be done manually using the configuration file.
2. The configuration will update the following attributes:

• • Interval – The frequency of data collection. Default is 5 min
  • Output Method – The method used to ship the collected metrics to the remote vuSmartMaps data collector. We support two options:
    • Kafka (default)

• • • TCP

• • Target/Shipper IP – The remote vuSmartMaps data collector/shipper where the collected metrics should be sent.

• • Port – The remote listening port on the vuSmartMaps data collector/shipper.

• • Topic – The Kafka topic name in case, the output method is chosen as Kafka.

• • Number of URLs – The number of URLs required to be checked for availability. (If the URL count is more than 5, we recommend you to directly edit the URL config file available in curlbeat/conf.d/ manually)

Parameters fetched by Curlbeat

1. Namelookup time:
   The time in seconds it took from the start until the name resolving was completed.
2. Connect time:
   The time in seconds it took from the start until the TCP connection to the remote host (or proxy) was completed.
3. Appconnect time:
   The time in seconds it took from the start until the SSL/SSH/etc connect/handshake to the remote host was completed.
4. Pretransfer time:
   The time in seconds it took from the start until the file transfer was just about to begin. This includes all pre-transfer commands and negotiations that are specific to the particular protocol(s) involved.
5. Redirect time:
   The time in seconds it took for all redirection steps including name lookup, connect, pretransfer, and transfer before the final transaction was started. time_redirect shows the complete execution time for multiple redirections.
6. Start Transfer time:
   The time in seconds it took from the start until the first byte was just about to be transferred. This includes time_pretransfer and also the time the server needed to calculate the result.
7. Total time:
   The total time in seconds that the full operation lasted. The time will be displayed with millisecond resolution.
8. Sharepoint request Duration:
   Applicable only for Sharepoint sites. This is fetched from the SPRequestDuration response header. This is the time in milliseconds that it took to process the request on the server. Fundamentally, this is the end-to-end processing time for the SharePoint page. Healthy pages have a value of around 200 ms, and a range of 100–500 ms, so if it’s greater than that, you may experience some issues.
9. Sharepoint IIS Latency:
   Applicable only for Sharepoint sites. This is fetched from the SPI Latency response header. This is the time in milliseconds taken by the front-end web server after the request has been received by that server, but before the web application begins to process the request. This value should be around 0 or 1, or very close to those figures.

How to Configure the Agent manually?

1. The configuration file host.config is available at the following location
   
   (INSTALL DIRECTORY)/curlbeat/conf.d
2. This host.config file contains
   • IP
   • PROTOCOL – TCP or Kafka
   • PORT
3. The urls.config file contains the list of URLs to be monitored.

Managing Curlbeat Service

1. To start the Curlbeat service, use the command below
   
   sudo service curlbeat start
2. To stop the Curlbeat service, use the command below
   
   sudo service curlbeat stop
3. To get the status of the Curlbeat service, use the command below
   
   sudo service curlbeat status

Uninstall

Users can directly uninstall it, by going to the installed location and removing the Curlbeat directory.

Example: Home directory: /home/vunet

The installation script will create Curlbeat directory inside the mentioned location at the time of installation /home/vunet/curlbeat/

From the home directory (/home/vunet) user can run the below command