Nginx
Introduction
Nginx (pronounced engine-x) is a popular open-source web server, reverse proxy server, and load balancer. It is well-known for its performance, scalability, and versatility.
Getting Started
Compatibility
The Nginx O11ySource is compatible with all versions greater than 1.25.3
Data Collection Method
vuSmartMaps collects health and performance data from Nginx server using an internal data collector. This data collector collects data based on the source configuration. Nginx logs are collected by VuNet's Logbeat agent
Prerequisites
Inputs for Configuring Data Source
- Nginx Server: The IP Address/FQDN of the Nginx server. This field is the key to identify each server you add here.
- Period [in seconds]: How frequently data is gathered. The period should be between 60 seconds – 3000 seconds.
- URL: The server URL to Nginx server-status page. Eg: http://<ipaddress>, https://<ipaddress>,http://<ipaddress>:<port> or https://<ipaddress>:<port>
- Access Log Path: Enter the Nginx access logfile's entire path
- Error Log Path: Enter the Nginx error logfile's entire path.
Firewall Requirement
To collect data from this O11ySource, ensure the following ports are opened:
| Source IP | Destination IP | Destination Port | Protocol | Direction |
|---|---|---|---|---|
| vuSmartMaps Data End Points | IP Address of the Nginx Server | 8080 | TCP | Outbound |
| IP address of the Nginx server | vuSmartMaps Data End Points | 9092* | TCP | Inbound |
*Before providing the firewall requirements, please update the port based on the customer environment.
Configuring the Target
Configure Metrics Collection from Nginx Server
VuNet's internal data collectors gather Nginx performance metrics using the stub_status module.This module provides a status page that helps monitor the server’s performance.To verify if the stub_status module is enabled or not, use nginx -V 2\>&1 | grep -o "with-http_stub_status_module".If the output includes "with-http_stub_status_module", the module is already enabled, and you can skip to the next step.If the status module is not enabled, Locate your Nginx configuration file (default locations vary): /etc/nginx/nginx.conf or /etc/nginx/sites-available/default for most systems, A specific virtual host configuration file.Add the following block to enable stub_status and restrict access. If you are using a prebuilt Nginx package,this module is usually enabled by default. To confirm, check if the stub_status on; line is present in your Nginx configuration. To indicate which IP addresses should view the status page, you must alter the block that begins with location /server_status. In the example that follows, vusmartmaps Network IP (192.168.1.0/24) access is permitted.
location /server_status {
stub_status;
allow vusmartmaps Network IP;
deny all;
stub_status on;
server_tokens on;
}
Configure Logs Collection from Nginx Server Nginx logs provide detailed information about client requests.VuNet’s Logbeat agent collects these logs for analysis. To ensure logs are collected in a structured format, add the following log_format directive to your Nginx configuration:
log_format access_format 'time: $time_iso8601,' ' bytes_sent: $bytes_sent,' ' connection_time: $connection_time,' ' nginx_host: $host,' ' pid: $pid,' ' proxy_host: $proxy_host,' ' proxy_port: $proxy_port,' ' realip_remote_addr: $realip_remote_addr,' ' realip_remote_port: $realip_remote_port,' ' remote_addr: $remote_addr,' ' remote_port: $remote_port,' ' request: $request,' ' request_filename: $request_filename,' ' request_method: $request_method,' ' request_time: $request_time,' ' request_uri: $request_uri,' ' server_addr: $server_addr,' ' server_name: $server_name,' ' server_port: $server_port,' ' server_protocol: $server_protocol,' ' ssl_client_verify: $ssl_client_verify,' ' ssl_protocol: $ssl_protocol,' ' ssl_server_name: $ssl_server_name,' ' ssl_session_id: $ssl_session_id,' ' ssl_session_reused: $ssl_session_reused,' ' status: $status,' ' upstream_addr: $upstream_addr,' ' upstream_bytes_received: $upstream_bytes_received,' ' upstream_bytes_sent: $upstream_bytes_sent,' ' upstream_cache_status: $upstream_cache_status,' ' upstream_connect_time: $upstream_connect_time,' ' upstream_response_time: $upstream_response_time';
Configuration Steps
- Enable the O11ySource.
- Select the Sources tab and press the
+button to add a new Nginx host to be monitored. - Set up metrics and log collection configurations. Afterwards, select
Save and Continueto proceed with downloading the Logbeat agent. - The following packages will be available for download based on the OS:Download <OS> Logbeat Installation Binary - Downloads the full Logbeat package with required configurations for a fresh installationDownload <OS> Logbeat Update Configuration Binary - Downloads the agent configuration package to update an existing Logbeat installation
- Download the agent installation or update package, then click
Finishto close the data source window.
Metrics Collected
| Name | Description | Data Type |
|---|---|---|
| @timestamp | The original timestamp as a string, often representing the time the data was recorded or received. | String |
| timestamp | The precise timestamp of the event, with microsecond precision. | DateTime64 |
| host | The hostname or IP address of the machine where the event originated. | String |
| tenant_id | The unique identifier for the tenant (user or customer) associated with this event. | UInt64 |
| bu_id | The unique identifier for the business unit associated with this event. | UInt64 |
| server | The name or identifier of the server handling the requests. | String |
| src_host | The source host from which the request originated. | String |
| src_instance | The source instance or specific instance of the application sending the request. | String |
| name | The name associated with this particular set of metrics, often the service or application name. | String |
| port | The port number on which the server is listening for incoming requests. | UInt64 |
| requests | The total number of requests received by the server or application. | UInt64 |
| accepts | The number of accepted connections. | UInt64 |
| handled | The number of successfully handled connections or requests. | UInt64 |
| reading | The number of connections where the server is currently reading the request. | UInt64 |
| writing | The number of connections where the server is currently writing a response. | UInt64 |
| waiting | The number of connections that are currently idle, waiting for a request. | UInt64 |
| active | The number of active connections at the time of measurement. | Float64 |
| dropped | The number of connections that were dropped or refused. | Float64 |
| requests_diff | The difference in the number of requests since the last measurement period. | Float64 |
| accepts_diff | The difference in the number of accepted connections since the last measurement period. | Float64 |
| handled_diff | The difference in the number of handled connections since the last measurement period. | Float64 |
| dropped_diff | The difference in the number of dropped connections since the last measurement period. | Float64 |
| requests_rate | The rate of requests per second. | Float64 |
| period | The duration of the measurement period, typically in seconds. | Float64 |
| accepts_rate | The rate of accepted connections per second. | Float64 |
| handled_rate | The rate of handled connections per second. | Float64 |
| dropped_rate | The rate of dropped connections per second. | Float64 |
| telegraf_unique_id | A unique identifier for the metrics record, often generated by the Telegraf agent collecting the data. | String |
| nginx_host | The hostname or IP address of the NGINX server that is handling the requests. | String |
| timestamp | The precise timestamp of the event, with microsecond precision. | DateTime64 |
| message | The log message content. | String |
| beat | The name or identifier of the Beat (e.g., Filebeat, Metricbeat) that collected the log. | String |
| bu_id | The unique identifier for the business unit associated with this event. | UInt64 |
| bytes_sent | The total number of bytes sent in the request/response. | UInt64 |
| server_addr | The IP address of the server that processed the request. | String |
| target | The target endpoint or resource being requested. | String |
| beat_hostname | The hostname of the machine where the Beat is running. | String |
| connections_active | The number of active connections at the time of the log entry. | UInt64 |
| connections_reading | The number of connections currently reading requests. | UInt64 |
| connection_requests | The number of connection requests. | UInt64 |
| connections_waiting | The number of connections currently idle, waiting for a request. | UInt64 |
| connections_writing | The number of connections currently writing a response. | UInt64 |
| content_length | The length of the content in the request/response. | UInt64 |
| host | The hostname of the server processing the request. | String |
| log_file_path | The file path of the log file where this entry is recorded. | String |
| source_id | A unique identifier for the source of the log entry. | String |
| topic_name | The name of the topic, if logs are being collected and organized by topic (e.g., in Kafka). | String |
| http_response | The HTTP response code returned by the server. | String |
| limit_rate | The rate limit applied to the request, if any. | UInt64 |
| msec | The time in milliseconds at which the event occurred. | Float64 |
| nginx_host | The hostname of the NGINX server that processed the request. | String |
| nginx_port | The port on which the NGINX server is listening. | UInt64 |
| pid | The process ID of the NGINX process handling the request. | String |
| proxy_host | The hostname of the proxy server, if the request passed through one. | String |
| proxy_port | The port of the proxy server. | String |
| realip_remote_port | The real IP address's remote port from where the request originated. | Float64 |
| remote_addr | The IP address of the client making the request. | String |
| realip_remote_addr | The actual remote IP address, especially useful if behind a proxy. | String |
| remote_port | The remote port from which the request was made. | String |
| client_ip | The IP address of the client, potentially different from remote_addr if the request passed through a proxy. | String |
| connection | The connection details or ID associated with the request. | String |
| error_message | The error message, if any occurred during processing. | String |
| log_level | The severity level of the log (e.g., INFO, ERROR, WARN). | String |
| upstream_url | The URL of the upstream server or service that handled the request. | String |
| tid | The thread ID of the process handling the request. | String |
| request | The full HTTP request line sent by the client. | String |
| request_method | The HTTP method used in the request (e.g., GET, POST). | String |
| request_uri | The URI or path requested by the client. | String |
| request_length | The length of the request in bytes. | UInt64 |
| request_time | The time taken to process the request, in seconds. | Float64 |
| request_filename | The name of the file or resource requested, if applicable. | String |
| server_name | The name of the server handling the request. | String |
| server_port | The port number on which the server is listening. | String |
| server_protocol | The protocol used by the server (e.g., HTTP/1.1, HTTP/2). | String |
| ssl_session_id | The SSL session ID, if the request used SSL/TLS. | String |
| ssl_server_name | The server name indicated by the client during SSL handshake. | String |
| ssl_protocol | The SSL/TLS protocol used for the connection. | String |
| o11ysource_name | The name of the observability source, if part of a broader observability platform. | String |
| ssl_client_verify | The result of SSL client certificate verification. | String |
| ssl_session_reused | Whether the SSL session was reused (e.g., YES or NO). | String |
| status | The HTTP status code returned by the server (e.g., 200, 404). | UInt64 |
| tenant_id | The unique identifier for the tenant (user or customer) associated with this event. | UInt64 |
| type | The type of event or log entry. | String |
| upstream_addr | The IP address of the upstream server that handled the request. | String |
| upstream_connect_time | The time taken to establish a connection to the upstream server. | Float64 |
| upstream_response_time | The time taken by the upstream server to respond. | Float64 |
| upstream_bytes_sent | The number of bytes sent to the upstream server. | UInt64 |
| upstream_bytes_received | The number of bytes received from the upstream server. | UInt64 |
| upstream_header_time | The time taken to receive the response headers from the upstream server. | Float64 |
| upstream_response_length | The length of the response received from the upstream server. | UInt64 |
| upstream_cache_status | The cache status of the response from the upstream server (e.g., HIT, MISS). | String |
