logstash.md

logstash

Monitor Type: logstash (Source)

Accepts Endpoints: Yes

Multiple Instances Allowed: Yes

Overview

Monitors the health and performance of Logstash deployments through Logstash's Monitoring APIs.

Configuration

To activate this monitor in the Smart Agent, add the following to your agent config:

monitors:  # All monitor config goes under this key
 - type: logstash
   ...  # Additional config

For a list of monitor options that are common to all monitors, see Common Configuration.

Config option	Required	Type	Description
host	no	string	The hostname of Logstash monitoring API (default: 127.0.0.1)
port	no	integer	The port number of Logstash monitoring API (default: 9600)
useHTTPS	no	bool	If true, the agent will connect to the host using HTTPS instead of plain HTTP. (default: false)
timeoutSeconds	no	integer	The maximum amount of time to wait for API requests (default: 5)

Metrics

These are the metrics available for this monitor. Metrics that are categorized as container/host (default) are in bold and italics in the list below.

Group events

All of the following metrics are part of the events metric group. All of the non-default metrics below can be turned on by adding events to the monitor config option extraGroups:

• node.stats.events.events.duration_in_millis (cumulative)
  Duration of events
• node.stats.events.events.filtered (cumulative)
  Number of filtered events
• node.stats.events.events.in (cumulative)
  Number of received events
• node.stats.events.events.out (cumulative)
  Number of events sent out
• node.stats.events.events.queue_push_duration_in_millis (cumulative)
  Duration that events waited in queue before being processed

Group hot_threads

All of the following metrics are part of the hot_threads metric group. All of the non-default metrics below can be turned on by adding hot_threads to the monitor config option extraGroups:

• node.hot_threads.hot_threads.busiest_threads (gauge)
  
• node.hot_threads.hot_threads.threads.percent_of_cpu_time (gauge)
  

Group jvm

All of the following metrics are part of the jvm metric group. All of the non-default metrics below can be turned on by adding jvm to the monitor config option extraGroups:

• node.stats.jvm.jvm.gc.collectors.old.collection_count (cumulative)
  Total number of garbage collection events
• node.stats.jvm.jvm.gc.collectors.old.collection_time_in_millis (cumulative)
  Amount of time spent garbage collecting in milliseconds
• node.stats.jvm.jvm.gc.collectors.young.collection_count (cumulative)
  Total number of garbage collection events
• node.stats.jvm.jvm.gc.collectors.young.collection_time_in_millis (cumulative)
  Amount of time spent garbage collecting in milliseconds
• node.stats.jvm.jvm.mem.heap_committed_in_bytes (gauge)
  Total heap committed by the process
• node.stats.jvm.jvm.mem.heap_max_in_bytes (gauge)
  Max memory being used
• node.stats.jvm.jvm.mem.heap_used_in_bytes (gauge)
  Total heap used
• node.stats.jvm.jvm.mem.heap_used_percent (gauge)
  Total heap used in percentage
• node.stats.jvm.jvm.mem.non_heap_committed_in_bytes (gauge)
  Total non-heap memory committed by the process
• node.stats.jvm.jvm.mem.non_heap_used_in_bytes (gauge)
  Total non-heap memory used
• node.stats.jvm.jvm.mem.pools.old.committed_in_bytes (gauge)
  Memory guaranteed to be available to JVM non-heap by Old gen
• node.stats.jvm.jvm.mem.pools.old.max_in_bytes (gauge)
  Max memory being used by Old Gen
• node.stats.jvm.jvm.mem.pools.old.peak_max_in_bytes (gauge)
  Memory used by Old gen
• node.stats.jvm.jvm.mem.pools.old.peak_used_in_bytes (gauge)
  Peak memory used by Old gen
• node.stats.jvm.jvm.mem.pools.old.used_in_bytes (gauge)
  Memory being used by Old Gen
• node.stats.jvm.jvm.mem.pools.survivor.committed_in_bytes (gauge)
  Memory guaranteed to be available to JVM non-heap by Survivor space
• node.stats.jvm.jvm.mem.pools.survivor.max_in_bytes (gauge)
  Max memory being used by Survivor space
• node.stats.jvm.jvm.mem.pools.survivor.peak_max_in_bytes (gauge)
  Memory used by Survivor space
• node.stats.jvm.jvm.mem.pools.survivor.peak_used_in_bytes (gauge)
  Peak memory used by Survivor space
• node.stats.jvm.jvm.mem.pools.survivor.used_in_bytes (gauge)
  Memory being used by Survivor space
• node.stats.jvm.jvm.mem.pools.young.committed_in_bytes (gauge)
  Memory guaranteed to be available to JVM non-heap by Young gen
• node.stats.jvm.jvm.mem.pools.young.max_in_bytes (gauge)
  Max memory being used by Young Gen
• node.stats.jvm.jvm.mem.pools.young.peak_max_in_bytes (gauge)
  Memory used by Young gen
• node.stats.jvm.jvm.mem.pools.young.peak_used_in_bytes (gauge)
  Peak memory used by Young gen
• node.stats.jvm.jvm.mem.pools.young.used_in_bytes (gauge)
  Memory being used by Young Gen
• node.stats.jvm.jvm.threads.count (gauge)
  Number of JVM threads
• node.stats.jvm.jvm.threads.peak_count (gauge)
  Highest number of JVM threads
• node.stats.jvm.jvm.uptime_in_millis (gauge)
  Uptime length of JVM

Group os

All of the following metrics are part of the os metric group. All of the non-default metrics below can be turned on by adding os to the monitor config option extraGroups:

• node.os.os.available_processors (gauge)
  Number of available processors
• node.stats.os.os.cgroup.cpu.cfs_period_micros (gauge)
  
• node.stats.os.os.cgroup.cpu.cfs_quota_micros (gauge)
  
• node.stats.os.os.cgroup.cpu.stat.number_of_elapsed_periods (cumulative)
  
• node.stats.os.os.cgroup.cpu.stat.number_of_times_throttled (cumulative)
  
• node.stats.os.os.cgroup.cpu.stat.time_throttled_nanos (cumulative)
  
• node.stats.os.os.cgroup.cpuacct.usage_nanos (gauge)
  

Group pipeline

All of the following metrics are part of the pipeline metric group. All of the non-default metrics below can be turned on by adding pipeline to the monitor config option extraGroups:

• node.pipelines.batch_delay (gauge)
  
• node.pipelines.batch_size (gauge)
  
• node.pipelines.workers (gauge)
  Number of workers in pipelines
• node.stats.pipelines.events.duration_in_millis (cumulative)
  Duration of events in pipelines
• node.stats.pipelines.events.filtered (cumulative)
  Number of filtered events in pipelines
• node.stats.pipelines.events.in (cumulative)
  Number of received events in pipelines
• node.stats.pipelines.events.out (cumulative)
  Number of events sent out from pipelines
• node.stats.pipelines.events.queue_push_duration_in_millis (cumulative)
  Duration that events waited in queue before being processed in pipelines
• node.stats.pipelines.plugins.codecs.decode.duration_in_millis (cumulative)
  Duration of decode events in codec plugins
• node.stats.pipelines.plugins.codecs.decode.out (cumulative)
  Number of decode events sent out from codecs
• node.stats.pipelines.plugins.codecs.decode.writes_in (cumulative)
  Number of received decode events in codecs
• node.stats.pipelines.plugins.codecs.encode.duration_in_millis (cumulative)
  Duration of encode events in codec plugins
• node.stats.pipelines.plugins.codecs.encode.writes_in (cumulative)
  Number of received encode events in codecs
• node.stats.pipelines.plugins.filters.events.duration_in_millis (cumulative)
  Duration of events in filter plugins
• node.stats.pipelines.plugins.filters.events.in (cumulative)
  Number of received events in filters
• node.stats.pipelines.plugins.filters.events.out (cumulative)
  Number of events sent out from filters
• node.stats.pipelines.plugins.inputs.events.out (cumulative)
  Number of events sent out from inputs
• node.stats.pipelines.plugins.inputs.events.queue_push_duration_in_millis (cumulative)
  Duration that events waited in queue before being processed in inputs
• node.stats.pipelines.plugins.outputs.events.duration_in_millis (cumulative)
  Duration of events in output plugins
• node.stats.pipelines.plugins.outputs.events.in (cumulative)
  Number of received events in output plugins
• node.stats.pipelines.plugins.outputs.events.out (cumulative)
  Number of events sent out from output plugins
• node.stats.pipelines.queue.events_count (gauge)
  Number of events waiting in queue
• node.stats.pipelines.queue.max_queue_size_in_bytes (gauge)
  Max queue size in pipelines
• node.stats.pipelines.queue.queue_size_in_bytes (gauge)
  Queue size in pipelines
• node.stats.pipelines.reloads.failures (cumulative)
  Number of failed reloads
• node.stats.pipelines.reloads.successes (cumulative)
  Number of successful reloads

Group pipelines

All of the following metrics are part of the pipelines metric group. All of the non-default metrics below can be turned on by adding pipelines to the monitor config option extraGroups:

Group plugins

All of the following metrics are part of the plugins metric group. All of the non-default metrics below can be turned on by adding plugins to the monitor config option extraGroups:

• node.plugins.total (gauge)
  Number of plugins

Group process

All of the following metrics are part of the process metric group. All of the non-default metrics below can be turned on by adding process to the monitor config option extraGroups:

• node.stats.process.process.cpu.load_average.15m (gauge)
  CPU Load average in 15 minutes
• node.stats.process.process.cpu.load_average.1m (gauge)
  CPU Load average in 1 minute
• node.stats.process.process.cpu.load_average.5m (gauge)
  CPU Load average in 5 minutes
• node.stats.process.process.cpu.percent (gauge)
  CPU usage in percent
• node.stats.process.process.cpu.total_in_millis (cumulative)
  Total CPU time (in milliseconds) used by the process on which the process is running
• node.stats.process.process.max_file_descriptors (gauge)
  Number of opened file descriptors associated with the current process
• node.stats.process.process.mem.total_virtual_in_bytes (gauge)
  Size of the virtual memory of this process
• node.stats.process.process.open_file_descriptors (gauge)
  Number of currently open file descriptors
• node.stats.process.process.peak_open_file_descriptors (gauge)
  Peak number of currently open file descriptors

Group reloads

All of the following metrics are part of the reloads metric group. All of the non-default metrics below can be turned on by adding reloads to the monitor config option extraGroups:

• node.stats.reloads.reloads.failures (cumulative)
  Number of failed reloads
• node.stats.reloads.reloads.successes (cumulative)
  Number of successful reloads

Non-default metrics (version 4.7.0+)

To emit metrics that are not default, you can add those metrics in the generic monitor-level extraMetrics config option. Metrics that are derived from specific configuration options that do not appear in the above list of metrics do not need to be added to extraMetrics.

To see a list of metrics that will be emitted you can run agent-status monitors after configuring this monitor in a running agent instance.