[[load-balancing]]
== Load Balancing

Filebeat provides configuration options that you can use to fine
tune load balancing when sending events to multiple hosts.

To enable load balancing, you specify `loadbalance: true` when you configure
the output. For example:

[source,yaml]
-------------------------------------------------------------------------------
output.logstash:
  hosts: ["localhost:5044", "localhost:5045"]
  loadbalance: true
-------------------------------------------------------------------------------

The `loadbalance` option is available for Redis, Logstash, and Elasticsearch
outputs. The Kafka output handles load balancing internally.

Filebeat can send events in a few different modes:

* **Send events to one host after another:**
+
By default, when you configure Filebeat to send events to multiple hosts
(`loadbalance: true`), Filebeat will send the events to one host after
another. This mode requires the least memory and CPU usage. This is not
true load balancing in the sense that Filebeat doesn't account for the load
being processed by each host.
+
The load balancer also supports multiple workers per host. The default is
`worker: 1`. If you increase the number of workers, additional network
connections will be used.  The total number of workers participating
in load balancing is `number of hosts * workers`.
+
Example:
+
[source,yaml]
-------------------------------------------------------------------------------
filebeat.prospectors:
- input_type: log
  paths:
    - /var/log/*.log
output.logstash:
  hosts: ["localhost:5044", "localhost:5045"]
  loadbalance: true
  worker: 2
-------------------------------------------------------------------------------
+
In this example, there are 4 workers participating in load balancing.

* **Send events to `N` hosts in lock-step:**
+
You can configure Filebeat to send events to `N` hosts in lock-step by setting
`spool_size = N * bulk_max_size`. In lock-step mode, the batch collected by the
spooler is split up into smaller batches of size `bulk_max_size`. These smaller
batches are load balanced between available connections. Filebeat waits for all
sub-batches to be published before it retrieves another batch from the spooler.
+
This mode requires more memory and CPU usage than the previous mode.
+
Example:
+
[source,yaml]
-------------------------------------------------------------------------------
filebeat.prospectors:
- input_type: log
  paths:
    - /var/log/*.log
filebeat.spool_size: 4096
output.logstash:
  hosts: ["localhost:5044", "localhost:5045"]
  loadbalance: true
  bulk_max_size: 2048
-------------------------------------------------------------------------------
