How to Configure Logstash Configuration File When Running as a Service on Ubuntu


2 views

When you install Logstash via APT repository as shown in your dpkg -s logstash output, the package creates several important files:

/etc/default/logstash
/etc/default/logstash-web
/etc/logrotate.d/logstash

The primary configuration directory for service-based Logstash installations is:

/etc/logstash/conf.d/

This follows the standard Unix/Linux convention for service configuration locations. Any .conf files placed here will be automatically loaded by the Logstash service.

Here's a basic configuration example you could place in /etc/logstash/conf.d/example.conf:

input {
  file {
    path => "/var/log/syslog"
    start_position => "beginning"
  }
}

filter {
  grok {
    match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:$$%{POSINT:syslog_pid}$$)?: %{GREEDYDATA:syslog_message}" }
  }
}

output {
  elasticsearch {
    hosts => ["localhost:9200"]
  }
  stdout { codec => rubydebug }
}

After adding your configuration files, manage the service with:

# Reload configurations without full restart
sudo service logstash reload

# Full restart when needed
sudo service logstash restart

# Check status
sudo service logstash status

Service logs are typically found at:

/var/log/logstash/logstash.log

For debugging configuration issues, you can temporarily run Logstash in foreground mode with:

sudo -u logstash /opt/logstash/bin/logstash -f /etc/logstash/conf.d/ --debug
  • Use separate .conf files for different log sources
  • Keep individual files under 100 lines when possible
  • Test configurations with --configtest flag first
  • Use proper file permissions (usually root:logstash and 640 mode)

When you install Logstash via APT repository on Ubuntu, it creates a system service that automatically handles startup and management. The key configuration files for the service are located in:

/etc/default/logstash
/etc/default/logstash-web
/etc/logrotate.d/logstash

The primary configuration directory for Logstash service is:

/etc/logstash/conf.d/

Any .conf files placed in this directory will be automatically loaded by the Logstash service in alphabetical order.

Here's how to create a custom configuration for your service:

sudo nano /etc/logstash/conf.d/01-my-config.conf

Example configuration for file input and elasticsearch output:

input {
  file {
    path => "/var/log/nginx/access.log"
    start_position => "beginning"
  }
}

output {
  elasticsearch {
    hosts => ["localhost:9200"]
  }
}

Before restarting the service, validate your config:

sudo /opt/logstash/bin/logstash --configtest -f /etc/logstash/conf.d/

After configuration changes, manage the service with:

# Reload configuration
sudo systemctl reload logstash

# Or full restart
sudo systemctl restart logstash

# Check status
sudo systemctl status logstash

To use a non-default configuration path, modify the service file:

sudo nano /etc/systemd/system/logstash.service

Add your custom path to the ExecStart command:

ExecStart=/opt/logstash/bin/logstash -f /path/to/your/config

Then reload systemd:

sudo systemctl daemon-reload
sudo systemctl restart logstash

The /etc/default/logstash file can be used to set environment variables:

LS_HEAP_SIZE="2g"
LS_JAVA_OPTS="-Djava.io.tmpdir=/var/lib/logstash"

Check Logstash logs for configuration errors:

sudo journalctl -u logstash -f

Or view the specific log file:

tail -f /var/log/logstash/logstash-plain.log