Adding SSL security to log forwarding from Filebeat to Elasticsearch, Logstash, Kibana (ELK Stack) on Ubuntu 14.04

01/07/2017 - ELASTICSEARCH, LINUX

In this example, we are going to configure SSL log forwarding from WEB server (where Filebeat is installed) to ELK server (where Elasticsearch Logstash Kibana (ELK stack) is installed) on Ubuntu 14.04. Filebeat watches regular Apache access logs on WEB server and forwards them to Logstash on ELK server. This is how our example works.


ELK server


Create SSL related folders.


$ sudo mkdir -p /etc/pki/tls/certs
$ sudo mkdir /etc/pki/tls/private

Find [ v3_ca ] and add line below underneath. This is to prevent "TLS handshake" error.


$ sudo nano /etc/ssl/openssl.cnf
subjectAltName = IP: 192.168.50.40

Create certificate and private key.


$ cd /etc/pki/tls
$ sudo openssl req -config /etc/ssl/openssl.cnf -x509 -days 3650 -batch -nodes -newkey rsa:2048 -keyout private/logstash-forwarder.key -out certs/logstash-forwarder.crt

Copy certificate file over WEB server.


$ sudo scp /etc/pki/tls/certs/logstash-forwarder.crt vagrant@192.168.50.50:/tmp

Update "input" block of Logstash config file. I assume that your config is actually working so I will only deal with "input" block for now.


$ sudo nano /etc/logstash/conf.d/web-apache-access.conf
input {
    beats {
        port => 5044
        ssl => true
        ssl_certificate => "/etc/pki/tls/certs/logstash-forwarder.crt"
        ssl_key => "/etc/pki/tls/private/logstash-forwarder.key"
    }
}

Validating configuration file. Takes about 10 seconds to get confirmation.


$ sudo /opt/logstash/bin/logstash --configtest -f /etc/logstash/conf.d/web-apache-access.conf
Configuration OK

Stop and start Logstash.


$ sudo service logstash stop
$ sudo service logstash start

WEB server


As you see below, certificate has been copied successfully.


$ ls -l /tmp/
-rw-r--r-- 1 vagrant vagrant 1229 Jul  1 18:24 logstash-forwarder.crt

Create SSL related folder.


$ sudo mkdir -p /etc/pki/tls/certs

Copy certificate.


$ sudo cp /tmp/logstash-forwarder.crt /etc/pki/tls/certs/

Update "tls" block of Filebeat config file.


output:
    logstash:
        tls:
            certificate_authorities: ["/etc/pki/tls/certs/logstash-forwarder.crt"]

Stop and start Filebeat.


$ sudo service filebeat stop
$ sudo service filebeat start

Validate the Logstash server's certificate.


$curl -v --cacert /etc/pki/tls/certs/logstash-forwarder.crt https://192.168.50.40:5044

* Rebuilt URL to: https://192.168.50.40:5044/
* Hostname was NOT found in DNS cache
*   Trying 192.168.50.40...
* Connected to 192.168.50.40 (192.168.50.40) port 5044 (#0)
* successfully set certificate verify locations:
*   CAfile: /etc/pki/tls/certs/logstash-forwarder.crt
  CApath: /etc/ssl/certs
* SSLv3, TLS handshake, Client hello (1):
* SSLv3, TLS handshake, Server hello (2):
* SSLv3, TLS handshake, CERT (11):
* SSLv3, TLS handshake, Server key exchange (12):
* SSLv3, TLS handshake, Server finished (14):
* SSLv3, TLS handshake, Client key exchange (16):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSL connection using ECDHE-RSA-AES128-SHA256
* Server certificate:
* 	 subject: C=AU; ST=Some-State; O=Internet Widgits Pty Ltd
* 	 start date: 2017-07-01 00:52:04 GMT
* 	 expire date: 2027-06-29 00:52:04 GMT
* 	 subjectAltName: 192.168.50.40 matched
* 	 issuer: C=AU; ST=Some-State; O=Internet Widgits Pty Ltd
* 	 SSL certificate verify ok.
> GET / HTTP/1.1
> User-Agent: curl/7.35.0
> Host: 192.168.50.40:5044
> Accept: */*
> 
* SSLv3, TLS alert, Client hello (1):
* Empty reply from server
* Connection #0 to host 192.168.50.40 left intact
curl: (52) Empty reply from server