Awesome Open Source
Awesome Open Source

Elastic Stack in a Box

This repository will install the Elastic Stack (Elasticsearch, Logstash, Kibana, and Beats) and optionally start a trial of commercial features. You can either start from scratch and configure everything with Vagrant and Ansible or you can download the final OVA image.

Features

  • Filebeat system, auditd, logstash, mongodb, nginx, osquery, and redis modules
  • Filebeat collecting Kibana JSON logs from /var/log/kibana/kibana.log
  • Auditbeat file_integrity module on /home/vagrant/ directory and auditd module
  • Heartbeat pinging nginx every 10s
  • Metricbeat system, docker, elasticsearch, kibana, logstash, mongodb, nginx and redis modules
  • Packetbeat sending its data via Redis + Logstash, monitoring flows, ICMP, DNS, HTTP (nginx and Kibana), Redis, and MongoDB (generate traffic with $ mongo /elastic-stack/mongodb.js)
  • The pattern for nginx is already prepared in /opt/logstash/patterns/ and you can collect /var/log/nginx/access.log with Filebeat and add a filter in Logstash with the pattern as an exercise

Vagrant and Ansible

Do a simple vagrant up by using Vagrant's Ansible provisioner. All you need is a working Vagrant installation (2.2.4+ but the latest version is always recommended), a provider (tested with the latest VirtualBox version), and 3GB of RAM.

With the Ansible playbooks in the /elastic-stack/ folder you can configure the whole system step by step. Just run them in the given order inside the Vagrant box:

> vagrant ssh
$ cd /elastic-stack/
$ ansible-playbook 1_configure-elasticsearch.yml
$ ansible-playbook 2_configure-kibana.yml
$ ansible-playbook 3_configure-logstash.yml
$ ansible-playbook 4_configure-auditbeat.yml
$ ansible-playbook 4_configure-filebeat.yml
$ ansible-playbook 4_configure-heartbeat.yml
$ ansible-playbook 4_configure-metricbeat.yml
$ ansible-playbook 4_configure-packetbeat.yml
$ ansible-playbook 5_configure-dashboards.yml

Or if you are in a hurry, run all playbooks with $ /elastic-stack/all.sh at once.

OVA Image

If Vagrant and Ansible sound too complicated, there is also the final result: An OVA image, which you can import directly into VirtualBox:

Kibana

Access Kibana at https://127.0.0.1:5601.

Test Data

You can use /opt/injector.jar to generate test data in the person index. To generate 100,000 documents in batches of 1,000 run the following command:

$ java -jar /opt/injector.jar 100000 1000

Logstash Demo

You can play around with a Logstash example by calling $ sudo /usr/share/logstash/bin/logstash --path.settings /etc/logstash -f /elastic-stack/raffle/raffle.conf (it can take some time) and you will find the result in the raffle index.



Alternative Project Comparisons
Related Awesome Lists
Top Programming Languages
Top Projects

Get A Weekly Email With Trending Projects For These Topics
No Spam. Unsubscribe easily at any time.
Shell (176,042
Docker (98,266
Mongodb (30,624
Redis (24,039
Ansible (21,412
Nginx (16,745
Stack (16,047
Vagrant (14,793
Elasticsearch (11,394
Virtualbox (4,494
Elastic (3,996
Kibana (3,046
Logstash (2,848
Heartbeat (1,141
Filebeat (549
Metricbeat (49
Ova (27
Packetbeat (10
Auditbeat (7