Install Elastic stack

1. 安装环境

系统版本：centos 6.9
java版本：1.8.0_181
程序版本：6.6 （整个stack需保持相同的版本）

2. 安装顺序

1 Elasticsearch 
2 Kibana 
3 Logstash 
4 Beats 
5 Elasticsearch Hadoop 

3. 安装 Elasticsearch

tar -xzf elasticsearch-6.6.2.tar.gz
useradd elasticsearch
chown -R elasticsearch:elasticsearch /opt/elasticsearch-6.6.2
su - elasticsearch
/opt/elasticsearch-6.6.2/bin/elasticsearch -d

启动报错：

Error: max number of threads [1024] for user [elasticsearch] is too low, increase to at least [4096]
		 vi /etc/security/limits.d/90-nproc.conf
		 add one line
		 elasticsearch          soft    nproc     4096
		 
Error: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
		  vi /etc/sysctl.conf
		  add one line
		  vm.max_map_count=655360
		  sysctl -p
Error: max file descriptors [4096] for elasticsearch process is too low, increase to at least [65536]
                  vi /etc/security/limits.conf

                  elasticsearch    hard    nofile          102400
                  elasticsearch    soft    nofile          102400

Error: system call filters failed to install; check the logs and fix your configuration or disable system call filters at your own risk
		  vi elasticsearch.yml
		  
		  network.host: 0.0.0.0
		  bootstrap.system_call_filter: false

curl http://localhost:9200/

 

4. 安装kibana

tar -xzf kibana-6.6.2-darwin-x86_64.tar.gz

vi /opt/kibana-6.6.2-linux-x86_64/config/kibana.yml
    server.host: "192.168.1.197"
    elasticsearch.hosts: ["http://localhost:9200"]

　　　

5. 安装logstash

tar xaf logstash-6.6.2.tar.gz

vi /opt/logstash-6.6.2/config/pipelines.yml
    - pipeline.id: test
      pipeline.workers: 1
      pipeline.batch.size: 1
      config.string: input { beats  { port=>"5045" } }  output { elasticsearch { index=>"logstat-test1" hosts=>["localhost:9200"] } }

vi /opt/logstash-6.6.2/config/pipelines.yml

- pipeline.id: test
  pipeline.workers: 1
  pipeline.batch.size: 1
  config.string: input { beats  { port=>"5045" } } filter {grok {match => { "message" => '%{IPV4:clientip} - - \[%{HTTPDATE:time}\] %{IPV4:serverip} "%{WORD:verb} %{NOTSPACE:request} HTTP/%{NUMBER:httpversion}|%{DATA:rawrequest}" %{NUMBER:response} %{NUMBER:bytes}|-' }}}  output { elasticsearch { index=>"httpd_log_188_%{+YYYY.MM.dd}" hosts=>["elasticsearch.oohome.net:9200"] } }

　

6. 安装filebeat

tar xf filebeat-6.6.2-linux-x86_64.tar.gz

vi  /opt/filebeat-6.6.2-linux-x86_64/filebeat.yml
    filebeat.inputs:
    - type: log
      enabled: true
      paths:
        - /var/log/secure

    output.logstash:
      hosts: ["localhost:5045"]