elasticsearch

注: 如果是虚拟机, 内存至少在1.5G以上, 否则自启无法生效

elasticsearch不支持root用户启动，所以需要新创建一个用户来启动。

创建组

groupadd elasticsearch

创建用户

useradd es

设置es用户的密码

passwd es

将用户es添加到elasticsearch 组里面： 

usermod -G elasticsearch es

在root下向用户授权目录

chown -R es:es /usr/local/elasticsearch

 设置权限： 

visudo

在root下载添加一行，es 后面的内容和root后面的一致
   

修改配置文件

vim /usr/local/elasticsearch/config/elasticsearch.yml

 取消注释：

cluster.name: my-application

node.name: node

取消注释，并修改为本机ip

network.host: xxx:xxx:xxx:xxx

取消注释：

http.port: 9200

取消注释，并修改

path.data: /usr/local/elasticsearch/data 

path.logs: /usr/local/elasticsearch/logs

修改/etc/sysctl.conf

vim /etc/sysctl.conf

在文件的末尾加： 

vm.max_map_count=655360

保存并退出后执行下面的命令：

sysctl -p

修改文件 /etc/security/limits.conf，在文件的末尾添加：

* soft nofile 65536
* hard nofile 65536
* soft nproc  4096
* hard nproc  4096

重启服务器

reboot

切换用户： 

su es

进入到 /usr/local/elasticsearch/bin，启动：

./elasticsearch

通过外网无法访问，在测试环境上可通过关闭防火墙解决：

systemctl disable firewalld

systemctl stop firewalld

运行上述两条命令后防火墙就关闭了，且不会开机自启动。注意：在生产环境切记不要随意关闭防火墙

设置elasticsearch为开机自启动

创建目录并设置权限

mkdir /usr/local/elasticsearch/run 

chmod 777 /usr/local/elasticsearch/run

创建es服务系统配置文件，在/etc/sysconfig/ 下创建文件elasticsearch，内容为

vim /etc/sysconfig/elasticsearch

#######################
# Elasticsearch #
#######################

# Elasticsearch home directory
ES_HOME=/usr/local/elasticsearch

# Elasticsearch Java path
JAVA_HOME=/usr/local/java
CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar:$JAVA_HOMR/jre/lib

# Elasticsearch configuration directory
ES_PATH_CONF=/usr/local/elasticsearch/config

# Elasticsearch PID directory
PID_DIR=/usr/local/elasticsearch/run

#############################
# Elasticsearch Service #
#############################

# SysV init.d
# The number of seconds to wait before checking if elasticsearch started successfully as a daemon process
ES_STARTUP_SLEEP_TIME=5

################################
# Elasticsearch Properties #
################################
# Specifies the maximum file descriptor number that can be opened by this process
# When using Systemd,this setting is ignored and the LimitNOFILE defined in
# /usr/lib/systemd/system/elasticsearch.service takes precedence
#MAX_OPEN_FILES=65536

# The maximum number of bytes of memory that may be locked into RAM
# Set to "unlimited" if you use the 'bootstrap.memory_lock: true' option
# in elasticsearch.yml.
# When using Systemd,LimitMEMLOCK must be set in a unit file such as
# /etc/systemd/system/elasticsearch.service.d/override.conf.
#MAX_LOCKED_MEMORY=unlimited

# Maximum number of VMA(Virtual Memory Areas) a process can own
# When using Systemd,this setting is ignored and the 'vm.max_map_count'
# property is set at boot time in /usr/lib/sysctl.d/elasticsearch.conf
#MAX_MAP_COUNT=262144

创建es服务，在 /usr/lib/systemd/system/ 目录下创建 elasticsearch.service文件，内容如下：

[Unit]
Description=Elasticsearch
Documentation=http://www.elastic.co
Wants=network-online.target
After=network-online.target

[Service]
Environment=ES_HOME=/usr/local/elasticsearch
Environment=ES_PATH_CONF=/usr/local/elasticsearch/config
# Environment=PID_DIR=/var/run/elasticsearch 此目录会随重启删除
Environment=PID_DIR=/usr/local/elasticsearch/run
EnvironmentFile=/etc/sysconfig/elasticsearch
WorkingDirectory=/usr/local/elasticsearch
User=es
Group=elasticsearch
ExecStart=/usr/local/elasticsearch/bin/elasticsearch -p ${PID_DIR}/elasticsearch.pid

# StandardOutput is configured to redirect to journalctl since
# some error messages may be logged in standard output before
# elasticsearch logging system is initialized. Elasticsearch
# stores its logs in /var/log/elasticsearch and does not use
# journalctl by default. If you also want to enable journalctl
# logging, you can simply remove the "quiet" option from ExecStart.
StandardOutput=journal
StandardError=inherit

# Specifies the maximum file descriptor number that can be opened by this process
LimitNOFILE=65536

# Specifies the maximum number of process
LimitNPROC=4096

# Specifies the maximum size of virtual memory
LimitAS=infinity

# Specifies the maximum file size
LimitFSIZE=infinity

# Disable timeout logic and wait until process is stopped
TimeoutStopSec=0

# SIGTERM signal is used to stop the Java process
KillSignal=SIGTERM

# Send the signal only to the JVM rather than its control group
KillMode=process

# Java process is never killed
SendSIGKILL=no

# When a JVM receives a SIGTERM signal it exits with code 143
SuccessExitStatus=143
 
[Install]
WantedBy=multi-user.target

给脚本赋权限

chmod +x /usr/lib/systemd/system/elasticsearch.service

重新加载systemd的守护线程

systemctl daemon-reload

开机启动生效

systemctl enable elasticsearch.service

启动elasticsearch.service

systemctl start elasticsearch.service

查看日志信息

journalctl -u elasticsearch.service

重启服务器

reboot

检查开机自启动是否生效,访问 http://xxx.xxx.xxx.xxx:9200/

自启失败, 查询

 

cat /usr/local/elasticsearch/logs/elasticsearch.log

 

安装ik中文分词器

下载地址

wget http://linux-1251121573.cosgz.myqcloud.com/soft/elk/elasticsearch-analysis-ik-6.2.2.zip

将压缩包解压，重命名为ik

unzip elasticsearch-analysis-ik-6.2.2.zip
mv elasticsearch ik

将ik文件夹上传到 

mv ik /usr/local/elasticsearch/plugins

配置 elasticsearch 集群

编辑 elasticsearch 配置文件：

/usr/local/elasticsearch/config/elasticsearch.yml

分别在两台服务器中填写如下配置：

切记 cluster.name: node 这个名字两台服务器需要一样。

服务器一

cluster.name: node          

node.name: node1

network.host: 192.168.1.223

discovery.zen.ping.unicast.hosts: ["192.168.1.224:9300"]

服务器二

cluster.name: node          

node.name: node2

network.host: 192.168.1.224

discovery.zen.ping.unicast.hosts: ["192.168.1.223:9300"]

完成之后保存即可，集群配置完毕

查看 elasticsearch 进程

ps -ef|grep elas

杀掉 elasticsearch 进程

kill -9 进程id

重新启动 elasticsearch

安装 kibana

下载安装包

 

wget https://artifacts.elastic.co/downloads/kibana/kibana-6.2.2-linux-x86_64.tar.gz

 

解压安装包

tar -zxvf kibana-6.2.2-linux-x86_64.tar.gz

修改配置文件

vi comfig/kibana.yml

去掉注释

server.port: 5601

server.host: "192.168.18.177"

elasticsearch.hosts: ["http://192.168.18.177:9200"]

访问： http://xxx.xxx.xxx.xxx:5601

安装Logstash

读取文件直接发送到es

修改 /usr/local/logstash/config/logstash-sample.conf

input {
  #beats {
   # port => 5044
  #}
  file {
    path => "/var/log/httpd/access_log"
    start_position => beginning
  }
}

output {
  elasticsearch {
    hosts => ["http://localhost:9200"]
    index => "%{[@metadata][logstash]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
    #user => "elastic"
    #password => "changeme"
  }
}

  检查配置文件是否正确：（假设当前目录为/usr/local/logstash/config/）

../bin/logstash -t -f logstash-sample.conf

  启动

../bin/logstash -f logstash-sample.conf 　

加载本文件夹所有配置文件启动 

../bin/logstash -f ./ 

或后台启动

nohup ../bin/logstash -f config/ &

五、安装elasticsearch-head（系统需要开放9300端口才能插件访问）

一、elasticsearch6.X版本之后，不支持命令行安装head插件，需要手动安装。

https://github.com/mobz/elasticsearch-head

或者：

wget http://linux-1251121573.cosgz.myqcloud.com/soft/elk/elasticsearch-head-master.zip

二、安装nodejs

直接无法yum安装noodejs的，需要添加nodejs源

curl --silent --location https://rpm.nodesource.com/setup | bash -

安装

yum install -y nodejs

安装完成后，进入 elasticsearch-head插件文件夹。

执行命令安装 grunt

npm install -g grunt -cli 

之后进行安装pathomjs

npm install 

以上即安装完所有插件

执行 命令即可启动elasticsearch-head

grunt server

通过访问：ip:9100 浏览web页面。

简单安装完elasticsearch-head是无法连接elasticsearch服务器的，

需要修改 elasticsearch中/config/elasticsearch.yml配置文件，添加如下参数：

增加如下字段

# 允许插件访问

http.cors.enabled: true

http.cors.allow-origin: "*"

插件同样需要开放9100端口

firewall-cmd --zone=public --add-port=9100/tcp --permanent

firewall-cmd --reload

docker方式安装elasticsearch-head插件

#拉取镜像

docker pull docker pull mobz/elasticsearch-head:5-apline

# 创建容器文件夹

mkdir -p /data/database/elasticsearch-6.2.2/es-head/

# 编辑配置文件

vi /data/database/elasticsearch-6.2.2/es-head/Gruntfile.js

# 添加如下内容

connect: {

        server: {

            options: {

                /* 默认监控：127.0.0.1,修改为：0.0.0.0 */

                hostname: '0.0.0.0',

                port: 9100,

                base: '.',

                keepalive: true

            }

        }
}　

#运行docker命令创建镜像

docker run -d --restart=always --name es-head -p 9100:9100 -v /data/database/elasticsearch-6.2.2/es-head/Gruntfile.js: