一、 Elasticeach介绍和安装
1.1 资源准备
| 主机 | 操作系统 | 软件及版本 | 数据盘 |
|---|---|---|---|
| study62 | centos7.6 | elk5.4.0 | sdb |
| study63 | centos7.6 | elk5.4.0 | sdb |
1.2 环境准备
1.2.1 准备数据盘
此处以study62为例，study63采取同样操作
[root@study62 src]# mkfs.xfs /dev/sdb
meta-data=/dev/sdb isize=512 agcount=4, agsize=1310720 blks
= sectsz=512 attr=2, projid32bit=1
= crc=1 finobt=0, sparse=0
data = bsize=4096 blocks=5242880, imaxpct=25
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii-ci=0 ftype=1
log =internal log bsize=4096 blocks=2560, version=2
= sectsz=512 sunit=0 blks, lazy-count=1
[root@study62 ~]# mkdir /data
[root@study62 ~]# blkid /dev/sdb
/dev/sdb: UUID="00d1e064-ebea-42ec-a341-19e439877463" TYPE="xfs"
[root@study62 ~]# vi /etc/fstab
UUID="00d1e064-ebea-42ec-a341-19e439877463" /data xfs defaults 0 0
[root@study62 ~]# mount -a
[root@study62 ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/centos-root 17G 1.5G 16G 9% /
devtmpfs 898M 0 898M 0% /dev
tmpfs 910M 0 910M 0% /dev/shm
tmpfs 910M 9.5M 901M 2% /run
tmpfs 910M 0 910M 0% /sys/fs/cgroup
/dev/sda1 1014M 146M 869M 15% /boot
tmpfs 182M 0 182M 0% /run/user/0
/dev/sdb 20G 33M 20G 1% /data
1.2.2 防火墙和selinux
关闭所有服务器的防火墙和selinux，包括web服务器、redis和logstash服务器的防火墙和selinux全部关闭，此步骤是为了避免出现防火墙策略或selinux安全权限引起的各种位置问题，一下以study62主机的命令为例，其他服务器命令参照示例
修改文件描述符
[root@study62 ~]# echo "* soft nofile 65536" >> /etc/security/limits.conf
[root@study62 ~]# echo "* hard nofile 65536" >> /etc/security/limits.conf
1.2.3 添加hosts
[root@study63 src]# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
10.0.0.62 study62.exmaple.com
10.0.0.63 study63.exmaple.com
10.0.0.64 study64.exmaple.com
10.0.0.65 study65.exmaple.com
10.0.0.66 study66.exmaple.com
10.0.0.67 study67.exmaple.com
10.0.0.68 study68.exmaple.com
1.2.4 设置epel源、安装基本操作命令并同步时间
[root@study62 ~]# curl -o /etc/yum.repos.d/epel.repo http://mirrors.aliyun.com/repo/epel-7.repo
[root@study62 ~]# yum install net-tools vim lrzsz tree screen lsof tcpdump wget ntpdate -y
[root@study62 ~]# ln -s /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
[root@study62 ~]# echo "*/5 * * * * ntpdate ntp1.aliyun.com &>/dev/null && hwclock -w" >>/var/spool/cron/root
[root@study62 ~]# systemctl restart crond
[root@study62 ~]# systemctl start ntpdate
[root@study62 ~]# systemctl enable ntpdate
[root@study62 ~]# reboot #重启检查各项配置是否生效，没有问题的话做快照以方便后期还原
1.2.5 软件包上传到主机上
[root@study62 ~]# mkdir /opt/src/
[root@study62 src]# ll
total 353392
-rw-r--r-- 1 root root 33211227 Apr 6 08:57 elasticsearch-5.4.0.rpm
-rw-r--r-- 1 root root 178939634 Apr 6 09:52 jdk-8u241-linux-x64.rpm
-rw-r--r-- 1 root root 56266315 Apr 6 09:02 kibana-5.4.0-x86_64.rpm
-rw-r--r-- 1 root root 93448667 Apr 6 09:00 logstash-5.4.0.rpm
1.3 安装部署
1.3.1 安装Elasticsearch
[root@study62 src]# yum install jdk-8u241-linux-x64.rpm elasticsearch-5.4.0.rpm
Loaded plugins: fastestmirror
Examining jdk-8u241-linux-x64.rpm: 2000:jdk1.8-1.8.0_241-fcs.x86_64
Marking jdk-8u241-linux-x64.rpm to be installed
Examining elasticsearch-5.4.0.rpm: elasticsearch-5.4.0-1.noarch
Marking elasticsearch-5.4.0.rpm to be installed
Resolving Dependencies
--> Running transaction check
---> Package elasticsearch.noarch 0:5.4.0-1 will be installed
---> Package jdk1.8.x86_64 2000:1.8.0_241-fcs will be installed
--> Finished Dependency Resolution
Dependencies Resolved
=================================================================================
Package Arch Version Repository Size
=================================================================================
Installing:
elasticsearch noarch 5.4.0-1 /elasticsearch-5.4.0 35 M
jdk1.8 x86_64 2000:1.8.0_241-fcs /jdk-8u241-linux-x64 295 M
Transaction Summary
=================================================================================
Install 2 Packages
1.3.2 编辑各elasticsearch服务器的服务配置文件
[root@study62 src]# vim /etc/elasticsearch/elasticsearch.yml
cluster.name: elk-cluster
node.name: elk-node1
path.data: /data/elkdata
path.logs: /data/logs
bootstrap.memory_lock: true #内存锁定
network.host: 10.0.0.62
http.port: 9200
discovery.zen.ping.unicast.hosts: ["10.0.0.62", "10.0.0.63"]
1.3.3 创建数据目录
[root@study62 src]# mkdir /data/elkdate /data/logs
[root@study62 src]# chown -R elasticsearch:elasticsearch /data
1.3.4 启动服务
[root@study62 src]# systemctl start elasticsearch
systemctl enable elasticsearch
1.3.5 查看端口
[root@study63 src]# ss -tnl
State Recv-Q Send-Q Local Address:Port Peer Address:Port
LISTEN 0 128 *:22 *:*
LISTEN 0 100 127.0.0.1:25 *:*
LISTEN 0 128 ::ffff:10.0.0.63:9200 :::*
LISTEN 0 128 ::ffff:10.0.0.63:9300 :::*
LISTEN 0 128 :::22 :::*
LISTEN 0 100 ::1:25 :::*
[root@study63 src]#
1.3.6 修改内存限制，并同步配置文件
vim /usr/lib/systemd/system/elasticsearch.service
LimitMEMLOCK=infinity #去掉注释 开启内存锁定需要此选项
[root@study62 src]# vim /etc/elasticsearch/jvm.options
-Xms2g
-Xmx2g
https://www.elastic.co/guide/en/elasticsearch/reference/5.4/heap-size.html
1.3.7 重启服务
[root@study62 src]# systemctl restart elasticsearch.service
1.3.8 其他节点参照study62主机配置
[root@study62 src]# grep "^[a-Z]" /etc/elasticsearch/elasticsearch.yml
grep "^[a-Z]" /etc/elasticsearch/elasticsearch.yml
cluster.name: elk-cluster #ELK集群名称，名称相同即属于同一个集群
node.name: elk-node1 #本机集群内的节点名称
path.data: /data/elkdata #数据保存目录
path.logs: /data/logs # 日志保存目录
bootstrap.memory_lock: true # 服务启动的时候锁定足够的内存，防止数据写入swap
network.host: 10.0.0.62 #监听IP（0.0.0.0）
http.port: 9200
discovery.zen.ping.unicast.hosts: ["10.0.0.62", "10.0.0.63"]
1.3.9 查看elasticsearch进程
[root@study62 src]# ps -ef | grep java
elastic+ 29106 1 0 10:31 ? 00:00:37 /bin/java -Xms2g -Xmx2g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+DisableExplicitGC -XX:+AlwaysPreTouch -server -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -Djdk.io.permissionsUseCanonicalPath=true -Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0 -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -Dlog4j.skipJansi=true -XX:+HeapDumpOnOutOfMemoryError -Des.path.home=/usr/share/elasticsearch -cp /usr/share/elasticsearch/lib/* org.elasticsearch.bootstrap.Elasticsearch -p /var/run/elasticsearch/elasticsearch.pid --quiet -Edefault.path.logs=/var/log/elasticsearch -Edefault.path.data=/var/lib/elasticsearch -Edefault.path.conf=/etc/elasticsearch
[root@study63 src]# ps -ef | grep java
elastic+ 9941 1 0 10:34 ? 00:00:32 /bin/java -Xms2g -Xmx2g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+DisableExplicitGC -XX:+AlwaysPreTouch -server -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -Djdk.io.permissionsUseCanonicalPath=true -Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0 -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -Dlog4j.skipJansi=true -XX:+HeapDumpOnOutOfMemoryError -Des.path.home=/usr/share/elasticsearch -cp /usr/share/elasticsearch/lib/* org.elasticsearch.bootstrap.Elasticsearch -p /var/run/elasticsearch/elasticsearch.pid --quiet -Edefault.path.logs=/var/log/elasticsearch -Edefault.path.data=/var/lib/elasticsearch -Edefault.path.conf=/etc/elasticsearch
[root@study62 src]# curl -i -XGET 'http://10.0.0.62:9200/_cluster/health?pretty=true'
HTTP/1.1 200 OK
content-type: application/json; charset=UTF-8
content-length: 327
{
"name" : "elk-node1",
"cluster_name" : "elk-cluster",
"cluster_uuid" : "dYokjxQERsKiRjLo2EXV5Q",
"version" : {
"number" : "5.4.0",
"build_hash" : "780f8c4",
"build_date" : "2017-04-28T17:43:27.229Z",
"build_snapshot" : false,
"lucene_version" : "6.5.0"
},
"tagline" : "You Know, for Search"
}
1.4 安装elasticsearch插件值head
插件是为了完成不同的功能，官方提供了一些插件但大部分是收费的，另外也有一些开发爱好者提供的插件，可以实现elasticsearch集群状态监控与管理配置等功能。
1.4.1 安装5.x版本的head插件
在elasticsearch5.x版本以后不再支持直接安装head插件，而是需要通过启动一个服务方式，git地址：https://github.com/mobz/elasticsearch-head
[root@study62 src]# yum install git -y
[root@study62 src]# git https://github.com/mobz/elasticsearch-head.git
[root@study62 src]# cd elasticsearch-head/
[root@study62 elasticsearch-head]# yum install npm -y
# NPM的全程是Node Package Manager，是随同NodeJS一起安装的包管理和开发工具，它很方便让JavaScript开发者下载、安装、上传以及管理已经安装的包。
[root@study62 elasticsearch-head]# npm install grunt -save
[root@study62 elasticsearch-head]# ll node_modules/grunt/bin/ #确认生成文件
total 4
-rwxr-xr-x 1 root root 53 Apr 6 2016 grunt
[root@study62 elasticsearch-head]# npm install #执行安装
[root@study62 elasticsearch-head]# npm start &
1.4.1.1 修改elasticsearch服务配置文件
开启跨域访问支持，然后重启elasticsearch服务：
[root@study62 ~]# vi /etc/elasticsearch/elasticsearch.yml
http.cors.enabled: true
http.cors.allow-origin: "*"
[root@study62 ~]# systemctl restart elasticsearch
安装完成后通过浏览器访问 http://10.0.0.62:9100
1.4.1.2 docker版本启动head插件
[root@study63 src]# yum install docker -y [root@study63 src]# vi /etc/docker/daemon.json { "registry-mirrors": ["https://5md6dgq1.mirror.aliyuncs.com"] } [root@study63 src]# systemctl start docker [root@study63 src]# systemctl enable docker [root@study63 src]# docker run -p 9100:9100 mobz/elasticsearch-head:5
浙公网安备 33010602011771号