从0开始搭建elfk单点、集群、加密集群
| 主机IP | 主机名 |
|---|---|
| 10.0.0.91 | elk91 |
| 10.0.0.92 | elk92 |
| 10.0.0.93 | elk93 |
环境准备
| 主机IP | 主机名 | 操作系统 |
|---|---|---|
| 10.0.0.91 | elk91 | Ubuntu 22.04 LTS |
| 10.0.0.92 | elk92 | Ubuntu 22.04 LTS |
| 10.0.0.93 | elk93 | Ubuntu 22.04 LTS |
elk版本选择
架构所有版本一定要一致
| 服务名 | 版本 |
|---|---|
| elasticsearch | 7.17.28 |
| Kibana | 7.17.28 |
| Logstash | 7.17.28 |
| Filebeat | 7.17.28 |
ES单点部署测试
1.下载软件包
root@elk91:~# wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.17.28-amd64.deb
2.安装ES程序
root@elk91:~# dpkg -i elasticsearch-7.17.28-amd64.deb
3.修改配置文件
root@elk91:~# vim /etc/elasticsearch/elasticsearch.yml
修改内容如下：
root@elk91:~# cat /etc/elasticsearch/elasticsearch.yml | egrep -v "^#|^$"
#集群名称
cluster.name: dezyan-linux96
#ES的数据存储路径
path.data: /var/lib/elasticsearch
#ES的日志存储路径
path.logs: /var/log/elasticsearch
#ES服务监听的地址
network.host: 0.0.0.0
#配置服务端口
http.port: 9200
#ES服务模式，此处设置单点只是为了测试
discovery.type: single-node
5.启动ElasticSearch服务
root@elk91:~# systemctl enable --now elasticsearch.service
#查看端口信息
root@elk91:~# ss -ntl | egrep "9[2|3]00"
LISTEN 0 4096 *:9300 *:*
LISTEN 0 4096 *:9200 *:*
6.查看服务验证启动成功
root@elk91:~# curl http://10.0.0.91:9200/_cat/nodes
10.0.0.91 34 97 7 0.11 0.09 0.03 cdfhilmrstw * elk91
ES集群未加密环境部署
1.停止单点服务
root@elk91:~# systemctl stop elasticsearch.service
2.拷贝软件包至其他节点
root@elk91:~# scp elasticsearch-7.17.28-amd64.deb 10.0.0.92:~
root@elk91:~# scp elasticsearch-7.17.28-amd64.deb 10.0.0.93:~
3.其他节点安装ES环境
root@elk92:~# dpkg -i elasticsearch-7.17.28-amd64.deb
root@elk93:~# dpkg -i elasticsearch-7.17.28-amd64.deb
4.修改ES集群配置文件
root@elk91:~# vim /etc/elasticsearch/elasticsearch.yml
…………
cluster.name: dezyan-96
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
network.host: 0.0.0.0
http.port: 9200
#此处去除了discovery.type: single-nod，改用了集群模式
discovery.seed_hosts: ["10.0.0.91", "10.0.0.92","10.0.0.93"]
cluster.initial_master_nodes: ["10.0.0.91","10.0.0.92","10.0.0.93"]
#复制配置文件至其他节点
root@elk91:~# scp /etc/elasticsearch/elasticsearch.yml 10.0.0.92:/etc/elasticsearch/
root@elk91:~# scp /etc/elasticsearch/elasticsearch.yml 10.0.0.93:/etc/elasticsearch/
5.所有节点同时启动ES服务
root@elk91:~# systemctl enable --now elasticsearch.service
root@elk92:~# systemctl enable --now elasticsearch.service
root@elk93:~# systemctl enable --now elasticsearch.service
6.检查集群是否正常工作
root@elk91:~# curl http://10.0.0.91:9200/_cat/nodes?v
ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
10.0.0.91 9 97 19 0.46 0.25 0.19 cdfhilmrstw * elk91
10.0.0.93 4 97 16 0.45 0.18 0.12 cdfhilmrstw - elk93
10.0.0.92 19 97 16 0.72 0.27 0.10 cdfhilmrstw - elk92
kibana对接未加密的ES集群
1.下载kibana
root@elk91:~# wget https://artifacts.elastic.co/downloads/kibana/kibana-7.17.28-amd64.deb
2.安装kibana
root@elk91:~# dpkg -i kibana-7.17.28-amd64.deb
3.修改kibana配置文件
root@elk91:~# egrep -v "^#|^$" /etc/kibana/kibana.yml
server.port: 5601
server.host: "0.0.0.0"
elasticsearch.hosts: ["http://10.0.0.91:9200","http://10.0.0.92:9200","http://10.0.0.93:9200"]
i18n.locale: "zh-CN"
- 相关参数说明
server.port
kibana监听的端口。
server.host
kibana监听的IP地址。
elasticsearch.hosts
kibana管理的ES集群信息。
i18n.locale
安装时选择的语言。
4.启动kibana服务
root@elk91:~# systemctl enable --now kibana.service
root@elk91:~# ss -ntl | grep 5601
LISTEN 0 511 0.0.0.0:5601 0.0.0.0:*
5.访问测试
http://10.0.0.91:5601/
对ES集群的加密
1.生成证书文件
root@elk91:~# /usr/share/elasticsearch/bin/elasticsearch-certutil cert -out /etc/elasticsearch/elastic-certificates.p12 -pass ""
#会生成如下文件
root@elk91:~# ll /etc/elasticsearch/elastic-certificates.p12
-rw------- 1 root elasticsearch 3596 Mar 27 19:09 /etc/elasticsearch/elastic-certificates.p12
2.将证书文件授权并拷贝到其他节点
root@elk91:~# chmod 640 /etc/elasticsearch/elastic-certificates.p12
root@elk91:~# scp -p /etc/elasticsearch/elastic-certificates.p12 10.0.0.92:/etc/elasticsearch
root@elk91:~# scp -p /etc/elasticsearch/elastic-certificates.p12 10.0.0.93:/etc/elasticsearch
3.修改ES集群的配置文件
root@elk91:~# vim /etc/elasticsearch/elasticsearch.yml
cluster.name: dezyan-96
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
network.host: 0.0.0.0
http.port: 9200
discovery.seed_hosts: ["10.0.0.91", "10.0.0.92","10.0.0.93"]
cluster.initial_master_nodes: ["10.0.0.91","10.0.0.92","10.0.0.93"]
#在最后一行添加如下内容
#启用Elasticsearch的X-Pack安全功能
xpack.security.enabled: true
#启用传输层的SSL/TLS加密
xpack.security.transport.ssl.enabled: true
#设置SSL/TLS证书验证模式为certificate
xpack.security.transport.ssl.verification_mode: certificate
#指定SSL/TLS密钥库（keystore）的路径
xpack.security.transport.ssl.keystore.path: elastic-certificates.p12
#指定SSL/TLS信任库（truststore）的路径。
xpack.security.transport.ssl.truststore.path: elastic-certificates.p12
4.同步ES配置文件到其他节点
root@elk91:~# scp /etc/elasticsearch/elasticsearch.yml 10.0.0.92:/etc/elasticsearch/
root@elk91:~# scp /etc/elasticsearch/elasticsearch.yml 10.0.0.93:/etc/elasticsearch/
5.所有节点重启ES集群
root@elk91:~# systemctl restart elasticsearch.service
root@elk92:~# systemctl restart elasticsearch.service
root@elk93:~# systemctl restart elasticsearch.service
6.测试验证ES集群访问
root@elk91:~# curl 10.0.0.91:9200/_cat/nodes?v
{"error":{"root_cause":[{"type":"security_exception","reason":"missing authentication credentials for REST request [/_cat/nodes?v]","header":{"WWW-Authenticate":"Basic realm=\"security\" charset=\"UTF-8\""}}],"type":"security_exception","reason":"missing authentication credentials for REST request [/_cat/nodes?v]","header":{"WWW-Authenticate":"Basic realm=\"security\" charset=\"UTF-8\""}},"status":401}
7.生成随机密码
root@elk92:~# /usr/share/elasticsearch/bin/elasticsearch-setup-passwords auto
Initiating the setup of passwords for reserved users elastic,apm_system,kibana,kibana_system,logstash_system,beats_system,remote_monitoring_user.
The passwords will be randomly generated and printed to the console.
Please confirm that you would like to continue [y/N]y
Changed password for user apm_system
PASSWORD apm_system = Axnq1l0YE5X7VQ7NyznI
Changed password for user kibana_system
PASSWORD kibana_system = u8SXpb5TM97itEEngA5U
Changed password for user kibana
PASSWORD kibana = u8SXpb5TM97itEEngA5U
Changed password for user logstash_system
PASSWORD logstash_system = ZkI5M1vNpGQris6sbfUL
Changed password for user beats_system
PASSWORD beats_system = pFjnfannxAcHWIYS2t3I
Changed password for user remote_monitoring_user
PASSWORD remote_monitoring_user = 3ZMR0LgQlGpAySyAKWVs
Changed password for user elastic
PASSWORD elastic = QiDB74k42rF25CntalUw
8.使用密码验证集群是否正常
root@elk92:~# curl -u elastic:QiDB74k42rF25CntalUw 10.0.0.91:9200/_cat/nodes
10.0.0.93 14 95 10 0.27 0.26 0.15 cdfhilmrstw - elk93
10.0.0.91 27 97 12 0.24 0.31 0.22 cdfhilmrstw - elk91
10.0.0.92 24 96 14 0.44 0.43 0.24 cdfhilmrstw * elk92
kibana对接加密的ES集群
1.修改kibana的配置文件
root@elk91:~# egrep -v "^#|^$" /etc/kibana/kibana.yml
server.port: 5601
server.host: "0.0.0.0"
elasticsearch.hosts: ["http://10.0.0.91:9200","http://10.0.0.92:9200","http://10.0.0.93:9200"]
elasticsearch.username: "kibana_system"
#使用上面步骤中生成的密码
elasticsearch.password: "u8SXpb5TM97itEEngA5U"
i18n.locale: "zh-CN"
2.重启kibana
root@elk91:~# systemctl restart kibana.service
3.访问kibana的webUI重置管理员密码
10.0.0.91:5601
ES集群配置https证书及kibana登录
1.自建ca证书
root@elk91:~# /usr/share/elasticsearch/bin/elasticsearch-certutil ca --out /etc/elasticsearch/elastic-stack-ca.p12 --pass ""
#会生成如下文件
root@elk91:~# ll /etc/elasticsearch/elastic-stack-ca.p12
-rw------- 1 root elasticsearch 2672 Mar 27 19:46 /etc/elasticsearch/elastic-stack-ca.p1
2.基于自建ca证书生成ES证书
root@elk91:~# /usr/share/elasticsearch/bin/elasticsearch-certutil cert --ca /etc/elasticsearch/elastic-stack-ca.p12 --out /etc/elasticsearch/elastic-certificates-https.p12 --pass "" --days 3650 --ca-pass ""
#生成如下文件
root@elk91:~# ll /etc/elasticsearch/elastic-stack-ca.p12
-rw------- 1 root elasticsearch 2672 Mar 27 19:46 /etc/elasticsearch/elastic-stack-ca.p12
root@elk91:~# ll /etc/elasticsearch/elastic-certificates-https.p12
-rw------- 1 root elasticsearch 3596 Mar 27 19:47 /etc/elasticsearch/elastic-certificates-https.p12
3.修改配置文件
root@elk91:~# egrep -v "^#|^$" /etc/elasticsearch/elasticsearch.yml
cluster.name: dezyan-linux96
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
network.host: 0.0.0.0
http.port: 9200
discovery.seed_hosts: ["10.0.0.91", "10.0.0.92","10.0.0.93"]
cluster.initial_master_nodes: ["10.0.0.91","10.0.0.92","10.0.0.93"]
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: elastic-certificates.p12
#添加如下配置
xpack.security.authc.api_key.enabled: true
xpack.security.authc.api_key.hashing.algorithm: pbkdf2
xpack.security.authc.api_key.cache.ttl: 1d
xpack.security.authc.api_key.cache.max_keys: 10000
xpack.security.authc.api_key.cache.hash_algo: ssha256
xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.keystore.path: elastic-certificates-https.p12
4.同步配置文件集群的其他节点
root@elk91:~# chmod 640 /etc/elasticsearch/elastic-certificates-https.p12
root@elk91:~# scp -p /etc/elasticsearch/elastic{-certificates-https.p12,search.yml} 10.0.0.92:/etc/elasticsearch/
root@elk91:~# scp -p /etc/elasticsearch/elastic{-certificates-https.p12,search.yml} 10.0.0.93:/etc/elasticsearch/
5.重启ES集群
root@elk91:~# systemctl restart elasticsearch.service
root@elk92:~# systemctl restart elasticsearch.service
root@elk93:~# systemctl restart elasticsearch.service
6.测试验证，使用https协议
root@elk91:~# curl https://10.0.0.91:9200/_cat/nodes -u elastic:123456 -k
10.0.0.92 12 97 91 0.81 0.46 0.46 cdfhilmrstw - elk92
10.0.0.91 21 97 62 0.97 0.32 0.22 cdfhilmrstw - elk91
10.0.0.93 24 96 66 0.60 0.18 0.12 cdfhilmrstw * elk93
7.修改kibana的配置跳过自建证书校验
root@elk91:~# egrep -v "^#|^$" /etc/kibana/kibana.yml
server.port: 5601
server.host: "0.0.0.0"
#修改指向ES集群的地址协议为https
elasticsearch.hosts: ["https://10.0.0.91:9200","https://10.0.0.92:9200","https://10.0.0.93:9200"]
elasticsearch.username: "kibana_system"
elasticsearch.password: "u8SXpb5TM97itEEngA5U"
#跳过证书校验
elasticsearch.ssl.verificationMode: none
i18n.locale: "zh-CN"
root@elk91:~# systemctl restart kibana.service
root@elk91:~# ss -ntl | grep 5601
LISTEN 0 511 0.0.0.0:5601 0.0.0.0:*
Filebeat的安装
1.下载Filebeat
root@elk92:~# wget https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-7.17.28-amd64.deb
2.安装Filebeat
root@elk92:~# dpkg -i filebeat-7.17.28-amd64.deb
Selecting previously unselected package filebeat.
(Reading database ... 111432 files and directories currently installed.)
Preparing to unpack filebeat-7.17.28-amd64.deb ...
Unpacking filebeat (7.17.28) ...
Setting up filebeat (7.17.28) ...
Logstash的安装
1.下载Logstash
root@elk91:~# wget https://artifacts.elastic.co/downloads/logstash/logstash-7.17.28-amd64.deb
2.安装Logstash
root@elk91:~# ll -h logstash-7.17.28-amd64.deb
-rw-r--r-- 1 root root 359M Mar 13 14:41 logstash-7.17.28-amd64.deb
root@elk91:~# dpkg -i logstash-7.17.28-amd64.deb
Selecting previously unselected package logstash.
(Reading database ... 157651 files and directories currently installed.)
Preparing to unpack logstash-7.17.28-amd64.deb ...
Unpacking logstash (1:7.17.28-1) ...
Setting up logstash (1:7.17.28-1) ...
Using bundled JDK: /usr/share/logstash/jdk
Using provided startup.options file: /etc/logstash/startup.options
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/pleaserun-0.0.32/lib/pleaserun/platform/base.rb:112: warning: constant ::Fixnum is deprecated
Successfully created system startup script for Logstash
3.创建符号链接，将Logstash命令添加到PATH环境变量
root@elk91:~# ln -svf /usr/share/logstash/bin/logstash /usr/local/bin/
'/usr/local/bin/logstash' -> '/usr/share/logstash/bin/logstash'
本文来自博客园，作者：丁志岩，转载请注明原文链接：https://www.cnblogs.com/dezyan/p/18797051
浙公网安备 33010602011771号