elk集群的docker搭建

elk的工作模式：

集群搭建

docker版本：

Docker version 20.10.14, build a224086

 

dokcer拉去镜像的命令：

docker pull kibana:7.17.3
docker pull logstash:7.17.2
docker pull elasticsearch:7.17.1

部署的主目录

/Users/liyabing/elasticsearch/v7.17/　

 

 创建对应的目录：

# 主目录在

  /Users/liyabing/elasticsearch/v7.17/

切换到主目录下
cd /Users/liyabing/elasticsearch/v7.17/

# 配置文件
sudo mkdir -p node-1/config
sudo mkdir -p node-2/config
sudo mkdir -p node-3/config

# 数据存储
sudo mkdir -p node-1/data
sudo mkdir -p node-2/data
sudo mkdir -p node-3/data

# 日志存储
sudo mkdir -p node-1/logs
sudo mkdir -p node-2/logs
sudo mkdir -p node-3/logs

# 插件管理
sudo mkdir -p node-1/plugins
sudo mkdir -p node-2/plugins
sudo mkdir -p node-3/plugins

# 开放权限
sudo chmod 777 /Users/liyabing/elasticsearch/v7.17/node-1/data
sudo chmod 777 /Users/liyabing/elasticsearch/v7.17/node-2/data
sudo chmod 777 /Users/liyabing/elasticsearch/v7.17/node-3/data

sudo chmod 777 /Users/liyabing/elasticsearch/v7.17/node-1/logs
sudo chmod 777 /Users/liyabing/elasticsearch/v7.17/node-2/logs
sudo chmod 777 /Users/liyabing/elasticsearch/v7.17/node-3/logs

elasticsearch配置文件编写

 我们以es01 为例，cd es01，增加elasticsearch-01.yml配置文件：

# es01.yml 根据自己环境修改ip地址
cluster.name: elasticsearch-cluster
node.name: es-node1
network.bind_host: 0.0.0.0
network.publish_host: 127.0.0.1
http.port: 9200
transport.tcp.port: 9300
http.cors.enabled: true
http.cors.allow-origin: "*"
node.master: true 
node.data: true  
discovery.zen.ping.unicast.hosts: ["127.0.0.1:9300","127.0.0.1:9301","127.0.0.1:9302"]
discovery.zen.minimum_master_nodes: 2

 

其他两个es配置文件类似：

# es02.yml 根据自己环境修改ip地址
cluster.name: elasticsearch-cluster
node.name: es-node2
network.bind_host: 0.0.0.0
network.publish_host: 127.0.0.1
http.port: 9201
transport.tcp.port: 9301
http.cors.enabled: true
http.cors.allow-origin: "*"
node.master: false 
node.data: true  
discovery.zen.ping.unicast.hosts: ["127.0.0.1:9300","127.0.0.1:9301","127.0.0.1:9302"]
discovery.zen.minimum_master_nodes: 2

# es03.yml 根据自己环境修改ip地址
cluster.name: elasticsearch-cluster
node.name: es-node3
network.bind_host: 0.0.0.0
network.publish_host: 127.0.0.1
http.port: 9202
transport.tcp.port: 9302
http.cors.enabled: true
http.cors.allow-origin: "*"
node.master: false 
node.data: true  
discovery.zen.ping.unicast.hosts: ["127.0.0.1:9300","127.0.0.1:9301","127.0.0.1:9302"]
discovery.zen.minimum_master_nodes: 2

 

配置参数说明：

cluster.name: 集群名称

node.name: 节点的名称
node.master: true  # 是不是有资格竞选主节点
node.data: true    # 是否存储数据
# 注意这两个路径不要配置物理机的路径了，这是【容器内部】的路径！！
path.data: /usr/share/elasticsearch/data # 数据存档位置
path.logs: /usr/share/elasticsearch/log   # 日志存放位置

# 配合network.publish_host 一起使用。参见下文的小窍门：
# 设置其它结点和该结点交互的ip地址，如果不设置它会自动判断，值必须是个真实的ip地址，设置当前物理机地址,如果是docker安装节点的IP将会是配置的IP而不是docker网管ip
# network.publish_host: 127.0.0.1
http.port: 9200  # 设置映射端口
transport.tcp.port: 9300  # 内部节点之间沟通端口 
# 组播地址
discovery.seed_hosts: ["127.0.0.1:9300","127.0.0.1:9300"]

由于默认es实例是1g，太吃内存了，我们修改一下jvm参数，每个es目录下放一个jvm.option文件：

-Xms128m

• -Xmx128m

• 修改jvm.options
• 创建es容器并启动

 

docker run -e ES_JAVA_OPTS="-Xms256m -Xmx256m" -d -p 9200:9200 -p 9300:9300 -p 5601:5601 -v /Users/liyabing/elasticsearch/v7.17/node-1/config/elasticsearch-01.yml:/usr/share/elasticsearch/config/elasticsearch.yml -v /Users/liyabing/elasticsearch/v7.17/node-1/plugins:/usr/share/elasticsearch/plugins -v /Users/liyabing/elasticsearch/v7.17/node-1/data:/usr/share/elasticsearch/data --name es01 elasticsearch:7.17.1

docker run -e ES_JAVA_OPTS="-Xms256m -Xmx256m" -d -p 9201:9201 -p 9301:9301 -v /Users/liyabing/elasticsearch/v7.17/node-2/config/elasticsearch-02.yml:/usr/share/elasticsearch/config/elasticsearch.yml -v /Users/liyabing/elasticsearch/v7.17/node-2/plugins:/usr/share/elasticsearch/plugins -v /Users/liyabing/elasticsearch/v7.17/node-2/data:/usr/share/elasticsearch/data --name es02 elasticsearch:7.17.1

docker run -e ES_JAVA_OPTS="-Xms256m -Xmx256m" -d -p 9202:9202 -p 9302:9302 -v /Users/liyabing/elasticsearch/v7.17/node-3/config/elasticsearch-03.yml:/usr/share/elasticsearch/config/elasticsearch.yml -v /Users/liyabing/elasticsearch/v7.17/node-3/plugins:/usr/share/elasticsearch/plugins -v /Users/liyabing/elasticsearch/v7.17/node-3/data:/usr/share/elasticsearch/data --name es03 elasticsearch:7.17.1