ansible安装es集群
环境准备
三个节点,每个节点都有一个普通用户admin
并且admin用户执行sudo的时候无需输入密码 visudo
其中一个主控节点通过ssh能够免密登录到其他两台节点
当前部署的es集群分为三个角色base,elastic-master和elasitc
ansible配置
执行ansible命令的时候使用的是root权限 [privilege_escalation] become=True become_method=sudo become_user=root become_ask_pass=False
[elastic-master] 192.168.30.174 [elastic] 192.168.30.174 192.168.30.175 192.168.30.176
- hosts: all gather_facts: false roles: - { role: base, tags: 'base' } - hosts: elastic-master gather_facts: true roles: - { role: elastic-master, tags: 'elastic-master' } - hosts: elastic gather_facts: true roles: - { role: elastic, tags: 'elastic' }
base角色配置
######nfs config###### nfs_dir: /data nfs_share_ip: 192.168.30.0/24 taishi_user: admin taishi_userid: 1000 nfs_client_dir: /home/admin/nfsdata #######base install config##### taishi_dir: /app/taishi #######elastic install config#### es_password: Transfar@2022 es_path_data: /home/admin/es-cluster/data es_path_logs: /home/admin/es-cluster/logs
- name: Install ntpdate yum: name=ntpdate state=present tags: ntp - name: ntpdate server time shell: > ntpdate time1.aliyun.com tags: ntp - name: set the time zone shell: > ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime tags: ntp - name: Disabled Selinux Server selinux: state=disabled tags: firwalld - name: Change file limit pam_limits: domain: "*" limit_type: "{{ item.limit_type }}" limit_item: "{{ item.limit_item }}" value: 65535 with_items: - { limit_type: 'soft', limit_item: 'nofile' } - { limit_type: 'hard', limit_item: 'nofile' } tags: limit - name: Create the timeserver crontab cron: name: "ntp time1.aliyun.com" minute: "1" hour: "1" job: "/usr/sbin/ntpdate time1.aliyun.com" tags: cron - name: Create taishi dir file: path={{ taishi_dir }} state=directory owner={{ taishi_user }} group={{ taishi_user }} recurse=yes tags: jdk - name: visit old_jdk shell: rpm -qa | grep jdk register: jdk_result ignore_errors: True tags: jdk - name: visit old_java shell: rpm -qa | grep java register: java_result ignore_errors: True tags: jdk - name: uninstall old_jdk shell: rpm -qa | grep jdk | xargs rpm -e --nodeps when: jdk_result is succeeded tags: jdk - name: uninstall old_java shell: rpm -qa | grep java | xargs rpm -e --nodeps when: jdk_result is succeeded - name: Create new jdk dir file: path="{{ taishi_dir }}/jdk" state=directory owner={{ taishi_user }} group={{ taishi_user }} recurse=yes tags: jdk - name: Copy jdkpackage to hosts copy: src=jdk-8u60-linux-x64.tar.gz dest=/tmp tags: jdk - name: Install new jdk1.8 for hosts unarchive: src="/tmp/jdk-8u60-linux-x64.tar.gz" dest="{{ taishi_dir }}/jdk" copy=no mode=0755 tags: jdk - name: "chown jdk dir to {{ taishi_user }}" file: path="{{ taishi_dir }}/jdk" state=directory owner={{ taishi_user }} group={{ taishi_user }} recurse=yes tags: jdk - name: set env lineinfile: dest=/etc/profile insertafter="{{item.position}}" line="{{item.value}}" state=present with_items: - {position: EOF, value: "export JAVA_HOME={{ taishi_dir }}/jdk/jdk1.8.0_60/"} - {position: EOF, value: "export PATH=$JAVA_HOME/bin:$PATH"} tags: jdk - name: set esmaster hosts lineinfile: dest=/etc/hosts insertafter="{{item.position}}" line="{{item.value}}" state=present with_items: - {position: EOF, value: "{{ groups['elastic-master'][0] }} instance"} tags: jdk - name: check java version with env shell: source /etc/profile && java -version tags: jdk - name: check java version with fullpath shell: "{{ taishi_dir }}/jdk/jdk1.8.0_60/bin/java -version" tags: jdk - name: Config sysctl file sysctl: name: vm.max_map_count value: 262144 sysctl_set: yes state: present reload: yes tags: elastic - name: Copy tcl to hosts copy: src=tcl-8.5.13-8.el7.x86_64.rpm dest=/tmp tags: esca - name: Install tcl for hosts shell: rpm -ivh /tmp/tcl-8.5.13-8.el7.x86_64.rpm ignore_errors: True tags: esca - name: Copy expect to hosts copy: src=expect-5.45-14.el7_1.x86_64.rpm dest=/tmp tags: esca - name: Install expect for hosts shell: rpm -ivh /tmp/expect-5.45-14.el7_1.x86_64.rpm ignore_errors: True tags: esca
elastic-maste角色配置
- name: Create elastic dir file: path="{{ taishi_dir }}/elastic" state=directory owner={{ taishi_user }} group={{ taishi_user }} recurse=yes tags: elastic-master - name: Copy elastic to host copy: src=../../common/packages/elastic/elasticsearch-7.8.1-linux-x86_64.tar.gz dest=/tmp tags: elastic-master - name: Install elastic for host unarchive: src="/tmp/elasticsearch-7.8.1-linux-x86_64.tar.gz" dest="{{ taishi_dir }}/elastic" copy=no mode=0755 tags: elastic-master - name: "chown elastic dir to {{ taishi_user }}" file: path="{{ taishi_dir }}/elastic" state=directory owner={{ taishi_user }} group={{ taishi_user }} recurse=yes tags: elastic-master - name: "Copy the elasticsearch.yml" template: src=elasticsearch.yml.j2 dest="{{ taishi_dir }}/elastic/elasticsearch-7.8.1/config/elasticsearch.yml" owner={{ taishi_user }} group={{ taishi_user }} mode=0755 tags: startes - name: "copy the esca sh" template: src=setesca.sh.j2 dest="{{ taishi_dir }}/elastic/elasticsearch-7.8.1/setesca.sh" owner={{ taishi_user }} group={{ taishi_user }} mode=0755 tags: esca - name: "set the es ca" shell: "{{ taishi_dir }}/elastic/elasticsearch-7.8.1/setesca.sh" ignore_errors: True tags: esca - name: "copy the pemcrt sh" template: src=pemcrt.sh.j2 dest="{{ taishi_dir }}/elastic/elasticsearch-7.8.1/pemcrt.sh" owner={{ taishi_user }} group={{ taishi_user }} mode=0755 tags: esca - name: "set the pemcrt" shell: "{{ taishi_dir }}/elastic/elasticsearch-7.8.1/pemcrt.sh" ignore_errors: True tags: esca - name: "unzip the certificate" shell: "cd {{ taishi_dir }}/elastic/elasticsearch-7.8.1/ && mkdir -p {{ taishi_dir }}/elastic/elasticsearch-7.8.1/config/certs/ && rm -fr {{ taishi_dir }}/elastic/elasticsearch-7.8.1/config/certs/* && cp certificate-bundle.zip {{ taishi_dir }}/elastic/elasticsearch-7.8.1/config/certs/ && cd {{ taishi_dir }}/elastic/elasticsearch-7.8.1/config/certs/ && unzip certificate-bundle.zip" tags: esca - name: "copy elastic-certificates.p12 to certs" shell: "cd {{ taishi_dir }}/elastic/elasticsearch-7.8.1/ && cp elastic-certificates.p12 {{ taishi_dir }}/elastic/elasticsearch-7.8.1/config/certs/" tags: esca - name: "chown elastic dir to {{ taishi_user }}" file: path="{{ taishi_dir }}/elastic" state=directory owner={{ taishi_user }} group={{ taishi_user }} recurse=yes tags: esca - name: "Create es logsfile" file: path="{{ es_path_logs }}/es-cluster.log" state=file owner={{ taishi_user }} group={{ taishi_user }} tags: startes - name: "Start elastic cluster" shell: "ulimit -n 65535 && ulimit -u 4096 && nohup su - {{ taishi_user }} -c {{ taishi_dir }}/elastic/elasticsearch-7.8.1/bin/elasticsearch &" tags: startes - name: "check es cluster status" wait_for: port: 9200 delay: 10 timeout: 300 tags: startes - name: "copy the user password sh" template: src=setpasswd.sh.j2 dest="{{ taishi_dir }}/elastic/setpasswd.sh" owner={{ taishi_user }} group={{ taishi_user }} mode=0755 tags: espasswd - name: "set the user password" shell: "{{ taishi_dir }}/elastic/setpasswd.sh" register: result failed_when: result.stdout.find('ERROR') != -1 ignore_errors: True tags: espasswd - name: "get the es cluster" shell: "netstat -nlp | grep 9200 |awk '{print $7}' | awk -F'/' '{ print $1 }'" register: result tags: stopesmaster - name: "show es pid" debug: msg: "{{ result.stdout }}" tags: stopesmaster - name: "stop the es cluster" shell: "kill -9 {{ result.stdout }}" ignore_errors: True tags: stopes - name: "chown certificate dir to {{ taishi_user }}" file: path="/tmp/elastic" state=directory owner={{ taishi_user }} group={{ taishi_user }} recurse=yes tags: fetchzip - name: fetch certificate-bundle.zip fetch: src: "{{ taishi_dir }}/elastic/elasticsearch-7.8.1/certificate-bundle.zip" dest: /tmp/elastic/certificate-bundle.zip flat: yes tags: fetchzip
cluster.name: es-cluster node.name: node-{{ ansible_default_ipv4['address'].split('.')| last }} cluster.initial_master_nodes: node-{{ ansible_default_ipv4['address'].split('.')| last }} path.data: {{ es_path_data }} path.logs: {{ es_path_logs }} network.host: 0.0.0.0 {% set hosts = [] %} {% for host in groups['elastic-master'] %} {{ hosts.append(host) }} {% endfor %} discovery.seed_hosts: {{ hosts }} xpack.security.enabled: true xpack.security.http.ssl.enabled: true xpack.security.transport.ssl.enabled: true xpack.security.http.ssl.keystore.path: {{ taishi_dir }}/elastic/elasticsearch-7.8.1/config/certs/elastic-certificates.p12 xpack.security.transport.ssl.keystore.path: {{ taishi_dir }}/elastic/elasticsearch-7.8.1/config/certs/elastic-certificates.p12
#!/bin/bash expect <<EOF spawn {{ taishi_dir }}/elastic/elasticsearch-7.8.1/bin/elasticsearch-certutil ca expect { "output" { send "\n";exp_continue} "password" { send "\n";exp_continue} } expect eof EOF expect <<EOF spawn {{ taishi_dir }}/elastic/elasticsearch-7.8.1/bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12 expect { "password" { send "\n";exp_continue} "output" { send "\n";exp_continue} } expect eof EOF expect <<EOF spawn sudo openssl pkcs12 -in elastic-stack-ca.p12 -out newfile.crt.pem -clcerts -nokeys expect { "Password" { send "\n"} } expect eof EOF expect <<EOF spawn {{ taishi_dir }}/elastic/elasticsearch-7.8.1/bin/elasticsearch-certutil cert --pem elastic-stack-ca.p12 expect { "output" { send "\n"} } expect eof EOF
#!/bin/bash cd {{ taishi_dir }}/elastic/elasticsearch-7.8.1 expect <<EOF spawn sudo openssl pkcs12 -in {{ taishi_dir }}/elastic/elasticsearch-7.8.1/elastic-stack-ca.p12 -out newfile.crt.pem -clcerts -nokeys expect { "Password" { send "\n"} } expect eof EOF
#!/bin/bash passwd={{ es_password }} expect <<EOF spawn {{ taishi_dir }}/elastic/elasticsearch-7.8.1/bin/elasticsearch-setup-passwords interactive --batch --url https://instance:9200 expect { "y/N" { send "y\n";exp_continue} "elastic" { send "$passwd\n";exp_continue} "elastic" { send "$passwd\n";exp_continue} "apm_system" { send "$passwd\n";exp_continue} "apm_system" { send "$passwd\n";exp_continue} "kibana_system" { send "$passwd\n";exp_continue} "kibana_system" { send "$passwd\n";exp_continue} "logstash_system" { send "$passwd\n";exp_continue} "logstash_system" { send "$passwd\n";exp_continue} "beats_system" { send "$passwd\n";exp_continue} "beats_system" { send "$passwd\n";exp_continue} "remote_monitoring_user" { send "$passwd\n";exp_continue} "remote_monitoring_user" { send "$passwd\n"} } expect eof EOF
elastic角色配置
- name: Create elastic dir file: path="{{ taishi_dir }}/elastic" state=directory owner={{ taishi_user }} group={{ taishi_user }} recurse=yes tags: elastic - name: Copy elastic to host copy: src=../../common/packages/elastic/elasticsearch-7.8.1-linux-x86_64.tar.gz dest=/tmp tags: elastic - name: Install elastic for host unarchive: src="/tmp/elasticsearch-7.8.1-linux-x86_64.tar.gz" dest="{{ taishi_dir }}/elastic" copy=no mode=0755 tags: elastic - name: "chown elastic dir to {{ taishi_user }}" file: path="{{ taishi_dir }}/elastic" state=directory owner={{ taishi_user }} group={{ taishi_user }} recurse=yes tags: elastic - name: "Copy the elasticsearch.yml" template: src=elasticsearch.yml.j2 dest="{{ taishi_dir }}/elastic/elasticsearch-7.8.1/config/elasticsearch.yml" owner={{ taishi_user }} group={{ taishi_user }} mode=0755 tags: elastic - name: "rm the old certificate" shell: "rm -fr {{ taishi_dir }}/elastic/elasticsearch-7.8.1/config/certs/*" tags: elastic - name: "copy the certificate" copy: src=/tmp/elastic/certificate-bundle.zip dest={{ taishi_dir }}/elastic/elasticsearch-7.8.1/config/certs/ tags: elastic - name: "chown elastic dir to {{ taishi_user }}" file: path="{{ taishi_dir }}/elastic" state=directory owner={{ taishi_user }} group={{ taishi_user }} recurse=yes tags: elastic - name: "unzip the certificate" shell: "cd {{ taishi_dir }}/elastic/elasticsearch-7.8.1/config/certs/ && unzip certificate-bundle.zip" tags: elastic - name: "chown elastic dir to {{ taishi_user }}" file: path="{{ taishi_dir }}/elastic" state=directory owner={{ taishi_user }} group={{ taishi_user }} recurse=yes tags: startescluster - name: "Start elastic cluster" shell: "ulimit -n 65535 && ulimit -u 4096 && nohup su - {{ taishi_user }} -c {{ taishi_dir }}/elastic/elasticsearch-7.8.1/bin/elasticsearch &" tags: startescluster - name: "check es cluster status" wait_for: port: 9200 delay: 10 timeout: 300 tags: startescluster - name: "get the es cluster" shell: "netstat -nlp | grep 9200 |awk '{print $7}' | awk -F'/' '{ print $1 }'" register: result tags: startescluster - name: "show es pid" debug: msg: "{{ result.stdout }}" tags: startescluster
# ======================== Elasticsearch Configuration ========================= # # NOTE: Elasticsearch comes with reasonable defaults for most settings. # Before you set out to tweak and tune the configuration, make sure you # understand what are you trying to accomplish and the consequences. # # The primary way of configuring a node is via this file. This template lists # the most important settings you may want to configure for a production cluster. # # Please consult the documentation for further information on configuration options: # https://www.elastic.co/guide/en/elasticsearch/reference/index.html # # ---------------------------------- Cluster ----------------------------------- # # Use a descriptive name for your cluster: # cluster.name: es-cluster # # ------------------------------------ Node ------------------------------------ # # Use a descriptive name for the node: # node.name: node-{{ ansible_default_ipv4['address'].split('.')| last }} # # Add custom attributes to the node: # #node.attr.rack: r1 # # ----------------------------------- Paths ------------------------------------ # # Path to directory where to store the data (separate multiple locations by comma): # path.data: {{ es_path_data }} # # Path to log files: # path.logs: {{ es_path_logs }} # # ----------------------------------- Memory ----------------------------------- # # Lock the memory on startup: # #bootstrap.memory_lock: true # # Make sure that the heap size is set to about half the memory available # on the system and that the owner of the process is allowed to use this # limit. # # Elasticsearch performs poorly when the system is swapping the memory. # # ---------------------------------- Network ----------------------------------- # # Set the bind address to a specific IP (IPv4 or IPv6): # network.host: 0.0.0.0 # # Set a custom port for HTTP: # #http.port: 9200 # # For more information, consult the network module documentation. # # --------------------------------- Discovery ---------------------------------- # # Pass an initial list of hosts to perform discovery when this node is started: # The default list of hosts is ["127.0.0.1", "[::1]"] # {% set hosts = [] %} {% for host in groups['elastic'] %} {{ hosts.append(host) }} {% endfor %} discovery.seed_hosts: {{ hosts }} # # Bootstrap the cluster using an initial set of master-eligible nodes: # cluster.initial_master_nodes: ["node-{{ groups['elastic-master'][0].split('.')| last }}"] # # For more information, consult the discovery and cluster formation module documentation. # # ---------------------------------- Gateway ----------------------------------- # # Block initial recovery after a full cluster restart until N nodes are started: # #gateway.recover_after_nodes: 3 # # For more information, consult the gateway module documentation. # # ---------------------------------- Various ----------------------------------- # # Require explicit names when deleting indices: # #action.destructive_requires_name: true xpack.security.enabled: true xpack.security.http.ssl.enabled: true xpack.security.transport.ssl.enabled: true xpack.license.self_generated.type: basic xpack.security.http.ssl.key: {{ taishi_dir }}/elastic/elasticsearch-7.8.1/config/certs/instance/instance.key xpack.security.http.ssl.certificate: {{ taishi_dir }}/elastic/elasticsearch-7.8.1/config/certs/instance/instance.crt xpack.security.http.ssl.certificate_authorities: {{ taishi_dir }}/elastic/elasticsearch-7.8.1/config/certs/ca/ca.crt xpack.security.transport.ssl.verification_mode: certificate xpack.security.transport.ssl.key: {{ taishi_dir }}/elastic/elasticsearch-7.8.1/config/certs/instance/instance.key xpack.security.transport.ssl.certificate: {{ taishi_dir }}/elastic/elasticsearch-7.8.1/config/certs/instance/instance.crt xpack.security.transport.ssl.certificate_authorities: {{ taishi_dir }}/elastic/elasticsearch-7.8.1/config/certs/ca/ca.crt
部署结果验证
问题总结
ansible-playbook -i hosts --tags fetchzip site.yml
ansible-playbook -i hosts --tags elastic site.yml
ansible-playbook -i hosts --tags elastic-master site.yml
ulimit -n 65535 && ulimit -u 4096 && su - admin -c /app/taishi/elastic/elasticsearch-7.8.1/bin/elasticsearch
curl -u elastic:Transfar@2022 --insecure https://instance:9200/_cat/indices
curl -u elastic:Transfar@123 --insecure -XPUT -H "Content-Type: application/json" https://192.168.30.102:9200/_all/_settings -d '{"index.blocks.read_only_allow_delete": null}'
rm -fr /home/admin/es-cluster/data/* //删除es原来的垃圾数据
ansible执行shell脚本的时候,command输出显示红色异常,但其实执行shell脚本成功
es重新设置用户名密码的时候需要先删除原来的信息,原来的密码信息在设置的时候会自动写入一个索引中,所以需要先把这个索引先删除
curl -XDELETE -u elastic:Transfar@2022 http://instance:9200/.security-7
curl -XDELETE --insecure -u elastic:Transfar@2022 https://instance:9200/.security-7
安装部署完毕
单节点的ES数据备份和恢复
把整个elasticsearch的安装目录进行备份,然后把用备份的整个目录替换整个新的elasticsearch目录
本地 验证 elastic账户
curl --insecure -u elastic 'https://172.17.1.3:9200/_xpack/security/_authenticate?pretty'
修改用户密码
curl --insecure -u kibana_system:Transfar@123 -XPUT 'https://172.17.1.3:9200/_xpack/security/user/elastic/_password?pretty' -H 'Content-Type: application/json' -d'
{
"password" : "Transfar@2022"
}
'
索引恢复完成
ES集群启动问题
es节点启动出错
client did not trust this server's certificate, closing connection Netty4TcpChannel{localAddress=0.0.0.0/0.0.0.0:9300, remoteAddress=/172.16.3.137:54781}
xpack.security.transport.ssl.verification_mode: certificate
修改成
xpack.security.transport.ssl.verification_mode: none
es集群配置文件要求
1.修改配置文件. 一定要按照配置文件模板来配置集群,否则集群会启动失败
# 增加以下内容 # 集群名称必须相同 cluster.name: es-yourname node.name: node-num # 当前节点是否可以被选举为master节点,是:true、否:false node.master: true # 当前节点是否用于存储数据,是:true、否:false node.data: true path.data: /data/es/data path.logs: /data/es/logs # 需求锁住物理内存,是:true、否:false bootstrap.memory_lock: false # SecComp检测,是:true、否:false bootstrap.system_call_filter: false network.host: 0.0.0.0 # 集群必须要配置此项,yourIp修改为自己的本机服务器的外网ip,否则集群失败 network.publish_host: yourIp # 主机访问的端口号 http.port: 9200 # es7.x 之后新增的配置,写入候选主节点的设备地址,在开启服务后可以被选为主节点 # es7之后,不需要discover.zen.ping.unicast.hosts这个参数,用discovery.seed_hosts替换 discovery.seed_hosts: ["10.10.10.1","10.10.10.2","10.10.10.3"] # es7.x 之后新增的配置,初始化一个新的集群时需要此配置来选举master cluster.initial_master_nodes:["10.10.10.1","10.10.10.2","10.10.10.3"] # 是否支持跨域,是:true,在使用head插件时需要此配置 http.cors.enabled: true # "*" 表示支持所有域名 http.cors.allow-origin: "*"
# ======================== Elasticsearch Configuration ========================= # # NOTE: Elasticsearch comes with reasonable defaults for most settings. # Before you set out to tweak and tune the configuration, make sure you # understand what are you trying to accomplish and the consequences. # # The primary way of configuring a node is via this file. This template lists # the most important settings you may want to configure for a production cluster. # # Please consult the documentation for further information on configuration options: # https://www.elastic.co/guide/en/elasticsearch/reference/index.html # # ---------------------------------- Cluster ----------------------------------- # # Use a descriptive name for your cluster: # cluster.name: {{ es_cluster_name }} # # ------------------------------------ Node ------------------------------------ # # Use a descriptive name for the node: # node.name: node-{{ ansible_default_ipv4['address'].split('.')| last }} # # Add custom attributes to the node: # #node.attr.rack: r1 # # ----------------------------------- Paths ------------------------------------ # # Path to directory where to store the data (separate multiple locations by comma): # path.data: {{ es_path_data }} # # Path to log files: # path.logs: {{ es_path_logs }} # # ----------------------------------- Memory ----------------------------------- # # Lock the memory on startup: # #bootstrap.memory_lock: true # # Make sure that the heap size is set to about half the memory available # on the system and that the owner of the process is allowed to use this # limit. # # Elasticsearch performs poorly when the system is swapping the memory. # # ---------------------------------- Network ----------------------------------- # # Set the bind address to a specific IP (IPv4 or IPv6): # network.host: {{ ansible_default_ipv4.address }} network.publish_host: {{ ansible_default_ipv4.address }} # # Set a custom port for HTTP: # http.port: 9200 # # For more information, consult the network module documentation. # # --------------------------------- Discovery ---------------------------------- # # Pass an initial list of hosts to perform discovery when this node is started: # The default list of hosts is ["127.0.0.1", "[::1]"] # {% set hosts = [] %} {% for host in groups['elastic'] %} {{ hosts.append(host) }} {% endfor %} discovery.seed_hosts: {{ hosts | to_json }} # # Bootstrap the cluster using an initial set of master-eligible nodes: # cluster.initial_master_nodes: ["node-{{ groups['elastic-master'][0].split('.')| last }}"] # # For more information, consult the discovery and cluster formation module documentation. # # ---------------------------------- Gateway ----------------------------------- # # Block initial recovery after a full cluster restart until N nodes are started: # #gateway.recover_after_nodes: 3 # # For more information, consult the gateway module documentation. # # ---------------------------------- Various ----------------------------------- # # Require explicit names when deleting indices: # #action.destructive_requires_name: true xpack.security.enabled: true xpack.security.http.ssl.enabled: true xpack.security.transport.ssl.enabled: true xpack.license.self_generated.type: basic xpack.security.http.ssl.key: {{ taishi_dir }}/elastic/elasticsearch-7.8.1/config/certs/instance/instance.key xpack.security.http.ssl.certificate: {{ taishi_dir }}/elastic/elasticsearch-7.8.1/config/certs/instance/instance.crt xpack.security.http.ssl.certificate_authorities: {{ taishi_dir }}/elastic/elasticsearch-7.8.1/config/certs/ca/ca.crt xpack.security.transport.ssl.verification_mode: none xpack.security.transport.ssl.key: {{ taishi_dir }}/elastic/elasticsearch-7.8.1/config/certs/instance/instance.key xpack.security.transport.ssl.certificate: {{ taishi_dir }}/elastic/elasticsearch-7.8.1/config/certs/instance/instance.crt xpack.security.transport.ssl.certificate_authorities: {{ taishi_dir }}/elastic/elasticsearch-7.8.1/config/certs/ca/ca.crt
# ======================== Elasticsearch Configuration ========================= # # NOTE: Elasticsearch comes with reasonable defaults for most settings. # Before you set out to tweak and tune the configuration, make sure you # understand what are you trying to accomplish and the consequences. # # The primary way of configuring a node is via this file. This template lists # the most important settings you may want to configure for a production cluster. # # Please consult the documentation for further information on configuration options: # https://www.elastic.co/guide/en/elasticsearch/reference/index.html # # ---------------------------------- Cluster ----------------------------------- # # Use a descriptive name for your cluster: # cluster.name: taishiescluster-174 # # ------------------------------------ Node ------------------------------------ # # Use a descriptive name for the node: # node.name: node-174 # # Add custom attributes to the node: # #node.attr.rack: r1 # # ----------------------------------- Paths ------------------------------------ # # Path to directory where to store the data (separate multiple locations by comma): # path.data: /home/admin/es-cluster/data # # Path to log files: # path.logs: /home/admin/es-cluster/logs # # ----------------------------------- Memory ----------------------------------- # # Lock the memory on startup: # #bootstrap.memory_lock: true # # Make sure that the heap size is set to about half the memory available # on the system and that the owner of the process is allowed to use this # limit. # # Elasticsearch performs poorly when the system is swapping the memory. # # ---------------------------------- Network ----------------------------------- # # Set the bind address to a specific IP (IPv4 or IPv6): # network.host: 192.168.30.174 network.publish_host: 192.168.30.174 # # Set a custom port for HTTP: # http.port: 9200 # # For more information, consult the network module documentation. # # --------------------------------- Discovery ---------------------------------- # # Pass an initial list of hosts to perform discovery when this node is started: # The default list of hosts is ["127.0.0.1", "[::1]"] # discovery.seed_hosts: ["192.168.30.174", "192.168.30.175", "192.168.30.176"] # # Bootstrap the cluster using an initial set of master-eligible nodes: # cluster.initial_master_nodes: ["node-174"] # # For more information, consult the discovery and cluster formation module documentation. # # ---------------------------------- Gateway ----------------------------------- # # Block initial recovery after a full cluster restart until N nodes are started: # #gateway.recover_after_nodes: 3 # # For more information, consult the gateway module documentation. # # ---------------------------------- Various ----------------------------------- # # Require explicit names when deleting indices: # #action.destructive_requires_name: true xpack.security.enabled: true xpack.security.http.ssl.enabled: true xpack.security.transport.ssl.enabled: true xpack.license.self_generated.type: basic xpack.security.http.ssl.key: /app/taishi/elastic/elasticsearch-7.8.1/config/certs/instance/instance.key xpack.security.http.ssl.certificate: /app/taishi/elastic/elasticsearch-7.8.1/config/certs/instance/instance.crt xpack.security.http.ssl.certificate_authorities: /app/taishi/elastic/elasticsearch-7.8.1/config/certs/ca/ca.crt xpack.security.transport.ssl.verification_mode: none xpack.security.transport.ssl.key: /app/taishi/elastic/elasticsearch-7.8.1/config/certs/instance/instance.key xpack.security.transport.ssl.certificate: /app/taishi/elastic/elasticsearch-7.8.1/config/certs/instance/instance.crt xpack.security.transport.ssl.certificate_authorities: /app/taishi/elastic/elasticsearch-7.8.1/config/certs/ca/ca.crt
# ======================== Elasticsearch Configuration ========================= # # NOTE: Elasticsearch comes with reasonable defaults for most settings. # Before you set out to tweak and tune the configuration, make sure you # understand what are you trying to accomplish and the consequences. # # The primary way of configuring a node is via this file. This template lists # the most important settings you may want to configure for a production cluster. # # Please consult the documentation for further information on configuration options: # https://www.elastic.co/guide/en/elasticsearch/reference/index.html # # ---------------------------------- Cluster ----------------------------------- # # Use a descriptive name for your cluster: # cluster.name: taishiescluster-174 # # ------------------------------------ Node ------------------------------------ # # Use a descriptive name for the node: # node.name: node-175 # # Add custom attributes to the node: # #node.attr.rack: r1 # # ----------------------------------- Paths ------------------------------------ # # Path to directory where to store the data (separate multiple locations by comma): # path.data: /home/admin/es-cluster/data # # Path to log files: # path.logs: /home/admin/es-cluster/logs # # ----------------------------------- Memory ----------------------------------- # # Lock the memory on startup: # #bootstrap.memory_lock: true # # Make sure that the heap size is set to about half the memory available # on the system and that the owner of the process is allowed to use this # limit. # # Elasticsearch performs poorly when the system is swapping the memory. # # ---------------------------------- Network ----------------------------------- # # Set the bind address to a specific IP (IPv4 or IPv6): # network.host: 192.168.30.175 network.publish_host: 192.168.30.175 # # Set a custom port for HTTP: # http.port: 9200 # # For more information, consult the network module documentation. # # --------------------------------- Discovery ---------------------------------- # # Pass an initial list of hosts to perform discovery when this node is started: # The default list of hosts is ["127.0.0.1", "[::1]"] # discovery.seed_hosts: ["192.168.30.174", "192.168.30.175", "192.168.30.176"] # # Bootstrap the cluster using an initial set of master-eligible nodes: # cluster.initial_master_nodes: ["node-174"] # # For more information, consult the discovery and cluster formation module documentation. # # ---------------------------------- Gateway ----------------------------------- # # Block initial recovery after a full cluster restart until N nodes are started: # #gateway.recover_after_nodes: 3 # # For more information, consult the gateway module documentation. # # ---------------------------------- Various ----------------------------------- # # Require explicit names when deleting indices: # #action.destructive_requires_name: true xpack.security.enabled: true xpack.security.http.ssl.enabled: true xpack.security.transport.ssl.enabled: true xpack.license.self_generated.type: basic xpack.security.http.ssl.key: /app/taishi/elastic/elasticsearch-7.8.1/config/certs/instance/instance.key xpack.security.http.ssl.certificate: /app/taishi/elastic/elasticsearch-7.8.1/config/certs/instance/instance.crt xpack.security.http.ssl.certificate_authorities: /app/taishi/elastic/elasticsearch-7.8.1/config/certs/ca/ca.crt xpack.security.transport.ssl.verification_mode: none xpack.security.transport.ssl.key: /app/taishi/elastic/elasticsearch-7.8.1/config/certs/instance/instance.key xpack.security.transport.ssl.certificate: /app/taishi/elastic/elasticsearch-7.8.1/config/certs/instance/instance.crt xpack.security.transport.ssl.certificate_authorities: /app/taishi/elastic/elasticsearch-7.8.1/config/certs/ca/ca.crt
# ======================== Elasticsearch Configuration ========================= # # NOTE: Elasticsearch comes with reasonable defaults for most settings. # Before you set out to tweak and tune the configuration, make sure you # understand what are you trying to accomplish and the consequences. # # The primary way of configuring a node is via this file. This template lists # the most important settings you may want to configure for a production cluster. # # Please consult the documentation for further information on configuration options: # https://www.elastic.co/guide/en/elasticsearch/reference/index.html # # ---------------------------------- Cluster ----------------------------------- # # Use a descriptive name for your cluster: # cluster.name: taishiescluster-174 # # ------------------------------------ Node ------------------------------------ # # Use a descriptive name for the node: # node.name: node-176 # # Add custom attributes to the node: # #node.attr.rack: r1 # # ----------------------------------- Paths ------------------------------------ # # Path to directory where to store the data (separate multiple locations by comma): # path.data: /home/admin/es-cluster/data # # Path to log files: # path.logs: /home/admin/es-cluster/logs # # ----------------------------------- Memory ----------------------------------- # # Lock the memory on startup: # #bootstrap.memory_lock: true # # Make sure that the heap size is set to about half the memory available # on the system and that the owner of the process is allowed to use this # limit. # # Elasticsearch performs poorly when the system is swapping the memory. # # ---------------------------------- Network ----------------------------------- # # Set the bind address to a specific IP (IPv4 or IPv6): # network.host: 192.168.30.176 network.publish_host: 192.168.30.176 # # Set a custom port for HTTP: # http.port: 9200 # # For more information, consult the network module documentation. # # --------------------------------- Discovery ---------------------------------- # # Pass an initial list of hosts to perform discovery when this node is started: # The default list of hosts is ["127.0.0.1", "[::1]"] # discovery.seed_hosts: ["192.168.30.174", "192.168.30.175", "192.168.30.176"] # # Bootstrap the cluster using an initial set of master-eligible nodes: # cluster.initial_master_nodes: ["node-174"] # # For more information, consult the discovery and cluster formation module documentation. # # ---------------------------------- Gateway ----------------------------------- # # Block initial recovery after a full cluster restart until N nodes are started: # #gateway.recover_after_nodes: 3 # # For more information, consult the gateway module documentation. # # ---------------------------------- Various ----------------------------------- # # Require explicit names when deleting indices: # #action.destructive_requires_name: true xpack.security.enabled: true xpack.security.http.ssl.enabled: true xpack.security.transport.ssl.enabled: true xpack.license.self_generated.type: basic xpack.security.http.ssl.key: /app/taishi/elastic/elasticsearch-7.8.1/config/certs/instance/instance.key xpack.security.http.ssl.certificate: /app/taishi/elastic/elasticsearch-7.8.1/config/certs/instance/instance.crt xpack.security.http.ssl.certificate_authorities: /app/taishi/elastic/elasticsearch-7.8.1/config/certs/ca/ca.crt xpack.security.transport.ssl.verification_mode: none xpack.security.transport.ssl.key: /app/taishi/elastic/elasticsearch-7.8.1/config/certs/instance/instance.key xpack.security.transport.ssl.certificate: /app/taishi/elastic/elasticsearch-7.8.1/config/certs/instance/instance.crt xpack.security.transport.ssl.certificate_authorities: /app/taishi/elastic/elasticsearch-7.8.1/config/certs/ca/ca.crt
2.修改内存参数
vim config/jvm.options
# 按需修改如下内存大小即可
-Xms4g
-Xmx4g
3.启动
在三台服务器上分别启动es,没有先后之分,7版本会自动选取主节点的
4.验证
https://192.168.30.174:9200/_cat/nodes
5.主从节点连接异常
集群成功启动
superivisor启动服务集群注意点
当superivisor的配置目录下的服务.ini文件没有发生变化的时候,使用superivosrctl update是不对相应服务进行启停操作的.supervisorctl update只能适应第一次启动服务或者服务的supervisor配置文件.ini有变化
卸载es集群的时候由于没有删除supervisor配置目录下的ini配置文件,再次单独es集群或者其它单独服务的时候supervisor是不会启动相关服务的。这个时候必须使用supervisorctl restart elasticsearch而不能用supervisorctl update
服务正常启动
异常排查实例
1.ERROR: Elasticsearch keystore file is missing
1.生成keystore文件 ./bin/elasticsearch-keystore create
2.修改文件权限
2.ansible "msg": "Missing sudo password"
添加配置sudo名无需输入密码
vi /etc/sudoers
3.缺少相关目录
4.ansible主机文件配置用户名和密码
本文来自博客园,作者:不懂123,转载请注明原文链接:https://www.cnblogs.com/yxh168/p/14825249.html