ansible role 案例3 二进制部署elasticsearch分布式集群
目录结构
[root@master-1 roles]# tree roles/es_cluster/ roles/es_cluster/ ├── defaults │ └── main.yml ├── files │ ├── elastic-certificates.p12 │ ├── elasticsearch-7.16.3-linux-x86_64.tar.gz │ ├── elasticsearch-analysis-ik-7.16.3.zip │ ├── elastic-stack-ca.p12 │ └── jdk-8u341-linux-x64.tar.gz ├── handlers │ └── main.yml ├── hosts │ └── inventory.yaml ├── meta │ └── main.yml ├── README.md ├── tasks │ ├── add_user.yml │ ├── check_process.yml │ ├── chown_dir.yml │ ├── copy_cert.yml │ ├── create_cert.yml │ ├── create_date_dir.yml │ ├── fetch_cert.yml │ ├── ik.yml │ ├── import_cert.yml │ ├── install_java.yml │ ├── keystore.yml │ ├── main.yml │ ├── set_es_jvm.yml │ ├── set_sysctl.yml │ ├── start.yml │ ├── template_es_config.yml │ └── unarchive_es.yml ├── templates │ └── elasticsearch.yml.j2 ├── tests │ ├── inventory │ └── test.yml └── vars └── main.yml 9 directories, 31 files
静态文件
[root@master-1 roles]# ll es_cluster/files/ 总用量 453132 -rw------- 1 root root 3596 2月 21 13:54 elastic-certificates.p12 -rw-r--r-- 1 root root 311327254 2月 21 13:03 elasticsearch-7.16.3-linux-x86_64.tar.gz -rw-r--r-- 1 root root 4504477 2月 21 13:03 elasticsearch-analysis-ik-7.16.3.zip -rw------- 1 root root 2672 2月 21 13:54 elastic-stack-ca.p12 -rw-r--r-- 1 root root 148162542 2月 21 13:25 jdk-8u341-linux-x64.tar.gz
变量文件
[root@master-1 roles]# cat roles/es_cluster/vars/main.yml --- # vars file for es_cluster JAVA_HOME: "/opt/jdk1.8.0_341" cluster_name: cluster listen_port: 9200 es_version: 7.16.3 es_install_dir: "/opt/elasticsearch-{{ es_version }}" password: "$1$6NXikaln$64e67B6Q2C0xK0fuNkVw30" es_ca_pass: 123456 es_cert_pass: 123456
对于ansible user模块，创建用户password 需要加密，以下是两种加密方式
echo "123456" |openssl passwd -1 -stdin python3 -c "import crypt; print(crypt.crypt('123456', crypt.mksalt(crypt.METHOD_MD5)))" $1$3r0I3UpX$lvFzoYQggE3WF/uwokQ.W0
也可以将涉密信息文件进行加密
ansible-vault create vars/main.yml
更多参考：
主机清单
[root@master-1 roles]# cat roles/es_cluster/hosts/inventory.yaml all: # Ansible 习惯使用 children 来定义组，而 all.hosts 一般只包含独立主机，所以这里设置为空 {} 让结构更清晰。 hosts: {} # 这里将下面hosts定义的单独主机用children权保部写到elasticsearch组。 children: elasticsearch: hosts: es-node1: ansible_host: 192.168.43.129 node_name: es-node1 # 用于when条件控制,生成证书 is_ca: true es-node2: ansible_host: 192.168.43.130 node_name: es-node2 es-node3: ansible_host: 192.168.43.131 node_name: es-node3
渲染文件
[root@master-1 roles]# cat roles/es_cluster/templates/elasticsearch.yml.j2 # ======================== Elasticsearch Configuration ========================= # # NOTE: Elasticsearch comes with reasonable defaults for most settings. # Before you set out to tweak and tune the configuration, make sure you # understand what are you trying to accomplish and the consequences. # # The primary way of configuring a node is via this file. This template lists # the most important settings you may want to configure for a production cluster. # # Please consult the documentation for further information on configuration options: # https://www.elastic.co/guide/en/elasticsearch/reference/index.html # # ---------------------------------- Cluster ----------------------------------- # # Use a descriptive name for your cluster: # cluster.name: {{ cluster_name }} # # ------------------------------------ Node ------------------------------------ # # Use a descriptive name for the node: # node.name: {{ node_name }} # # Add custom attributes to the node: # #node.attr.rack: r1 # # ----------------------------------- Paths ------------------------------------ # # Path to directory where to store the data (separate multiple locations by comma): # path.data: {{ es_install_dir }}/data # # Path to log files: # path.logs: {{ es_install_dir }}/logs # # ----------------------------------- Memory ----------------------------------- # # Lock the memory on startup: # #bootstrap.memory_lock: true # # Make sure that the heap size is set to about half the memory available # on the system and that the owner of the process is allowed to use this # limit. # # Elasticsearch performs poorly when the system is swapping the memory. # # ---------------------------------- Network ----------------------------------- # # By default Elasticsearch is only accessible on localhost. Set a different # address here to expose this node on the network: # network.host: {{ ansible_host }} # # By default Elasticsearch listens for HTTP traffic on the first free port it # finds starting at 9200. Set a specific HTTP port here: # http.port: {{ listen_port }} # # For more information, consult the network module documentation. # # --------------------------------- Discovery ---------------------------------- # # Pass an initial list of hosts to perform discovery when this node is started: # The default list of hosts is ["127.0.0.1", "[::1]"] # 生成为列表格式 discovery.seed_hosts: {% for host in groups['elasticsearch'] %} - "{{ hostvars[host]['ansible_host'] }}" {% endfor %} # 生成为列表格式 # Bootstrap the cluster using an initial set of master-eligible nodes: # cluster.initial_master_nodes: {% for host in groups['elasticsearch'] %} - "{{ hostvars[host]['node_name'] }}" {% endfor %} # # For more information, consult the discovery and cluster formation module documentation. # # ---------------------------------- Various ----------------------------------- # # Require explicit names when deleting indices: # #action.destructive_requires_name: true # # ---------------------------------- Security ---------------------------------- # # *** WARNING *** # # Elasticsearch security features are not enabled by default. # These features are free, but require configuration changes to enable them. # This means that users don’t have to provide credentials and can get full access # to the cluster. Network connections are also not encrypted. # # To protect your data, we strongly encourage you to enable the Elasticsearch security features. # Refer to the following documentation for instructions. # # https://www.elastic.co/guide/en/elasticsearch/reference/7.16/configuring-stack-security.html transport.tcp.port: 9300 xpack.security.enabled: true xpack.security.transport.ssl.enabled: true xpack.security.transport.ssl.verification_mode: certificate xpack.security.transport.ssl.keystore.path: {{ es_install_dir }}/config/elastic-certificates.p12 xpack.security.transport.ssl.truststore.path: {{ es_install_dir }}/config/elastic-certificates.p12 xpack.monitoring.collection.cluster.stats.timeout: '30s' ingest.geoip.downloader.enabled: false
task任务
main文件
[root@master-1 roles]# cat roles/es_cluster/tasks/main.yml --- # tasks file for es_cluster - include: install_java.yml - include: set_sysctl.yml - include: add_user.yml - include: unarchive_es.yml - include: ik.yml - include: create_date_dir.yml - include: set_es_jvm.yml - include: keystore.yml - include: create_cert.yml - include: fetch_cert.yml - include: copy_cert.yml - include: import_cert.yml - include: template_es_config.yml - include: chown_dir.yml - include: start.yml - include: check_process.yml
install_java.yml
- name: search java command ansible.builtin.shell: which java register: java_status ignore_errors: yes # 忽略错误，但不修改返回rc状态 - name: unarchive java ansible.builtin.unarchive: src: jdk-8u341-linux-x64.tar.gz dest: /opt/ when: java_status.rc != 0 - name: Configure Java environment variables ansible.builtin.blockinfile: path: /etc/profile block: | export JAVA_HOME={{ JAVA_HOME }} export CLASSPATH=.:{{ JAVA_HOME }}/jre/lib/rt.jar:{{ JAVA_HOME }}/lib/dt.jar:{{ JAVA_HOME }}/lib/tools.jar export PATH={{ JAVA_HOME }}/bin:$PATH marker: "# {mark} ANSIBLE MANAGED BLOCK - JAVA" when: java_status.rc != 0
set_sysctl.yml
- name: 修改并应用内核参数 ansible.builtin.sysctl: name: vm.max_map_count # 只能用'', ""语法报错 value: '262144' state: present reload: yes
add_user.yml
- name: 创建系统es登录用户 ansible.builtin.user: name: es shell: /bin/bash password: "{{ password }}"
unarchive_es.yml
- name: 解压es到远程服务器 ansible.builtin.unarchive: src: elasticsearch-7.16.3-linux-x86_64.tar.gz dest: /opt
ik.yml
- name: 创建ik分词器目录 ansible.builtin.file: path: "{{ es_install_dir }}/plugins/ik" state: directory - name: 导入ik分词器 ansible.builtin.unarchive: src: elasticsearch-analysis-ik-7.16.3.zip dest: "{{ es_install_dir }}/plugins/ik/"
create_date_dir.yml
- name: 创建数据目录 ansible.builtin.file: path: "{{ es_install_dir}}/data" state: directory
set_es_jvm.yml
- name: 修改JVM内存 ansible.builtin.lineinfile: path: "{{ es_install_dir }}/config/jvm.options" regexp: "^## -Xms" line: "-Xms{{ (ansible_memtotal_mb // 3)|int }}m" tags: xms - name: 修改JVM最大使用内存 ansible.builtin.lineinfile: path: "{{ es_install_dir }}/config/jvm.options" regexp: "^## -Xmx" line: "-Xmx{{ (ansible_memtotal_mb // 3)|int }}m" tags: xmx
keystore.yml
- name: 创建keystore ansible.builtin.shell: | cd "{{ es_install_dir }}/bin" ./elasticsearch-keystore create args: creates: "{{ es_install_dir }}/config/elasticsearch.keystore"
create_cert.yml
- name: 生成CA证书 ansible.builtin.shell: | cd "{{ es_install_dir }}/bin" ./elasticsearch-certutil ca --pass "{{ es_ca_pass }}" -out "{{ es_install_dir }}/config/elastic-stack-ca.p12" args: creates: "{{ es_install_dir }}/config/elastic-stack-ca.p12" when: inventory_hostname in groups['elasticsearch'] and hostvars[inventory_hostname].is_ca is defined and hostvars[inventory_hostname].is_ca - name: 生成节点证书 ansible.builtin.shell: | cd "{{ es_install_dir }}/bin" echo "{{ es_cert_pass}}" |./elasticsearch-certutil cert --ca "{{ es_install_dir }}/config/elastic-stack-ca.p12" --ca-pass "{{ es_ca_pass }}" -out "{{ es_install_dir }}/config/elastic-certificates.p12" args: creates: "{{ es_install_dir }}/config/elastic-certificates.p12" when: inventory_hostname in groups['elasticsearch'] and hostvars[inventory_hostname].is_ca is defined and hostvars[inventory_hostname].is_ca
fetch_cert.yml
- name: 拉取CA证书 ansible.builtin.fetch: src: "{{ es_install_dir }}/config/elastic-stack-ca.p12" dest: "/opt/ansible/playbook/roles/roles/es_cluster/files/" # 如果dest以/结尾，则复制到目录下，否则为文件名 flat: yes #when: inventory_hostname in groups['elasticsearch'] and hostvars[inventory_hostname].is_ca is defined and hostvars[inventory_hostname].is_ca when: is_ca is defined and is_ca - name: 拉取服务证书 ansible.builtin.fetch: src: "{{ es_install_dir }}/config/elastic-certificates.p12" dest: "/opt/ansible/playbook/roles/roles/es_cluster/files/" # 如果dest以/结尾，则复制到目录下，否则为文件名 flat: yes #when: inventory_hostname in groups['elasticsearch'] and hostvars[inventory_hostname].is_ca is defined and hostvars[inventory_hostname].is_ca when: is_ca is defined and is_ca
copy_cert.yml
- name: 拷贝CA证书 ansible.builtin.copy: src: "elastic-stack-ca.p12" dest: "{{ es_install_dir }}/config/elastic-stack-ca.p12" mode: '0440' - name: 拷贝服务证书 ansible.builtin.copy: src: "elastic-certificates.p12" dest: "{{ es_install_dir }}/config/" mode: '0400'
import_cert.yml
- name: 导入证书 ansible.builtin.shell: | cd "{{ es_install_dir }}/bin" echo "{{ es_cert_pass}}" |./elasticsearch-keystore add xpack.security.transport.ssl.keystore.secure_password echo "{{ es_cert_pass}}" |./elasticsearch-keystore add xpack.security.transport.ssl.truststore.secure_password
template_es_config.yml
- name: 渲染配置文件 ansible.builtin.template: src: elasticsearch.yml.j2 dest: "{{ es_install_dir }}/config/elasticsearch.yml" notify: restart elasticsearch
chown_dir.yml
- name: 修改目录权限 ansible.builtin.file: path: "{{ es_install_dir }}" owner: es group: es recurse: yes state: directory
start.yml
- name: 切换用户es，并启动es程序 ansible.builtin.shell: | source /etc/profile #cd "{{ es_install_dir }}/bin" && "./elasticsearch -d" nohup "{{ es_install_dir }}/bin/elasticsearch" 2>&1 >/dev/null & become: true become_user: es
check_process.yml
- name: 确保 Elasticsearch 进程存在 ansible.builtin.shell: | echo "正在启动es服务，请等待..." sleep 30 pgrep -f "elasticsearch" register: es_process # 判断命令是否执行成功 # return code 0:正常 1:错误 failed_when: es_process.rc != 0
执行任务
[root@master-1 roles]# ansible-playbook -i roles/es_cluster/hosts/inventory.yaml main_es_cluster.yml PLAY [elasticsearch] ********************************************************************************************************************************************************** TASK [Gathering Facts] ******************************************************************************************************************************************************** ok: [es-node3] ok: [es-node1] ok: [es-node2] TASK [es_cluster : search java command] *************************************************************************************************************************************** fatal: [es-node2]: FAILED! => {"changed": true, "cmd": "which java", "delta": "0:00:00.015623", "end": "2025-02-25 16:40:29.869761", "msg": "non-zero return code", "rc": 1, "start": "2025-02-25 16:40:29.854138", "stderr": "which: no java in (/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin)", "stderr_lines": ["which: no java in (/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin)"], "stdout": "", "stdout_lines": []} ...ignoring fatal: [es-node3]: FAILED! => {"changed": true, "cmd": "which java", "delta": "0:00:00.015531", "end": "2025-02-25 16:40:29.869835", "msg": "non-zero return code", "rc": 1, "start": "2025-02-25 16:40:29.854304", "stderr": "which: no java in (/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin)", "stderr_lines": ["which: no java in (/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin)"], "stdout": "", "stdout_lines": []} ...ignoring fatal: [es-node1]: FAILED! => {"changed": true, "cmd": "which java", "delta": "0:00:00.015946", "end": "2025-02-25 16:40:29.888443", "msg": "non-zero return code", "rc": 1, "start": "2025-02-25 16:40:29.872497", "stderr": "which: no java in (/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin)", "stderr_lines": ["which: no java in (/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin)"], "stdout": "", "stdout_lines": []} ...ignoring TASK [es_cluster : unarchive java] ******************************************************************************************************************************************** ^[aok: [es-node1] ok: [es-node3] ok: [es-node2] TASK [es_cluster : Configure Java environment variables] ********************************************************************************************************************** ok: [es-node2] ok: [es-node3] ok: [es-node1] TASK [es_cluster : 修改并应用内核参数] ************************************************************************************************************************************************* ok: [es-node2] ok: [es-node3] ok: [es-node1] TASK [es_cluster : 创建系统es登录用户] ************************************************************************************************************************************************ ok: [es-node2] ok: [es-node1] ok: [es-node3] TASK [es_cluster : 解压es到远程服务器] ************************************************************************************************************************************************ changed: [es-node2] changed: [es-node1] changed: [es-node3] TASK [es_cluster : 创建ik分词器目录] ************************************************************************************************************************************************* changed: [es-node2] changed: [es-node3] changed: [es-node1] TASK [es_cluster : 导入ik分词器] *************************************************************************************************************************************************** changed: [es-node2] changed: [es-node3] changed: [es-node1] TASK [es_cluster : 创建数据目录] **************************************************************************************************************************************************** changed: [es-node3] changed: [es-node2] changed: [es-node1] TASK [es_cluster : 修改JVM内存] *************************************************************************************************************************************************** changed: [es-node3] changed: [es-node2] changed: [es-node1] TASK [es_cluster : 修改JVM最大使用内存] *********************************************************************************************************************************************** changed: [es-node3] changed: [es-node2] changed: [es-node1] TASK [es_cluster : 创建keystore] ************************************************************************************************************************************************ changed: [es-node2] changed: [es-node3] changed: [es-node1] TASK [es_cluster : 生成CA证书] **************************************************************************************************************************************************** skipping: [es-node3] skipping: [es-node2] changed: [es-node1] TASK [es_cluster : 生成节点证书] **************************************************************************************************************************************************** skipping: [es-node3] skipping: [es-node2] changed: [es-node1] TASK [es_cluster : 拉取CA证书] **************************************************************************************************************************************************** skipping: [es-node3] skipping: [es-node2] changed: [es-node1] TASK [es_cluster : 拉取服务证书] **************************************************************************************************************************************************** skipping: [es-node3] skipping: [es-node2] changed: [es-node1] TASK [es_cluster : 拷贝CA证书] **************************************************************************************************************************************************** changed: [es-node3] changed: [es-node2] changed: [es-node1] TASK [es_cluster : 拷贝服务证书] **************************************************************************************************************************************************** changed: [es-node3] changed: [es-node2] changed: [es-node1] TASK [es_cluster : 导入证书] ****************************************************************************************************************************************************** changed: [es-node3] changed: [es-node2] changed: [es-node1] TASK [es_cluster : 渲染配置文件] **************************************************************************************************************************************************** changed: [es-node3] changed: [es-node2] changed: [es-node1] TASK [es_cluster : 修改目录权限] **************************************************************************************************************************************************** changed: [es-node3] changed: [es-node2] changed: [es-node1] TASK [es_cluster : 切换用户es，并启动es程序] ******************************************************************************************************************************************** [WARNING]: Unable to use /home/es/.ansible/tmp as temporary directory, failing back to system: [Errno 13] 权限不够: '/home/es/.ansible' changed: [es-node3] changed: [es-node2] changed: [es-node1] TASK [es_cluster : 确保 Elasticsearch 进程存在] ************************************************************************************************************************************* changed: [es-node3] changed: [es-node2] changed: [es-node1] RUNNING HANDLER [es_cluster : restart elasticsearch] ************************************************************************************************************************** changed: [es-node2] changed: [es-node1] changed: [es-node3] PLAY RECAP ******************************************************************************************************************************************************************** es-node1 : ok=25 changed=20 unreachable=0 failed=0 skipped=0 rescued=0 ignored=1 es-node2 : ok=21 changed=16 unreachable=0 failed=0 skipped=4 rescued=0 ignored=1 es-node3 : ok=21 changed=16 unreachable=0 failed=0 skipped=4 rescued=0 ignored=1
本文来自博客园，作者：不会跳舞的胖子，转载请注明原文链接：https://www.cnblogs.com/rtnb/p/18736522
浙公网安备 33010602011771号