saltstack之数据系统Grains、Pillar
1 Grains简介
Grains是saltstack的组件，用于收集salt-minion在启动时候的信息，又称为静态信息。可以理解为Grains记录着每台Minion的一些常用属性，
比如CPU、内存、磁盘、网络信息等。我们可以通过grains.items来查看某台Minion的所有Grains信息。
Grains是服务器的一系列粒子信息，也就是服务器的一系列物理，软件环境信息。在执行salt的sls时候可以根据Grains信息的不同对服务器进行匹配分组，
例如可以根据系统是centos服务器跟系统是redhat环境的安装不同的软件包。
Grains功能：
1.收集资产信息
2.信息查询
2 Grains 功能使用
2.1 grains信息查询
[root@slave1 ~]# salt '*' grains.items #查看所有grains的key和values
[root@slave1 ~]# salt '*' grains.get saltversion #查看salt的版本
slave2:
2015.5.10
slave1:
2015.5.10
[root@slave1 ~]# salt '*' grains.get ip4_interfaces:eth0 #查看ip
slave2:
- 10.0.0.212
slave1:
- 10.0.0.211
2.2 grains目标匹配
使用salt -G ：grains可以用于进行目标匹配，比如让所有的centos系统进行某个操作
对os系统为centos系统执行一个uptime的命令： [root@slave1 ~]# salt -G 'init:systemd' cmd.run 'uptime' #查看负载 slave1: 15:46:28 up 2:37, 1 user, load average: 0.17, 0.09, 0.13 slave2: 15:46:28 up 2:36, 1 user, load average: 0.00, 0.01, 0.05 在init为systemd的系统上执行查看内存 [root@slave1 ~]# salt -G 'init:systemd' cmd.run 'free -m' slave1: total used free shared buff/cache available Mem: 974 529 145 1 299 266 Swap: 551 9 542 slave2: total used free shared buff/cache available Mem: 974 378 250 6 346 411 Swap: 551 1 550
2.3 grains在top file 文件中匹配
[root@slave1 ~]# cat /srv/salt/base/top.sls base: 'os:CentOS': - match: grain - web.apache
[root@slave1 ~]# salt '*' state.highstate slave2: ---------- ID: apache-install Function: pkg.installed Name: httpd Result: True Comment: The following packages were installed/updated: httpd Started: 15:52:16.434685 Duration: 18666.578 ms Changes: ---------- httpd: ---------- new: 2.4.6-80.el7.centos.1 old: ---------- ID: apache-service Function: service.running Name: httpd Result: True Comment: Service httpd has been enabled, and is running Started: 15:52:35.154340 Duration: 1390.149 ms Changes: ---------- httpd: True Summary ------------ Succeeded: 2 (changed=2) Failed: 0 ------------ Total states run: 2 slave1: ---------- ID: apache-install Function: pkg.installed Name: httpd Result: True Comment: The following packages were installed/updated: httpd Started: 15:52:17.061840 Duration: 30150.387 ms Changes: ---------- httpd: ---------- new: 2.4.6-80.el7.centos.1 old: ---------- ID: apache-service Function: service.running Name: httpd Result: True Comment: Service httpd has been enabled, and is running Started: 15:52:47.722916 Duration: 2312.489 ms Changes: ---------- httpd: True Summary ------------ Succeeded: 2 (changed=2) Failed: 0 ------------ Total states run: 2
2.4 grains自定义：
Grains的四种存在形式：
①Core grains.
②在 /etc/salt/grains 自定义grains。
③在 /etc/salt/minion 自定义grains。
④在 _grains 目录自定义grain，同步到minions。
#生产环境使用自定义一个grains [root@slave1 ~]# cat /etc/salt/grains hw: hello,world! ip: 10.0.0.211 [root@slave1 ~]# systemctl restart salt-minion [root@slave1 ~]# salt '*' grains.get hw slave2: slave1: hello,world! root@slave2 ~]# vim /etc/salt/grains test-grains: slave2 ip: 10.0.0.212 wq: wangqu [root@slave1 ~]# salt '*' saltutil.sync_grains slave1: slave2: [root@slave1 ~]# salt '*' grains.get wq slave2: wangqu slave1:
3 Pillar简介
Pillar是Salt最重要的系统之一，它跟grains的结构一样，也是一个字典格式，数据通过key/value的格式进行存储。
在Salt的设计中，Pillar使用独立的加密sessiion。可用于提供开发接口，用于在master端定义数据，然后再minion中使用，
一般传输敏感的数据，例如ssh key，加密证书等。
pillar和states建立方式类似，由sls文件组成，有一个入口文件top.sls，通过这个文件关联其他sls文件，
默认路径在/srv/pillar,可通过/etc/salt/master里面pillar_roots:指定位置。
pillar到底什么作用呢？那么下面介绍一个简单的例子，你就明白了。
例如：用zabbix监控新上架的服务器（10台），需要将zabbix_agentd.conf分发到被监控主机，
这个文件中hostname的ip每台都不同，我们不可能写10分配置文件吧！那么如何让hostname在分发的时候就根据被监控主机IP，修改成自己的呢？
这时就用到渲染了，默认渲染器是jinja，支持for in循环判断，格式是{%…%}{% end* %}，这样一来salt会先让jinja渲染，然后交给yaml处理。
4 Pillar的功能使用
4.1 定义义Pillar数据
1）master配置文件中定义pillar：
默认情况下，master配置文件中的所有数据都添加到Pillar中，且对所有minion可用。如果要禁用这一默认值
可以在master配置文件中添加如下数据，重启服务后生效：
#默认的pillar的items为空，需要修改/etc/salt/master [root@slave1 ~]# salt '*' pillar.items slave1: ---------- slave2: ---------- [root@slave1 ~]# vim /etc/salt/master #pillar_opts: False 打开该项，修改成True pillar_opts: True
#重启：
[root@slave1 ~]# systemctl restart salt-master.service
[root@slave1 ~]# salt '*' pillar.items
slave2: ---------- master: ---------- __role: master auth_mode: 1 auto_accept: False cache_sreqs: True cachedir: /var/cache/salt/master cli_summary: False client_acl: ---------- client_acl_blacklist: ---------- cluster_masters: cluster_mode: paranoid con_cache: False conf_file: /etc/salt/master config_dir: /etc/salt cython_enable: False daemon: False default_include: master.d/*.conf enable_gpu_grains: False enforce_mine_cache: False enumerate_proxy_minions: False environment: None event_return: event_return_blacklist: event_return_queue: 0 event_return_whitelist: ext_job_cache: ext_pillar: extension_modules: /var/cache/salt/extmods external_auth: ---------- failhard: False file_buffer_size: 1048576 file_client: local file_ignore_glob: None file_ignore_regex: None file_recv: False file_recv_max_size: 100 file_roots: ---------- base: - /srv/salt/base dev: - /srv/salt/dev prod: - /srv/salt/prod test: - /srv/salt/test/ fileserver_backend: - roots fileserver_followsymlinks: True fileserver_ignoresymlinks: False fileserver_limit_traversal: False gather_job_timeout: 10 gitfs_base: master gitfs_env_blacklist: gitfs_env_whitelist: gitfs_insecure_auth: False gitfs_mountpoint: gitfs_passphrase: gitfs_password: gitfs_privkey: gitfs_pubkey: gitfs_remotes: gitfs_root: gitfs_user: hash_type: md5 hgfs_base: default hgfs_branch_method: branches hgfs_env_blacklist: hgfs_env_whitelist: hgfs_mountpoint: hgfs_remotes: hgfs_root: id: slave2 interface: 0.0.0.0 ioflo_console_logdir: ioflo_period: 0.01 ioflo_realtime: True ioflo_verbose: 0 ipv6: False jinja_lstrip_blocks: False jinja_trim_blocks: False job_cache: True keep_jobs: 24 key_logfile: /var/log/salt/key keysize: 2048 log_datefmt: %H:%M:%S log_datefmt_logfile: %Y-%m-%d %H:%M:%S log_file: /var/log/salt/master log_fmt_console: [%(levelname)-8s] %(message)s log_fmt_logfile: %(asctime)s,%(msecs)03.0f [%(name)-17s][%(levelname)-8s][%(process)d] %(message)s log_granular_levels: ---------- log_level: warning loop_interval: 60 maintenance_floscript: /usr/lib/python2.7/site-packages/salt/daemons/flo/maint.flo master_floscript: /usr/lib/python2.7/site-packages/salt/daemons/flo/master.flo master_job_cache: local_cache master_pubkey_signature: master_pubkey_signature master_roots: ---------- base: - /srv/salt-master master_sign_key_name: master_sign master_sign_pubkey: False master_tops: ---------- master_use_pubkey_signature: False max_event_size: 1048576 max_minions: 0 max_open_files: 100000 minion_data_cache: True minionfs_blacklist: minionfs_env: base minionfs_mountpoint: minionfs_whitelist: nodegroups: ---------- open_mode: False order_masters: False outputter_dirs: peer: ---------- permissive_pki_access: False pidfile: /var/run/salt-master.pid pillar_opts: True pillar_roots: ---------- base: - /srv/pillar pillar_safe_render_error: True pillar_source_merging_strategy: smart pillar_version: 2 pillarenv: None ping_on_rotate: False pki_dir: /etc/salt/pki/master preserve_minion_cache: False pub_hwm: 1000 publish_port: 4505 publish_session: 86400 queue_dirs: raet_alt_port: 4511 raet_clear_remotes: False raet_main: True raet_mutable: False raet_port: 4506 range_server: range:80 reactor: reactor_refresh_interval: 60 reactor_worker_hwm: 10000 reactor_worker_threads: 10 renderer: yaml_jinja ret_port: 4506 root_dir: / rotate_aes_key: True runner_dirs: saltversion: 2015.5.10 search: search_index_interval: 3600 serial: msgpack show_jid: False show_timeout: True sign_pub_messages: False sock_dir: /var/run/salt/master sqlite_queue_dir: /var/cache/salt/master/queues ssh_passwd: ssh_port: 22 ssh_scan_ports: 22 ssh_scan_timeout: 0.01 ssh_sudo: False ssh_timeout: 60 ssh_user: root state_aggregate: False state_auto_order: True state_events: False state_output: full state_top: salt://top.sls state_top_saltenv: None state_verbose: True sudo_acl: False svnfs_branches: branches svnfs_env_blacklist: svnfs_env_whitelist: svnfs_mountpoint: svnfs_remotes: svnfs_root: svnfs_tags: tags svnfs_trunk: trunk syndic_dir: /var/cache/salt/master/syndics syndic_event_forward_timeout: 0.5 syndic_jid_forward_cache_hwm: 100 syndic_master: syndic_max_event_process_time: 0.5 syndic_wait: 5 timeout: 5 token_dir: /var/cache/salt/master/tokens token_expire: 43200 transport: zeromq user: root verify_env: True win_gitrepos: - https://github.com/saltstack/salt-winrepo.git win_repo: /srv/salt/win/repo win_repo_mastercachefile: /srv/salt/win/repo/winrepo.p worker_floscript: /usr/lib/python2.7/site-packages/salt/daemons/flo/worker.flo worker_threads: 5 zmq_filtering: False slave1: ---------- master: ---------- __role: master auth_mode: 1 auto_accept: False cache_sreqs: True cachedir: /var/cache/salt/master cli_summary: False client_acl: ---------- client_acl_blacklist: ---------- cluster_masters: cluster_mode: paranoid con_cache: False conf_file: /etc/salt/master config_dir: /etc/salt cython_enable: False daemon: False default_include: master.d/*.conf enable_gpu_grains: False enforce_mine_cache: False enumerate_proxy_minions: False environment: None event_return: event_return_blacklist: event_return_queue: 0 event_return_whitelist: ext_job_cache: ext_pillar: extension_modules: /var/cache/salt/extmods external_auth: ---------- failhard: False file_buffer_size: 1048576 file_client: local file_ignore_glob: None file_ignore_regex: None file_recv: False file_recv_max_size: 100 file_roots: ---------- base: - /srv/salt/base dev: - /srv/salt/dev prod: - /srv/salt/prod test: - /srv/salt/test/ fileserver_backend: - roots fileserver_followsymlinks: True fileserver_ignoresymlinks: False fileserver_limit_traversal: False gather_job_timeout: 10 gitfs_base: master gitfs_env_blacklist: gitfs_env_whitelist: gitfs_insecure_auth: False gitfs_mountpoint: gitfs_passphrase: gitfs_password: gitfs_privkey: gitfs_pubkey: gitfs_remotes: gitfs_root: gitfs_user: hash_type: md5 hgfs_base: default hgfs_branch_method: branches hgfs_env_blacklist: hgfs_env_whitelist: hgfs_mountpoint: hgfs_remotes: hgfs_root: id: slave1 interface: 0.0.0.0 ioflo_console_logdir: ioflo_period: 0.01 ioflo_realtime: True ioflo_verbose: 0 ipv6: False jinja_lstrip_blocks: False jinja_trim_blocks: False job_cache: True keep_jobs: 24 key_logfile: /var/log/salt/key keysize: 2048 log_datefmt: %H:%M:%S log_datefmt_logfile: %Y-%m-%d %H:%M:%S log_file: /var/log/salt/master log_fmt_console: [%(levelname)-8s] %(message)s log_fmt_logfile: %(asctime)s,%(msecs)03.0f [%(name)-17s][%(levelname)-8s][%(process)d] %(message)s log_granular_levels: ---------- log_level: warning loop_interval: 60 maintenance_floscript: /usr/lib/python2.7/site-packages/salt/daemons/flo/maint.flo master_floscript: /usr/lib/python2.7/site-packages/salt/daemons/flo/master.flo master_job_cache: local_cache master_pubkey_signature: master_pubkey_signature master_roots: ---------- base: - /srv/salt-master master_sign_key_name: master_sign master_sign_pubkey: False master_tops: ---------- master_use_pubkey_signature: False max_event_size: 1048576 max_minions: 0 max_open_files: 100000 minion_data_cache: True minionfs_blacklist: minionfs_env: base minionfs_mountpoint: minionfs_whitelist: nodegroups: ---------- open_mode: False order_masters: False outputter_dirs: peer: ---------- permissive_pki_access: False pidfile: /var/run/salt-master.pid pillar_opts: True pillar_roots: ---------- base: - /srv/pillar pillar_safe_render_error: True pillar_source_merging_strategy: smart pillar_version: 2 pillarenv: None ping_on_rotate: False pki_dir: /etc/salt/pki/master preserve_minion_cache: False pub_hwm: 1000 publish_port: 4505 publish_session: 86400 queue_dirs: raet_alt_port: 4511 raet_clear_remotes: False raet_main: True raet_mutable: False raet_port: 4506 range_server: range:80 reactor: reactor_refresh_interval: 60 reactor_worker_hwm: 10000 reactor_worker_threads: 10 renderer: yaml_jinja ret_port: 4506 root_dir: / rotate_aes_key: True runner_dirs: saltversion: 2015.5.10 search: search_index_interval: 3600 serial: msgpack show_jid: False show_timeout: True sign_pub_messages: False sock_dir: /var/run/salt/master sqlite_queue_dir: /var/cache/salt/master/queues ssh_passwd: ssh_port: 22 ssh_scan_ports: 22 ssh_scan_timeout: 0.01 ssh_sudo: False ssh_timeout: 60 ssh_user: root state_aggregate: False state_auto_order: True state_events: False state_output: full state_top: salt://top.sls state_top_saltenv: None state_verbose: True sudo_acl: False svnfs_branches: branches svnfs_env_blacklist: svnfs_env_whitelist: svnfs_mountpoint: svnfs_remotes: svnfs_root: svnfs_tags: tags svnfs_trunk: trunk syndic_dir: /var/cache/salt/master/syndics syndic_event_forward_timeout: 0.5 syndic_jid_forward_cache_hwm: 100 syndic_master: syndic_max_event_process_time: 0.5 syndic_wait: 5 timeout: 5 token_dir: /var/cache/salt/master/tokens token_expire: 43200 transport: zeromq user: root verify_env: True win_gitrepos: - https://github.com/saltstack/salt-winrepo.git win_repo: /srv/salt/win/repo win_repo_mastercachefile: /srv/salt/win/repo/winrepo.p worker_floscript: /usr/lib/python2.7/site-packages/salt/daemons/flo/worker.flo worker_threads: 5 zmq_filtering: False
2）使用sls文件定义Pillar
Pillar使用与State相似的sls文件。Pillar文件放在master配置文件中pillar_roots定义的目录下。示例如下：
[root@slave1 ~]# vim /etc/salt/master pillar_roots: base: - /srv/pillar/base prod: - /srv/pillar/prod
#说明：此段代码定义了base环境下的Pillar文件保存在/srv/pillar/base目录下。prod环境下的Pillar文件保存在/srv/pillar/prod下
[root@slave1 ~]# mkdir -p /srv/pillar/{base,prod}
[root@slave1 ~]# tree /srv/pillar/
/srv/pillar/
├── base
└── prod
[root@linux-node1 ~]# systemctl restart salt-master
#创建base环境下的pillar文件为apache
[root@slave1 ~]# vim /srv/pillar/base/apache.sls
{% if grains['os'] == 'CentOS' %}
apache: httpd
{% elif grains['os'] == 'Debian' %}
apache: apache2
{% endif %}
#与State相似，Pillar也有top file，也使用相同的匹配方式将数据应用到minion上。示例如下：
[root@slave1 ~]# vim /srv/pillar/base/top.sls
base:
'*':
- apache
[root@slave1 ~]# salt '*' pillar.items
slave2:
----------
apache:
httpd
slave1:
----------
apache:
httpd
#在base环境下，引用pillar
# vim /srv/salt/base/web/apache.sls
apache-install:
pkg.installed:
- name: {{ pillar['apache'] }}
apache-service:
service.running:
- name: {{ pillar['apache'] }}
- enable: True
[root@slave1 ~]# salt '*' state.highstate
slave2:
----------
ID: apache-install
Function: pkg.installed
Name: httpd
Result: True
Comment: The following packages were installed/updated: httpd
Started: 17:29:02.383663
Duration: 18596.536 ms
Changes:
----------
httpd:
----------
new:
2.4.6-80.el7.centos.1
old:
----------
ID: apache-service
Function: service.running
Name: httpd
Result: True
Comment: Service httpd has been enabled, and is running
Started: 17:29:20.998274
Duration: 1374.484 ms
Changes:
----------
httpd:
True
Summary
------------
Succeeded: 2 (changed=2)
Failed: 0
------------
Total states run: 2
slave1:
----------
ID: apache-install
Function: pkg.installed
Name: httpd
Result: True
Comment: The following packages were installed/updated: httpd
Started: 17:29:02.760729
Duration: 35200.554 ms
Changes:
----------
httpd:
----------
new:
2.4.6-80.el7.centos.1
old:
----------
ID: apache-service
Function: service.running
Name: httpd
Result: True
Comment: Service httpd has been enabled, and is running
Started: 17:29:38.038008
Duration: 2393.187 ms
Changes:
----------
httpd:
True
Summary
------------
Succeeded: 2 (changed=2)
Failed: 0
------------
Total states run: 2
小结：
1.pillar和状态一样，有pillar_roots，在master中配置
2.到配置的地方/srv/pillar/base下写一个apache.sls
3.pillar必须在top file指定才能使用，在top.sls中指定所有的minion，都需要执行在base环境下的apache.sls
4.用之前查看是否能获取到pillar值：salt ‘*’ pillar.items
5.更改状态配置，把name改为一个pillar的引用，这是一个jinja的语法
补充：
存储位置 类型 采集方式 场景 Grains minion 静态 minion启动时，可以刷新 1.获取信息 2.匹配 Pillar master 动态 指定， 实时生效 1.匹配 2.敏感数据配置 配置中心：https://www.cnblogs.com/xiaoqi/p/configserver-compair.html 分布式配置中心选择方案：https://blog.csdn.net/z960339491/article/details/80521882
浙公网安备 33010602011771号