linux-安装zookeeper及相关操作
下载两个安装包并解压:
配置jdk环境变量:
[root@VM-0-10-centos zookeeper]# cat /root/.bash_profile # .bash_profile # Get the aliases and functions if [ -f ~/.bashrc ]; then . ~/.bashrc fi # User specific environment and startup programs #PATH=$PATH:$HOME/bin:$JAVA_HOME JAVA_HOME=/home/zookeeper/jdk1.8.0_131/ export JAVA_HOME PATH=$PATH:$HOME/bin:$JAVA_HOME export PATH
测试jkd是否生效:
[root@VM-0-10-centos zookeeper]# java -version java version "1.8.0_131" Java(TM) SE Runtime Environment (build 1.8.0_131-b11) Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode)
进如zookeeper配置目录,编写配置文件:
[root@VM-0-10-centos conf]# cp zoo_sample.cfg zoo.cfg 建立zookeepr数据目录 [root@VM-0-10-centos data]# pwd /home/zookeeper/zookeeper-3.4.10/data
配置文件:
启动查看状态:
[root@VM-0-10-centos bin]# pwd /home/zookeeper/zookeeper-3.4.10/bin [root@VM-0-10-centos bin]# sh zkServer.sh start ZooKeeper JMX enabled by default Using config: /home/zookeeper/zookeeper-3.4.10/bin/../conf/zoo.cfg Starting zookeeper ... STARTED [root@VM-0-10-centos bin]# sh zkServer.sh status ZooKeeper JMX enabled by default Using config: /home/zookeeper/zookeeper-3.4.10/bin/../conf/zoo.cfg Mode: standalone [root@VM-0-10-cent
登陆:
[root@VM-0-10-centos bin]# pwd /home/zookeeper/zookeeper-3.4.10/bin [root@VM-0-10-centos bin]# sh zkCli.sh Connecting to localhost:2181
zookeeper常用的Shell命令:
新增节点
格式:节点的路径 节点的数据
create [-s] [-e] path data #其中-s 为有序节点,-e 临时节点
注意:默认什么都不加的是无序持久节点
登陆服务器:
新增一个持久化节点,默认就是持久化节点 [zk: localhost:2181(CONNECTED) 0] create /hadoop "1234" Created /hadoop 查看节点的数据 [zk: localhost:2181(CONNECTED) 1] get /hadoop 1234 cZxid = 0x6 ctime = Sat Sep 05 15:05:31 CST 2020 mZxid = 0x6 mtime = Sat Sep 05 15:05:31 CST 2020 pZxid = 0x6 cversion = 0 dataVersion = 0 aclVersion = 0 ephemeralOwner = 0x0 dataLength = 4 numChildren = 0
创建持久化有序节点:
应用场景:为分布式环境创建一个唯一id
[zk: localhost:2181(CONNECTED) 1] create -s /a "aaaa" Created /a0000000001 [zk: localhost:2181(CONNECTED) 2] get /a0000000001 aaaa cZxid = 0x9 ctime = Sat Sep 05 15:09:10 CST 2020 mZxid = 0x9 mtime = Sat Sep 05 15:09:10 CST 2020 pZxid = 0x9 cversion = 0 dataVersion = 0 aclVersion = 0 ephemeralOwner = 0x0 dataLength = 4 numChildren = 0 [zk: localhost:2181(CONNECTED) 3] get /a Node does not exist: /a
注意:创建时候的路径名字不再是获取数据的哪个名字,a---->a00000001
创建临时节点:
与会话共存亡
[zk: localhost:2181(CONNECTED) 4] create -e /tmp "tmp" Created /tmp [zk: localhost:2181(CONNECTED) 5] get /tmp tmp cZxid = 0xa ctime = Sat Sep 05 15:13:15 CST 2020 mZxid = 0xa mtime = Sat Sep 05 15:13:15 CST 2020 pZxid = 0xa cversion = 0 dataVersion = 0 aclVersion = 0 ephemeralOwner = 0x1745d11ec160002 dataLength = 3 numChildren = 0
注意:登陆后不存在
临时有序节点
临时节点会在会话过期后被删除,建立分布式锁
[zk: localhost:2181(CONNECTED) 6] create -s -e /quan "quan" Created /quan0000000003 [zk: localhost:2181(CONNECTED) 7] get /quan0000000003 quan cZxid = 0xb ctime = Sat Sep 05 15:15:11 CST 2020 mZxid = 0xb mtime = Sat Sep 05 15:15:11 CST 2020 pZxid = 0xb cversion = 0 dataVersion = 0 aclVersion = 0 ephemeralOwner = 0x1745d11ec160002 dataLength = 4 numChildren = 0
修改节点
格式1
set 节点路径 新的节点的值
[zk: localhost:2181(CONNECTED) 8] get /hadoop 1234 cZxid = 0x6 ctime = Sat Sep 05 15:05:31 CST 2020 mZxid = 0x6 mtime = Sat Sep 05 15:05:31 CST 2020 pZxid = 0x6 cversion = 0 dataVersion = 0 aclVersion = 0 ephemeralOwner = 0x0 dataLength = 4 numChildren = 0 [zk: localhost:2181(CONNECTED) 9] set /hadoop "879" cZxid = 0x6 ctime = Sat Sep 05 15:05:31 CST 2020 mZxid = 0xc mtime = Sat Sep 05 15:17:35 CST 2020 pZxid = 0x6 cversion = 0 dataVersion = 1 aclVersion = 0 ephemeralOwner = 0x0 dataLength = 3 numChildren = 0 [zk: localhost:2181(CONNECTED) 10] get /hadoop 879 cZxid = 0x6 ctime = Sat Sep 05 15:05:31 CST 2020 mZxid = 0xc mtime = Sat Sep 05 15:17:35 CST 2020 pZxid = 0x6 cversion = 0 dataVersion = 1
格式2
基于版本号进行更改,此时类似于乐观锁机制,当你传入的数据版本号
(dataVersion) 和当前节点的数据版本号不符合时,zookeeper 会拒绝本次修改:
可以联想一些关系型数据库来理解
注意:第一次创建的节点,节点信息会附带一个信息,就是dataversion,从0开始
每一次修改都会+1
[zk: localhost:2181(CONNECTED) 11] set /hadoop "2222" 0 version No is not valid : /hadoop [zk: localhost:2181(CONNECTED) 12] set /hadoop "2222" 1 cZxid = 0x6 ctime = Sat Sep 05 15:05:31 CST 2020 mZxid = 0xe mtime = Sat Sep 05 15:23:13 CST 2020 pZxid = 0x6 cversion = 0 dataVersion = 2 aclVersion = 0 ephemeralOwner = 0x0 dataLength = 4 numChildren = 0
删除节点:
格式:
delete path [version]
和更新节点数据一样,也可以传入版本号,当你传入的数据版本号 (dataVersion)
和当前节点的数据版本号不符合时,zookeeper 不会执行删除操作。
[zk: localhost:2181(CONNECTED) 13] delete /hadoop [zk: localhost:2181(CONNECTED) 16] get /hadoop Node does not exist: /hadoop [zk: localhost:2181(CONNECTED) 17] create /hadoop "99" Created /hadoop [zk: localhost:2181(CONNECTED) 18] get /hadoop 99 cZxid = 0x10 ctime = Sat Sep 05 15:25:31 CST 2020 mZxid = 0x10 mtime = Sat Sep 05 15:25:31 CST 2020 pZxid = 0x10 cversion = 0 dataVersion = 0 aclVersion = 0 ephemeralOwner = 0x0 dataLength = 2 numChildren = 0 [zk: localhost:2181(CONNECTED) 19] delete /hadoop 1 version No is not valid : /hadoop [zk: localhost:2181(CONNECTED) 20] delete /hadoop 0 [zk: localhost:2181(CONNECTED) 21] get /hadoop Node does not exist: /hadoop
注意:要想删除某个节点及其所有后代节点,可以使用递归删除,命令为 rmr
[zk: localhost:2181(CONNECTED) 22] create /hadoop "99" Created /hadoop [zk: localhost:2181(CONNECTED) 23] create /hadoop/son "998" Created /hadoop/son [zk: localhost:2181(CONNECTED) 24] get /hadoop 99 cZxid = 0x13 ctime = Sat Sep 05 15:27:07 CST 2020 mZxid = 0x13 mtime = Sat Sep 05 15:27:07 CST 2020 pZxid = 0x14 cversion = 1 dataVersion = 0 aclVersion = 0 ephemeralOwner = 0x0 dataLength = 2 numChildren = 1 [zk: localhost:2181(CONNECTED) 25] get /hadoop/son 998 cZxid = 0x14 ctime = Sat Sep 05 15:27:16 CST 2020 mZxid = 0x14 mtime = Sat Sep 05 15:27:16 CST 2020 pZxid = 0x14 cversion = 0 dataVersion = 0 aclVersion = 0 ephemeralOwner = 0x0 dataLength = 3 numChildren = 0 [zk: localhost:2181(CONNECTED) 26] del delquota delete [zk: localhost:2181(CONNECTED) 26] del delquota delete [zk: localhost:2181(CONNECTED) 26] delete /hadoop Node not empty: /hadoop [zk: localhost:2181(CONNECTED) 27] rmr /hadoop
查看节点:
格式:
get path
[zk: localhost:2181(CONNECTED) 29] get /hadoop 99 cZxid = 0x18 数据节点创建时的事务ID ctime = Sat Sep 05 15:29:57 CST 2020 创建时的时间 mZxid = 0x18 最后一次更新时的事务ID mtime = Sat Sep 05 15:29:57 CST 2020 最后一次更新时的时间 pZxid = 0x18 数据节点的子节点最后一次被修改时的事务ID cversion = 0 子节点的更改次数 dataVersion = 0 节点数据的更改次数即版本数 aclVersion = 0 节点的ACL的更改次数 ephemeralOwner = 0x0 临时节点,表示创建该节点的会话SessionID,持久节点,改属性为0 dataLength = 2 数据内容长度 numChildren = 0 数据节点当前的子节点个数
注意:Zxid(ZooKeeper TransactionId),ZooKeeper 节点的每一次更改都具有唯一的 Zxid
如果 Zxid1 小于 Zxid2,则Zxid1 的更改发生在 Zxid2 更改之前
查看节点状态
可以使用 stat 命令查看节点状态,它的返回值和 get 命令类似,但不会返回
节点数据
[zk: localhost:2181(CONNECTED) 30] stat /hadoop cZxid = 0x18 ctime = Sat Sep 05 15:29:57 CST 2020 mZxid = 0x18 mtime = Sat Sep 05 15:29:57 CST 2020 pZxid = 0x18 cversion = 0 dataVersion = 0 aclVersion = 0 ephemeralOwner = 0x0 dataLength = 2 numChildren = 0
查看节点列表
ls path 返回当前节点的子节点列表
ls2 path 放回当前节点的子节点列表和当前节点的状态信息
[zk: localhost:2181(CONNECTED) 33] ls /hadoop [qq1, qq2] [zk: localhost:2181(CONNECTED) 34] ls2 /hadoop [qq1, qq2] cZxid = 0x18 ctime = Sat Sep 05 15:29:57 CST 2020 mZxid = 0x18 mtime = Sat Sep 05 15:29:57 CST 2020 pZxid = 0x1a cversion = 2 dataVersion = 0 aclVersion = 0 ephemeralOwner = 0x0 dataLength = 2 numChildren = 2
监听器:一次性用品,用完就没有了,要重新买
get path [watch]
注册的监听器能够在节点内容发生改变的时候,向客
户端发出通知。需要注意的是 zookeeper 的触发器是一次性的 (One-time trigger),即
触发一次后就会立即失效。
stat path [watch]
注册的监听器能够在节点状态发生改变的时候,向客户端发出通知
ls/ls2 path [watch]
使用 ls path [watch] 或 ls2 path [watch] 注册的监听器能够监听该节点下所有子节点的增加和删除操作。
Zookeeper的ACL权限控制
zookeeper的access control list 访问控制列表
acl 权限控制,使用scheme:id:permission 来标识
权限模式(scheme):授权的策略
授权对象(id):授权的对象
权限(permission):授予的权限
规则
1zooKeeper的权限控制是基于每个znode节点的,需要对每个节点设置权限
2每个znode支持设置多种权限控制方案和多个权限
3子节点不会继承父节点的权限,客户端无权访问某节点,但可能可以访问它的子节点
权限模式scheme
方案 描述
world 只有一个用户:anyone,代表登录zokeeper所有人(默认)
ip 对客户端使用IP地址认证
auth 使用已添加认证的用户认证---可以是明文密码
digest 使用“用户名:密码”方式认证--密文密码
授权对象:
授权对象ID是指,权限赋予的实体,
例如:IP 地址或用户或者anyone
授予的权限
create
delete
read
writer
admin
也就是 增、删、改、查、管理权限,这5种权限简写为cdrwa,
注意:这5种权限中,delete是指对子节点的删除权限,其它4种权限指对自身节点的操作权限
授权相关命令:
命令格式:
setAcl 节点 授权模式:授权对象:授权权限
练习:
word模式
新建节点 [zk: localhost:2181(CONNECTED) 5] create /node1 "node1" Created /node1 查看默认的权限 [zk: localhost:2181(CONNECTED) 6] getAcl /node1 'world,'anyone : cdrwa 新创建的节点的子节点 [zk: localhost:2181(CONNECTED) 7] create /node1/node11 "node11" Created /node1/node11 [zk: localhost:2181(CONNECTED) 8] create /node1/node22 "node22" Created /node1/node22 [zk: localhost:2181(CONNECTED) 9] ls /node1 [node11, node22] 修改节点权限为不能添加子节点 [zk: localhost:2181(CONNECTED) 10] setAcl /node1 world:anyone:drwa cZxid = 0x20 ctime = Sat Sep 05 16:11:38 CST 2020 mZxid = 0x20 mtime = Sat Sep 05 16:11:38 CST 2020 pZxid = 0x22 cversion = 2 dataVersion = 0 aclVersion = 1 ephemeralOwner = 0x0 dataLength = 5 numChildren = 2 [zk: localhost:2181(CONNECTED) 11] getAcl /node1 'world,'anyone : drwa 添加子节点失败 [zk: localhost:2181(CONNECTED) 12] create /node1/node33 "node33" Authentication is not valid : /node1/node33 [zk: localhost:2181(CONNECTED) 13]
IP授权模式
格式:
setAcl <path> ip:<ip>:<acl>
注意:zkCli.sh -server ip
Auth授权模式
addauth digest <user>:<password> #添加认证用户
setAcl <path> auth:<user>:<acl>
注意:必须先进行授权认证用户的添加
[zk: localhost:2181(CONNECTED) 13] create /node3 "node3" Created /node3 [zk: localhost:2181(CONNECTED) 14] getAcl /node3 'world,'anyone : cdrwa 添加认证用户密码 [zk: localhost:2181(CONNECTED) 15] addauth digest quan:1234 设置acl auth模式的权限 [zk: localhost:2181(CONNECTED) 16] setAcl /node3 auth:quan:cdrwa cZxid = 0x26 ctime = Sat Sep 05 16:27:56 CST 2020 mZxid = 0x26 mtime = Sat Sep 05 16:27:56 CST 2020 pZxid = 0x26 cversion = 0 dataVersion = 0 aclVersion = 1 ephemeralOwner = 0x0 dataLength = 5 numChildren = 0 查看权限 [zk: localhost:2181(CONNECTED) 17] getAcl /node3 'digest,'quan:oEBCQutcK+DX5snUyMCk9GdEdbY= : cdrwa
[zk: localhost:2181(CONNECTED) 0] get /node3 Authentication is not valid : /node3 [zk: localhost:2181(CONNECTED) 1] addauth digest quan:12345 [zk: localhost:2181(CONNECTED) 2] get /node3 Authentication is not valid : /node3 [zk: localhost:2181(CONNECTED) 3] addauth digest quan:123 [zk: localhost:2181(CONNECTED) 4] get /node3 Authentication is not valid : /node3 需要添加正确的账号密码才能查看 [zk: localhost:2181(CONNECTED) 5] addauth digest quan:1234 [zk: localhost:2181(CONNECTED) 6] get /node3 node3 cZxid = 0x26 ctime = Sat Sep 05 16:27:56 CST 2020 mZxid = 0x26 mtime = Sat Sep 05 16:27:56 CST 2020 pZxid = 0x26 cversion = 0 dataVersion = 0 aclVersion = 1 ephemeralOwner = 0x0 dataLength = 5 numChildren = 0
Digest授权模式
格式:
setAcl <path> digest:<user>:<password>:<acl>
注意:密码是经过SHA1及BASE64处理的密文,在SHELL中可以通过以下命令计算
echo -n <user>:<password> | openssl dgst -binary -sha1 | openssl base64
[root@VM-0-10-centos data]# echo -n quan:1234 |openssl dgst -binary -sha1 |openssl base64 oEBCQutcK+DX5snUyMCk9GdEdbY=
授权
[zk: localhost:2181(CONNECTED) 7] create /node4 "node4" Created /node4 [zk: localhost:2181(CONNECTED) 8] getAcl /node4 'world,'anyone : cdrwa [zk: localhost:2181(CONNECTED) 9] setAcl /node4 digest:quan:oEBCQutcK+DX5snUyMCk9GdEdbY=:cdrwa cZxid = 0x2a ctime = Sat Sep 05 16:38:27 CST 2020 mZxid = 0x2a mtime = Sat Sep 05 16:38:27 CST 2020 pZxid = 0x2a cversion = 0 dataVersion = 0 aclVersion = 1 ephemeralOwner = 0x0 dataLength = 5 numChildren = 0 [zk: localhost:2181(CONNECTED) 10] getAcl /node4 'digest,'quan:oEBCQutcK+DX5snUyMCk9GdEdbY= : cdrwa
多种模式授权
[zk: localhost:2181(CONNECTED) 25] getAcl /node555 Node does not exist: /node555 [zk: localhost:2181(CONNECTED) 26] create /node555 "node555" Created /node555 [zk: localhost:2181(CONNECTED) 27] getAcl /node555 'world,'anyone : cdrwa [zk: localhost:2181(CONNECTED) 28] setAcl /node555 ip:192.168.1.120:cdre,auth:qq:cdrwa,digest:qq:oEBCQu Unknown perm type: e cZxid = 0x32 ctime = Sat Sep 05 16:45:23 CST 2020 mZxid = 0x32 mtime = Sat Sep 05 16:45:23 CST 2020 pZxid = 0x32 cversion = 0 dataVersion = 0 aclVersion = 1 ephemeralOwner = 0x0 dataLength = 7 numChildren = 0 [zk: localhost:2181(CONNECTED) 29] getAcl /node555 'ip,'192.168.1.120 : cdr 'digest,'qq:oEBCQutcK+DX5snUyMCk9GdEdbY= : cdrwa 'digest,'quan:M593vXym1b1qBYmZzb9KjzJUYPY= : cdrwa 'digest,'quan:99P5xbaTA/8xWBtZZ6O50EznJ2g= : cdrwa 'digest,'quan:oEBCQutcK+DX5snUyMCk9GdEdbY= : cdrwa 'digest,'quan:Ux4TTJ10q7ejbdzki3X+T/mIkBY= : cdrwa 'digest,'qq:3CK5BvP+nSNTC8vS4NgrdptCQdk= : cdrwa [zk: localhost:2181(CONNECTED) 30]
ACL超级管理园:
在zkServer.sh文件的nohup哪一行的\号之前加入
"- Dzookeeper.DigestAuthenticationProvider.superDigest=super:xQJmxLMiHGwaqBv st5y6rkB6HQs="
行号的密码自己用上面的生成
addauth digest super:admin #添加认证用户