zookeeper 实战案例分享:cruator客户端编程

上两篇介绍了zookeeper服务器端的安装和配置,今天分享下利用zookeeper客户端编程来实现配置文件的统一管理,包括文件添加、删除、更新的同步。

比如,连接数据库信息的配置文件,一般每个应用服务器代码上都会存放。某个时候如果我想添加一个新数据库用户连接,那么对应用到的服务器上配置文件都要修改一遍,对于上百台的服务器,一一修改显然不现实,这时,我们可以把配置文件统一放到zookeeper服务器上,我们只需要更改zookeeper服务器上的配置文件,然后所有应用服务器上的zookeeper客户端监听到zookeeper服务器上的配置文件被改变了,进行实时同步就可以了。

主要的结构图如下:

 

【环境介绍】:为了看得清楚,我们这次引入两台机器,分别扮演服务器端和客户端

   服务器端(liunx系统):10.126.101.153

   客户端(windows):  10.249.9.19

编程环境:java+eclipse+maven+cruator

一、启动zookeeper服务(10.126.101.153)

[root@test bin]# ./zkServer.sh start
JMX enabled by default
Using config: /home/mysql/darren/software/zookeeper-3.4.6/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED

二、编写客户端代码(java)

由于ZooKeeper自带的客户端API太底层, 程序员在使用的时候需要自己处理很多事情,更为关键的是,使用过程中如果不当会产生很多问题,如:

  • 初始化连接的问题: 在client与server之间握手建立连接的过程中, 如果握手失败, 执行所有的同步方法(比如create, getData等)将抛出异常
  • 自动恢复(failover)的问题: 当client与一台server的连接丢失,并试图去连接另外一台server时, client将回到初始连接模式
  • session过期的问题: 在极端情况下, 出现ZooKeeper session过期, 客户端需要自己去监听该状态并重新创建

为了方便我们编程和简化应用,这里使用cruator框架。关于cruator框架的一些介绍可以参考这里curator解决了zookeeper client哪些问题

使用cruator时,建议采用maven管理项目,maven可以为我们很容易的引入cruator依赖包,关于maven安装和配置参考这里。eclipse4.5版本以上一般都已经安装m2了,只需要配置即可。 

1、采用eclipse创建maven项目

New->Other->Maven->Maven Project,然后一直默认next,填写Group Id,Artifact Id,Package,点击finish即可。

2、配置pom.xml文件,引入cruator依赖包

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>

  <groupId>com.darren</groupId>
  <artifactId>zk_client</artifactId>
  <version>0.0.1-SNAPSHOT</version>
  <packaging>jar</packaging>

  <name>zk_client</name>
  <url>http://maven.apache.org</url>

  <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    </properties>

    <dependencies>

        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>3.8.1</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>com.netflix.curator</groupId>
            <artifactId>curator-recipes</artifactId>
            <version>1.3.0</version>
        </dependency>
        <dependency>
            <groupId>com.googlecode.json-simple</groupId>
            <artifactId>json-simple</artifactId>
            <version>1.1.1</version>
        </dependency>
        <dependency>
            <groupId>commons-daemon</groupId>
            <artifactId>commons-daemon</artifactId>
            <version>1.0.10</version>
        </dependency>
    </dependencies>  
</project>

3、编写zookeeper 客户端操作代码

使用cruator编写zk客户端,一般分为以下几个步骤:

 1)使用CuratorFrameworkFactory建立客户端到zookeeper服务器端的连接;

 2)创建PathChildrenCache对象,表示zk路径节点及数据

 3)为PathChildrenCache对象建立监听,这里需要实现PathChildrenCacheListener接口,用于处理监听事件逻辑。

代码如下:

 1 package com.darren.zk_client;
 2 
 3 import com.netflix.curator.framework.CuratorFramework;
 4 import com.netflix.curator.framework.CuratorFrameworkFactory;
 5 import com.netflix.curator.framework.recipes.cache.PathChildrenCache;
 6 import com.netflix.curator.framework.recipes.cache.PathChildrenCacheEvent;
 7 import com.netflix.curator.framework.recipes.cache.PathChildrenCacheListener;
 8 import com.netflix.curator.retry.ExponentialBackoffRetry;
 9 import com.netflix.curator.retry.RetryUntilElapsed;
10 import com.netflix.curator.utils.ZKPaths;
11 
12 public class App 
13 {
14     private static final String PATH = "/db/pools";
15     
16     @SuppressWarnings("resource")
17     public static void main( String[] args ) throws Exception
18     {
19         CuratorFramework client = null;
20         PathChildrenCache cache = null;
21         try{
//使用CuratorFrameworkFactory建立客户端到zookeeper服务器端的连接并启动
22 client = CuratorFrameworkFactory.newClient("127.0.0.1:2181", new RetryUntilElapsed(3000, 1000)); 23 client.start();
//创建PathChildrenCache对象,表示zk路径节点及数据
24 cache = new PathChildrenCache(client,PATH,true);
//为PathChildrenCache对象建立监听,这里需要实现PathChildrenCacheListener接口,用于处理监听事件逻辑
25 cache.getListenable().addListener(new Listener()); 26 cache.start(); 27 } catch(Exception e){ 28 29 } 30 //表示该java程序一直运行,如果没有,每次java启动后就自动退出了 31 while (true) { 32 Thread.sleep(500); 33 } 34 } 35 36 //内部类,用户实现监听接口 37 private static class Listener implements PathChildrenCacheListener{ 38 39 @Override 40 public void childEvent(CuratorFramework client, PathChildrenCacheEvent event) throws Exception { 41 // TODO Auto-generated method stub 42 switch (event.getType()){ 43 case CHILD_ADDED:{ 44 System.out.println("Node added: " + ZKPaths.getNodeFromPath(event.getData().getPath())); 45 break; 46 } 47 case CHILD_UPDATED:{ 48 System.out.println("Node changed: " + ZKPaths.getNodeFromPath(event.getData().getPath())); 49 break; 50 } 51 case CHILD_REMOVED:{ 52 System.out.println("Node removed: " + ZKPaths.getNodeFromPath(event.getData().getPath())); 53 break; 54 } 55 default: 56 System.out.println("no node changed!!!"); 57 break; 58 } 59 } 60 61 } 62 63 }

运行效果:

2016-12-01 16:52:42,394 INFO  com.netflix.curator.framework.imps.CuratorFrameworkImpl - Starting
2016-12-01 16:52:42,482 INFO  org.apache.zookeeper.ZooKeeper - Client environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52 GMT
2016-12-01 16:52:42,482 INFO  org.apache.zookeeper.ZooKeeper - Client environment:host.name=507B9D97E083.anjuke.net
2016-12-01 16:52:42,482 INFO  org.apache.zookeeper.ZooKeeper - Client environment:java.version=1.7.0_79
2016-12-01 16:52:42,482 INFO  org.apache.zookeeper.ZooKeeper - Client environment:java.vendor=Oracle Corporation
2016-12-01 16:52:42,482 INFO  org.apache.zookeeper.ZooKeeper - Client environment:java.home=C:\Program Files\Java\jdk1.7.0_79\jre
2016-12-01 16:52:42,482 INFO  org.apache.zookeeper.ZooKeeper - Client environment:java.class.path=E:\workdir\java\zk_client\target\classes;E:\workdir\java\repo\com\netflix\curator\curator-recipes\1.3.0\curator-recipes-1.3.0.jar;E:\workdir\java\repo\com\google\guava\guava\11.0.1\guava-11.0.1.jar;E:\workdir\java\repo\com\google\code\findbugs\jsr305\1.3.9\jsr305-1.3.9.jar;E:\workdir\java\repo\com\netflix\curator\curator-framework\1.3.0\curator-framework-1.3.0.jar;E:\workdir\java\repo\com\netflix\curator\curator-client\1.3.0\curator-client-1.3.0.jar;E:\workdir\java\repo\org\apache\zookeeper\zookeeper\3.4.5\zookeeper-3.4.5.jar;E:\workdir\java\repo\org\slf4j\slf4j-api\1.6.1\slf4j-api-1.6.1.jar;E:\workdir\java\repo\org\slf4j\slf4j-log4j12\1.6.1\slf4j-log4j12-1.6.1.jar;E:\workdir\java\repo\log4j\log4j\1.2.15\log4j-1.2.15.jar;E:\workdir\java\repo\javax\mail\mail\1.4\mail-1.4.jar;E:\workdir\java\repo\javax\activation\activation\1.1\activation-1.1.jar;E:\workdir\java\repo\jline\jline\0.9.94\jline-0.9.94.jar;E:\workdir\java\repo\org\jboss\netty\netty\3.2.2.Final\netty-3.2.2.Final.jar;E:\workdir\java\repo\com\googlecode\json-simple\json-simple\1.1.1\json-simple-1.1.1.jar;E:\workdir\java\repo\commons-daemon\commons-daemon\1.0.10\commons-daemon-1.0.10.jar
2016-12-01 16:52:42,483 INFO  org.apache.zookeeper.ZooKeeper - Client environment:java.library.path=C:\Program Files\Java\jdk1.7.0_79\bin;C:\Windows\Sun\Java\bin;C:\Windows\system32;C:\Windows;C:/Program Files/Java/jdk1.7.0_79/bin/../jre/bin/server;C:/Program Files/Java/jdk1.7.0_79/bin/../jre/bin;C:/Program Files/Java/jdk1.7.0_79/bin/../jre/lib/amd64;C:\Program Files\Java\jdk1.7.0_79\bin;C:\Program Files\Java\jdk1.7.0_79\jre\bin;C:\Perl64\site\bin;C:\Perl64\bin;C:\Python27\;C:\Python27\Scripts;C:\Program Files (x86)\Intel\iCLS Client\;C:\Program Files\Intel\iCLS Client\;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Program Files\Intel\Intel(R) Management Engine Components\DAL;C:\Program Files (x86)\Intel\Intel(R) Management Engine Components\DAL;C:\Program Files\Intel\Intel(R) Management Engine Components\IPT;C:\Program Files (x86)\Intel\Intel(R) Management Engine Components\IPT;C:\Program Files (x86)\MIT\Kerberos\bin;C:\CygwinPortable\App\Cygwin\bin;C:\Program Files\TortoiseGit\bin;C:\Program Files\Git\cmd;D:\Program files\apache-maven-3.3.9\bin;D:\Program files\eclipse;;.
2016-12-01 16:52:42,483 INFO  org.apache.zookeeper.ZooKeeper - Client environment:java.io.tmpdir=C:\Users\HAILIA~1\AppData\Local\Temp\
2016-12-01 16:52:42,483 INFO  org.apache.zookeeper.ZooKeeper - Client environment:java.compiler=<NA>
2016-12-01 16:52:42,483 INFO  org.apache.zookeeper.ZooKeeper - Client environment:os.name=Windows 7
2016-12-01 16:52:42,484 INFO  org.apache.zookeeper.ZooKeeper - Client environment:os.arch=amd64
2016-12-01 16:52:42,484 INFO  org.apache.zookeeper.ZooKeeper - Client environment:os.version=6.1
2016-12-01 16:52:42,484 INFO  org.apache.zookeeper.ZooKeeper - Client environment:user.name=hailiangzhao
2016-12-01 16:52:42,484 INFO  org.apache.zookeeper.ZooKeeper - Client environment:user.home=C:\Users\hailiangzhao
2016-12-01 16:52:42,484 INFO  org.apache.zookeeper.ZooKeeper - Client environment:user.dir=E:\workdir\java\zk_client
2016-12-01 16:52:42,486 INFO  org.apache.zookeeper.ZooKeeper - Initiating client connection, connectString=10.126.101.153:2181 sessionTimeout=60000 watcher=com.netflix.curator.ConnectionState@271816d1
2016-12-01 16:52:42,718 INFO  org.apache.zookeeper.ClientCnxn - Opening socket connection to server tjtx-101-153.58os.org/10.126.101.153:2181. Will not attempt to authenticate using SASL (unknown error)
2016-12-01 16:52:42,752 INFO  org.apache.zookeeper.ClientCnxn - Socket connection established to tjtx-101-153.58os.org/10.126.101.153:2181, initiating session
2016-12-01 16:52:42,796 INFO  org.apache.zookeeper.ClientCnxn - Session establishment complete on server tjtx-101-153.58os.org/10.126.101.153:2181, sessionid = 0x158b95fadec0001, negotiated timeout = 40000
2016-12-01 16:52:42,801 INFO  com.netflix.curator.framework.state.ConnectionStateManager - State change: CONNECTED
Node added: darren.php

上面的控制台信息表示client已经与10.126.101.153:2181 zookeeper服务端建立了连接,并且对服务器上的/db/pools节点进行了监控,只要这下面的数据有任何变化,都会输出变化的node和内容。

这时,我们可以在服务器端进行如下操作:

[zk: 10.126.101.153:2181(CONNECTED) 5] create /db/pools/darren1.php "darren1.php"
Created /db/pools/darren1.php
[zk: 10.126.101.153:2181(CONNECTED) 6] set  /db/pools/darren1.php "xxxxxx"       
cZxid = 0x7
ctime = Thu Dec 01 16:58:09 CST 2016
mZxid = 0x8
mtime = Thu Dec 01 16:58:30 CST 2016
pZxid = 0x7
cversion = 0
dataVersion = 1
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 8
numChildren = 0
[zk: 10.126.101.153:2181(CONNECTED) 7] 

这时客户端控制台追加输出,表示已经监控到了darren1.php内容改变:

Node added: darren1.php
Node changed: darren1.php

okay,使用cruator框架编写zk客户端是不是很容易呢。

当然,这个小例子很简单,还没有进行配置文件的同步,我们可以对以上程序进行如下几个改进即可实现:

1、在处理监听事件的内部类里,我们可以对变化的节点内容写入本地文件中,实现了文件同步。

2、使用daemon接口,把该java程序封装成windows或者linux服务,永久运行,保证实时同步文件。

 

遇到的错误:

  【Multiple markers at this line @Override的解决方法】http://blog.csdn.net/mazhaojuan/article/details/28931375
  【maven编译时出现读取XXX时出错invalid LOC header (bad signature)】http://blog.csdn.net/woshixuye/article/details/14669929
  【log4j:WARN No appenders could be found for loggerhttp://www.cnblogs.com/jbelial/archive/2012/06/05/2536814.html 

【moven参考资料】

  【maven安装配置】http://jingyan.baidu.com/article/1709ad808ad49f4634c4f00d.html

  【maven profile配置】http://elim.iteye.com/blog/1900568 
  【maven处理资源文件】http://www.cnblogs.com/now-fighting/p/4888343.html 
  【maven pom.xml配置说明】http://www.cnblogs.com/qq78292959/p/3711501.html

【java参考资料】

  【java api官方文档】http://docs.oracle.com/javase/7/docs/api/                    
  【java程序以服务方式运行】http://blog.csdn.net/jason5186/article/details/9146167 

【cruator参考资料】

  【cruator简介】http://macrochen.iteye.com/blog/1366136/
  【cruator java api】http://curator.apache.org/apidocs/index.html
  【zookeeper客户端编程API】http://zookeeper.apache.org/doc/r3.4.6/javaExample.html
  【cruator 案例】http://blog.sina.com.cn/s/blog_616e189f01018axz.html




 

posted @ 2016-12-01 17:22  茁壮的小草  阅读(1609)  评论(0编辑  收藏  举报