httpclient的使用(三)

一、HTTP链接管理

---

HttpClient中有一个链接管理者(HttpClientConnectionManager)来管理链接，它使用工厂模式来产生新链接、管理链接的生命周期和同步对长链接的使用。其内部有个ManagedHttpClientConnection(采用代理模式)来处理真正的链接和控制I/O操作。如果链接被释放或者被使用主动关闭，底层的链接将会从代理中去掉并返回给管理者。下面是个例子来获得一个链接

HttpClientContext context = HttpClientContext.create();
HttpClientConnectionManager connMrg = new BasicHttpClientConnectionManager();
HttpRoute route = new HttpRoute(new HttpHost("localhost", 80));
// 请求一个新的链接
ConnectionRequest connRequest = connMrg.requestConnection(route, null);
// 等待10s
HttpClientConnection conn = connRequest.get(10, TimeUnit.SECONDS);
try {
    // 链接没有被打开
    if (!conn.isOpen()) {
        // 通过路由信息建立链接
        connMrg.connect(conn, route, 1000, context);
        // 标记路由完成
        connMrg.routeComplete(conn, route, context);
    }
    // TODO
} finally {
    connMrg.releaseConnection(conn, null, 1, TimeUnit.MINUTES);
}

二、HTTP链接池使用

---

 在链接池有许多建立好的长链接，当需要发起请求时可以从链接池里直接获取链接，而不是马上创建一个新的链接。

PoolingHttpClientConnectionManager cm = new PoolingHttpClientConnectionManager();
// 最大链接数为200
cm.setMaxTotal(200);
// 每个路由上的最大链接数为20
cm.setDefaultMaxPerRoute(20);
// 到目标主机的最大链接数为50
HttpHost host = new HttpHost("targetHost", 80);
cm.setMaxPerRoute(new HttpRoute(host), 50);
CloseableHttpClient httpClient = HttpClients.custom()
        .setConnectionManager(cm)
        .build();

三、多线程的请求执行

---

 在使用多线程发起请求时，可以通过链接池来实现

PoolingHttpClientConnectionManager cm = new PoolingHttpClientConnectionManager();
CloseableHttpClient httpClient = HttpClients.custom()
        .setConnectionManager(cm)
        .build();
// URIs to perform GETs on
String[] urisToGet = {
    "http://www.domain1.com/",
    "http://www.domain2.com/",
    "http://www.domain3.com/",
    "http://www.domain4.com/"
};
// create a thread for each URI
GetThread[] threads = new GetThread[urisToGet.length];
for (int i = 0; i < threads.length; i++) {
    HttpGet httpget = new HttpGet(urisToGet[i]);
    threads[i] = new GetThread(httpClient, httpget);
}
// start the threads
for (int j = 0; j < threads.length; j++) {
    threads[j].start();
}
// join the threads
for (int j = 0; j < threads.length; j++) {
    threads[j].join();
}
static class GetThread extends Thread {
    private final CloseableHttpClient httpClient;
    private final HttpContext context;
    private final HttpGet httpget;
    public GetThread(CloseableHttpClient httpClient, HttpGet httpget) {
        this.httpClient = httpClient;
        this.context = HttpClientContext.create();
        this.httpget = httpget;
    }
    @Override
    public void run() {
        try {
            CloseableHttpResponse response = httpClient.execute(
                    httpget, context);
            try {
                HttpEntity entity = response.getEntity();
            } finally {
                response.close();
            }
        } catch (ClientProtocolException ex) {
            // Handle protocol errors
        } catch (IOException ex) {
            // Handle I/O errors
        }
    }
}

 四、链接回收策略

---

当一个链接被释放后，这个链接有可能是新鲜的，仍能被使用，又可能是过时(服务器一端关闭了链接)的，但是链接管理者是无法知道链接状态的。一种比较可行的策略就是创建一个守护线程用户回收过期的链接。

public static class IdleConnectionMonitorThread extends Thread {
    
    private final HttpClientConnectionManager connMgr;
    private volatile boolean shutdown;
    
    public IdleConnectionMonitorThread(HttpClientConnectionManager connMgr) {
        super();
        this.connMgr = connMgr;
    }
    @Override
    public void run() {
        try {
            while (!shutdown) {
                synchronized (this) {
                    wait(5000);
                    // Close expired connections
                    connMgr.closeExpiredConnections();
                    // Optionally, close connections
                    // that have been idle longer than 30 sec
                    connMgr.closeIdleConnections(30, TimeUnit.SECONDS);
                }
            }
        } catch (InterruptedException ex) {
            // terminate
        }
    }
    
    public void shutdown() {
        shutdown = true;
        synchronized (this) {
            notifyAll();
        }
    }
    
}

五、定制自己的Keep-Alive策略

---

一些HTTP服务器在应答头中虽然使用了keep-Alive，但是出于资源的考虑，可能会在一定时间之后关闭不活跃的链接但并不会告知用户。对于这种情况，我们可以采取下面的方式来定制自己的链接

ConnectionKeepAliveStrategy myStrategy = new ConnectionKeepAliveStrategy() {
    public long getKeepAliveDuration(HttpResponse response, HttpContext context) {
        // Honor 'keep-alive' header
        HeaderElementIterator it = new BasicHeaderElementIterator(
                response.headerIterator(HTTP.CONN_KEEP_ALIVE));
        while (it.hasNext()) {
            HeaderElement he = it.nextElement();
            String param = he.getName();
            String value = he.getValue();
            if (value != null && param.equalsIgnoreCase("timeout")) {
                try {
                    return Long.parseLong(value) * 1000;
                } catch(NumberFormatException ignore) {
                }
            }
        }
        HttpHost target = (HttpHost) context.getAttribute(
                HttpClientContext.HTTP_TARGET_HOST);
        if ("www.naughty-server.com".equalsIgnoreCase(target.getHostName())) {
            // Keep alive for 5 seconds only
            return 5 * 1000;
        } else {
            // otherwise keep alive for 30 seconds
            return 30 * 1000;
        }
    }
};
CloseableHttpClient client = HttpClients.custom()
        .setKeepAliveStrategy(myStrategy)
        .build();

六、定制自己的socket链接

---

HttpClient提供了PlainConnectionSocketFactory工厂接口，可以让用户获取一个空白链接从而定义自己的链接。

HttpClientContext clientContext = HttpClientContext.create();
PlainConnectionSocketFactory sf = PlainConnectionSocketFactory.getSocketFactory();
Socket socket = sf.createSocket(clientContext);
int timeout = 1000; //ms
HttpHost target = new HttpHost("localhost");
InetSocketAddress remoteAddress = new InetSocketAddress(
        InetAddress.getByAddress(new byte[] {127,0,0,1}), 80);
sf.connectSocket(timeout, socket, target, remoteAddress, null, clientContext);

 七、定义自己的链接管理者

---

以上的套接字都是为HTTP协议服务的，如果我需要加入SSL/TSL协议的从而形成HTTPS的话，就必须对套接字再分层。HttpClient中提供了套接字分层工厂，能够对已存在套接字进行分层。下面是实例代码

ConnectionSocketFactory plainsf = <...>
LayeredConnectionSocketFactory sslsf = <...>
Registry<ConnectionSocketFactory> r = RegistryBuilder.<ConnectionSocketFactory>create()
        .register("http", plainsf)
        .register("https", sslsf)
        .build();
HttpClientConnectionManager cm = new PoolingHttpClientConnectionManager(r);
HttpClients.custom()
        .setConnectionManager(cm)
        .build();

八、SSL/TLS的定制

---

通过使用HttpClient中的SSLConnectionSocketFactory创建SSL链接,但是需要使用到SSLContext(参考JSSE以及javax.net.ssl.SSLContext文档描述)

KeyStore myTrustStore = <...>
SSLContext sslContext = SSLContexts.custom()
        .loadTrustMaterial(myTrustStore)
        .build();
SSLConnectionSocketFactory sslsf = new SSLConnectionSocketFactory(sslContext);

 九、代理配置

---

在实际应用中我们可能会用到代理来访问网页，下面三种方式介绍如何用httpClient来设置代理

方法一：最简单的方式

HttpHost proxy = new HttpHost("Host/IP", 8080);
DefaultProxyRoutePlanner routePlanner = new DefaultProxyRoutePlanner(proxy);
CloseableHttpClient httpclient = HttpClients.custom()
        .setRoutePlanner(routePlanner)
        .build();

方法二：使用JRE代理选择器

SystemDefaultRoutePlanner routePlanner = new SystemDefaultRoutePlanner(
        ProxySelector.getDefault());
CloseableHttpClient httpclient = HttpClients.custom()
        .setRoutePlanner(routePlanner)
        .build();

方法三：自定义路由策略

HttpRoutePlanner routePlanner = new HttpRoutePlanner() {
    public HttpRoute determineRoute(
            HttpHost target,
            HttpRequest request,
            HttpContext context) throws HttpException {
        return new HttpRoute(target, null,  new HttpHost("someproxy", 8080),
                "https".equalsIgnoreCase(target.getSchemeName()));
    }
};
CloseableHttpClient httpclient = HttpClients.custom()
        .setRoutePlanner(routePlanner)
        .build();
    }
}