RestTemplate往header里添加Host不生效问题

一、背景

对接一个外部系统，要求在header里写入Host来调他们不同的环境。

以为是一个普通的请求，就正常通过restTemplate设置header中的host：

        HttpHeaders headers = new HttpHeaders();
        headers.setContentType(MediaType.APPLICATION_JSON);
        headers.setHost(InetSocketAddress.createUnresolved(meshHeader, 0));
        HttpEntity<String> httpEntity = new HttpEntity<>(JSON.toJSONString(checkQo), headers);
        RestTemplate restTemplate = new RestTemplate();
        ResponseEntity<String> responseEntity = restTemplate.postForEntity(url, httpEntity, String.class);

二、问题

发现请求始终报404，通过postman调用发现，设置了host后能正常访问，不设置就报404，所以就怀疑是header中的host没设置进去。

2.1 通过fiddler抓包：

1) fiddler正常启动；

2) idea中配置上启动参数（2776是我自定义的fiddler端口）

-Dhttp.proxyHost=127.0.0.1
-Dhttp.proxyPort=2776

3) 通过fiddler抓包发现请求中的header完全没有设置进去。

2.2 通过wireshark抓包：

1) 启动wireshark，过滤请求ip.dst eq xxx.xxx.xxx.xxx || http 查看 Hypertext Transfer Protocol 里的 Host，确实也不对；

三、解决

google搜了一下，需要加上这个配置：

System.setProperty("sun.net.http.allowRestrictedHeaders", "true");

或者在启动参数里加上： 

-Dsun.net.http.allowRestrictedHeaders=true

 

但是这两个我都加了，还是没有Host，所以就去看了下源码

四、排查

1) 从 restTemplate.postForEntity(url, httpEntity, String.class); 入手，直接看源码：

	// 1.进入postForEntity
	@Override
	public <T> ResponseEntity<T> postForEntity(String url, @Nullable Object request,
			Class<T> responseType, Object... uriVariables) throws RestClientException {

		RequestCallback requestCallback = httpEntityCallback(request, responseType);
		ResponseExtractor<ResponseEntity<T>> responseExtractor = responseEntityExtractor(responseType);
		return nonNull(execute(url, HttpMethod.POST, requestCallback, responseExtractor, uriVariables));
	}

	// 2.进入execute
	@Override
	@Nullable
	public <T> T execute(String url, HttpMethod method, @Nullable RequestCallback requestCallback,
			@Nullable ResponseExtractor<T> responseExtractor, Object... uriVariables) throws RestClientException {

		URI expanded = getUriTemplateHandler().expand(url, uriVariables);
		return doExecute(expanded, method, requestCallback, responseExtractor);
	}

	// 3.进入doExecute
	@Nullable
	protected <T> T doExecute(URI url, @Nullable HttpMethod method, @Nullable RequestCallback requestCallback,
			@Nullable ResponseExtractor<T> responseExtractor) throws RestClientException {

		Assert.notNull(url, "URI is required");
		Assert.notNull(method, "HttpMethod is required");
		ClientHttpResponse response = null;
		try {
			ClientHttpRequest request = createRequest(url, method);
			if (requestCallback != null) {
				requestCallback.doWithRequest(request);
			}
            // 4.这里开始执行请求
			response = request.execute();
			handleResponse(url, method, response);
			return (responseExtractor != null ? responseExtractor.extractData(response) : null);
		}
		catch (IOException ex) {
			String resource = url.toString();
			String query = url.getRawQuery();
			resource = (query != null ? resource.substring(0, resource.indexOf('?')) : resource);
			throw new ResourceAccessException("I/O error on " + method.name() +
					" request for \"" + resource + "\": " + ex.getMessage(), ex);
		}
		finally {
			if (response != null) {
				response.close();
			}
		}
	}

2) 上面的 request.execute() 是直接进入了 AbstractClientHttpRequest 这个抽象类的 execute()：

    @Override
	public final ClientHttpResponse execute() throws IOException {
		assertNotExecuted();
		ClientHttpResponse result = executeInternal(this.headers);
		this.executed = true;
		return result;
	}
    
    // 是个抽象方法
    protected abstract ClientHttpResponse executeInternal(HttpHeaders headers) throws IOException;

3) 上面的抽象方法实际是进入了 AbstractBufferingClientHttpRequest 这个抽象类：

	// 1.进入executeInternal
	@Override
	protected ClientHttpResponse executeInternal(HttpHeaders headers) throws IOException {
		byte[] bytes = this.bufferedOutput.toByteArray();
		if (headers.getContentLength() < 0) {
			headers.setContentLength(bytes.length);
		}
        // 2.进入executeInternal
		ClientHttpResponse result = executeInternal(headers, bytes);
		this.bufferedOutput = new ByteArrayOutputStream(0);
		return result;
	}

	@Override
	protected ClientHttpResponse executeInternal(HttpHeaders headers, byte[] bufferedOutput) throws IOException {
		// 3.关键：在这里加入header
		addHeaders(this.connection, headers);
		// JDK <1.8 doesn't support getOutputStream with HTTP DELETE
		if (getMethod() == HttpMethod.DELETE && bufferedOutput.length == 0) {
			this.connection.setDoOutput(false);
		}
		if (this.connection.getDoOutput() && this.outputStreaming) {
			this.connection.setFixedLengthStreamingMode(bufferedOutput.length);
		}
		this.connection.connect();
		if (this.connection.getDoOutput()) {
			FileCopyUtils.copy(bufferedOutput, this.connection.getOutputStream());
		}
		else {
			// Immediately trigger the request in a no-output scenario as well
			this.connection.getResponseCode();
		}
		return new SimpleClientHttpResponse(this.connection);
	}

	static void addHeaders(HttpURLConnection connection, HttpHeaders headers) {
		String method = connection.getRequestMethod();
		if (method.equals("PUT") || method.equals("DELETE")) {
			if (!StringUtils.hasText(headers.getFirst(HttpHeaders.ACCEPT))) {
				// Avoid "text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2"
				// from HttpUrlConnection which prevents JSON error response details.
				headers.set(HttpHeaders.ACCEPT, "*/*");
			}
		}
		headers.forEach((headerName, headerValues) -> {
			if (HttpHeaders.COOKIE.equalsIgnoreCase(headerName)) {  // RFC 6265
				String headerValue = StringUtils.collectionToDelimitedString(headerValues, "; ");
				connection.setRequestProperty(headerName, headerValue);
			}
			else {
				for (String headerValue : headerValues) {
					String actualHeaderValue = headerValue != null ? headerValue : "";
					// 4.关键：在这里往 HttpURLConnection 中添加请求参数
					connection.addRequestProperty(headerName, actualHeaderValue);
				}
			}
		});
	}

4) 进入  HttpURLConnection 中看看是怎么 addRequestProperty 的：

    public synchronized void addRequestProperty(String var1, String var2) {
        if (!this.connected && !this.connecting) {
            if (var1 == null) {
                throw new NullPointerException("key is null");
            } else {
                // 关键：有个判断，如果没通过就会跳过这个header设置，应该就是这里了（通过名称也能判断出来：该外部的header是否是被允许的）
                if (this.isExternalMessageHeaderAllowed(var1, var2)) {
                    this.requests.add(var1, var2);
                    if (!var1.equalsIgnoreCase("Content-Type")) {
                        this.userHeaders.add(var1, var2);
                    }
                }

            }
        } else {
            throw new IllegalStateException("Already connected");
        }
    }

    private boolean isExternalMessageHeaderAllowed(String var1, String var2) {
        this.checkMessageHeader(var1, var2);
        // 关键校验，从名称就能看出来是判断header是否是被限制的header，如果是限制的添加不进去
        return !this.isRestrictedHeader(var1, var2);
    }

    private boolean isRestrictedHeader(String var1, String var2) {
        // 关键1
        if (allowRestrictedHeaders) {
            return false;
        } else {
            var1 = var1.toLowerCase();
            // 关键2
            if (restrictedHeaderSet.contains(var1)) {
                return !var1.equals("connection") || !var2.equalsIgnoreCase("close");
            } else {
                return var1.startsWith("sec-");
            }
        }
    }

通过关键1 的 allowRestrictedHeaders搜索可以看到 ：

    .....
    private static final boolean allowRestrictedHeaders;
    private static final Set<String> restrictedHeaderSet;
    private static final String[] restrictedHeaders = new String[]{"Access-Control-Request-Headers", "Access-Control-Request-Method", "Connection", "Content-Length", "Content-Transfer-Encoding", "Host", "Keep-Alive", "Origin", "Trailer", "Transfer-Encoding", "Upgrade", "Via"};
    .....
        
    static {
        ......
        allowRestrictedHeaders = (Boolean)AccessController.doPrivileged(new GetBooleanAction("sun.net.http.allowRestrictedHeaders"));
        if (!allowRestrictedHeaders) {
            restrictedHeaderSet = new HashSet(restrictedHeaders.length);

            for(int var2 = 0; var2 < restrictedHeaders.length; ++var2) {
                restrictedHeaderSet.add(restrictedHeaders[var2].toLowerCase());
            }
        } else {
            restrictedHeaderSet = null;
        }
        ......
    }

所以通过设置 sun.net.http.allowRestrictedHeaders=true 可以实现自定义Host

同时通过 关键2 的 restrictedHeaderSet 可以看到具体有哪些限制的header。

 

五、遗留问题

虽然我都加了 sun.net.http.allowRestrictedHeaders=true 这个配置，也看到了HttpURLConnection 中正确设置了host，但最后抓包的请求中还是没有自定义的host，被改成了url中的ip。

继续排查：

写了一个 main 方法，抓包发送的请求是带了自定义的host，因此怀疑是 spring 哪里篡改了。

这里留个坑待补......

 

六、临时解决

刚开始打算使用 okHttp 发送 post 请求，但发现通过 main 来发送没问题，但集成到 Spring 中发送的请求 header 还是不对。

最后使用 httpclient 包中的 HttpPost 进行发送，临时解决了这个问题。