HTTP的GET请求RUL保留字处理方式（多个解决方案）

背景：

最近花了一天在处理一个生产环境问题，

客户端（发送数据）：通过HTTP的GET请求，传输参数中带有“+”加号。

服务端（接收数据）：“+”加号变为空格。

因为是签名数据，导致服务端验证签名不通过，算比较严重的问题。

 

解决问题示例（多个解决方案）：

示例1（请求url的参数采用直接拼装的方式）（失败）：

package com.qhfax.test;

import com.qhfax.common.util.HttpClientUtil;

public class HttpGetTest1 {

    public static void main(String[] args) throws Exception {
        //签名
        String sign = "abcde+fghij";
        //请求的服务地址
        String serviceUrl = "http://localhost:8080/qhfaxWeb/httpGetTest/paramTest";//请求路径
        //请求参数
        String queryString = "?sign="+sign;
        //URI编码处理
        String getUrl = serviceUrl + queryString;
        String response = HttpClientUtil.get(getUrl);
        System.out.println(response);
    }
}

服务端接收的的值为：abcde fghij

服务端返回：false

 

示例2（使用URIUtil.encodeQuery方法对请求参数进行编码）（失败）

package com.qhfax.test;

import org.apache.commons.httpclient.util.URIUtil;

import com.qhfax.common.util.HttpClientUtil;

public class HttpGetTest2 {

    public static void main(String[] args) throws Exception {
        //签名
        String sign = "abcde+fghij";
        //请求的服务地址
        String serviceUrl = "http://localhost:8080/qhfaxWeb/httpGetTest/paramTest";//请求路径
        //请求参数
        String queryString = "?sign="+sign;
        //URI编码处理
        String getUrl = serviceUrl + URIUtil.encodeQuery(queryString);
        String response = HttpClientUtil.get(getUrl);
        System.out.println(response);
    }
}

 

服务端接收的的值为：abcde fghij

服务端返回：false

 

示例3（对URL保留字符进行ASCII码转换，把“+”替换为“%2B”）（成功）

package com.qhfax.test;

import com.qhfax.common.util.HttpClientUtil;

public class HttpGetTest3 {

    public static void main(String[] args) throws Exception {
        //签名
        String sign = "abcde+fghij";
        //把“+”替换为“%2B”
        sign = sign.replaceAll("\\+", "%2B");
        //请求的服务地址
        String serviceUrl = "http://localhost:8080/qhfaxWeb/httpGetTest/paramTest";//请求路径
        //请求参数
        String queryString = "?sign="+sign;
        //URI编码处理
        String getUrl = serviceUrl + queryString;
        String response = HttpClientUtil.get(getUrl);
        System.out.println(response);
    }
}

服务端接收的的值为：abcde+fghij

服务端返回：true

 

示例4（基于示例3，使用java自带的URLEncoder.encode方法对参数进行编码）（成功）

package com.qhfax.test;

import com.qhfax.common.util.HttpClientUtil;

public class HttpGetTest4 {

    public static void main(String[] args) throws Exception {
        //签名
        String sign = "abcde+fghij";
        //使用java自带的URLEncoder.encode方法对参数进行编码
        sign = java.net.URLEncoder.encode(sign);
        //请求的服务地址
        String serviceUrl = "http://localhost:8080/qhfaxWeb/httpGetTest/paramTest";//请求路径
        //请求参数
        String queryString = "?sign="+sign;
        //URI编码处理
        String getUrl = serviceUrl + queryString;
        String response = HttpClientUtil.get(getUrl);
        System.out.println(response);
    }
}

服务端接收的的值为：abcde+fghij

服务端返回：true

 

示例5（基于示例3、4，使用httpClient包中的UrlEncodedFormEntity类，详见上面的HttpClientUtil工具类）（最终版本）（成功）

package com.qhfax.test;

import java.util.HashMap;
import java.util.Map;

import com.qhfax.common.util.HttpClientUtil;

public class HttpGetTest5 {

    public static void main(String[] args) throws Exception {
        //签名
        String sign = "abcde+fghij";
        //请求的服务地址
        String serviceUrl = "http://localhost:8080/qhfaxWeb/httpGetTest/paramTest";//请求路径
        //请求参数
        Map<String, String> params = new HashMap<String, String>();
        params.put("sign", sign);
        String response = HttpClientUtil.get(serviceUrl, params);
        System.out.println(response);
    }
}

服务端接收的的值为：abcde+fghij

服务端返回：true

 

示例6（换一种方式，也是Httpclientjar包中的类，使用Request.Get）（成功）

 

package com.qhfax.test;

import org.apache.http.client.fluent.Request;
import org.apache.http.client.utils.URIBuilder;

public class HttpGetTest6 {

    public static void main(String[] args) throws Exception {
        //签名
        String sign = "abcde+fghij";
        //请求的服务地址
        String serviceUrl = "http://localhost:8080/qhfaxWeb/httpGetTest/paramTest";//请求路径

        String result = Request.Get(
                new URIBuilder(serviceUrl)
                    .addParameter("sign", sign)
                    .build())
            .connectTimeout(5000)
            .socketTimeout(5000).execute()
            .returnContent().asString();
        System.out.println(result);
    }
}

服务端接收的的值为：abcde+fghij

服务端返回：true

 

总结：

1、找到问题：

根据示例1中问题，HTTP的GET请求URL中有包括“! * ' ( ) ; : @ & =+ $ , / ? # [ ]”的保留字符，需要对它们进行转码，这是解决问题思路的第一步，找到原因。

2、验证解决方案：

根据示例3中的处理方式，将把“+”替换为“%2B”，验证处理方法是正确的。这是解决问题思路的第二步，尝试方案，验证能否解决问题。

3、完善解决方案：

示例4、5、6是基于示例3的完善，利用现有的工具类，更好更全面的处理问题。这是解决问题思路的第三步，完善解决方案。

（备注：示例2中的URIUtil.encodeQuery是有编码效果的，只是好像对“+”不会编码，这个点在我处理时有点误导了我的思路）

 

演示准备前需准备的代码(注：SpringMVC框架环境需自己搭建，httpclient的jar版本为4.4.1)：

客户端：

HttpClientUtil工具类代码如下：

package com.qhfax.common.util;

import java.io.IOException;
import java.io.UnsupportedEncodingException;
import java.net.SocketTimeoutException;
import java.nio.charset.Charset;
import java.security.GeneralSecurityException;
import java.security.cert.CertificateException;
import java.security.cert.X509Certificate;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import java.util.Map.Entry;
import java.util.Set;

import javax.net.ssl.SSLContext;
import javax.net.ssl.SSLException;
import javax.net.ssl.SSLSession;
import javax.net.ssl.SSLSocket;

import org.apache.commons.io.IOUtils;
import org.apache.commons.lang.StringUtils;
import org.apache.http.Consts;
import org.apache.http.HttpEntity;
import org.apache.http.HttpResponse;
import org.apache.http.NameValuePair;
import org.apache.http.ParseException;
import org.apache.http.client.HttpClient;
import org.apache.http.client.config.RequestConfig;
import org.apache.http.client.config.RequestConfig.Builder;
import org.apache.http.client.entity.UrlEncodedFormEntity;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.conn.ConnectTimeoutException;
import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
import org.apache.http.conn.ssl.SSLContextBuilder;
import org.apache.http.conn.ssl.TrustStrategy;
import org.apache.http.conn.ssl.X509HostnameVerifier;
import org.apache.http.entity.ContentType;
import org.apache.http.entity.StringEntity;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClientBuilder;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.impl.conn.PoolingHttpClientConnectionManager;
import org.apache.http.message.BasicNameValuePair;
import org.apache.http.protocol.HTTP;
import org.apache.http.util.EntityUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

/**
 * @Description: 使用httpclient4.0以上组件
 * @author : huangaming
 * @date : 2017年4月6日 上午11:49:48
 */
@SuppressWarnings("deprecation")
public class HttpClientUtil {

    private static Logger logger = LoggerFactory.getLogger(HttpClientUtil.class);

    public static String postJsonString(String uri, String jsonStr) {
        String result = "";
        Charset charset = Charset.forName("UTF-8");
        // 实例化http客户端
        HttpClient httpClient = HttpClientBuilder.create().build();
        HttpPost post = null;
        try {
            post = new HttpPost(uri);
            StringEntity stringEntity = new StringEntity(jsonStr, ContentType.create("application/json", charset));
            // 实例化post提交方式
            post.addHeader(HTTP.CONTENT_TYPE, ContentType.APPLICATION_JSON.toString());
            // 将参数加入post请求体中
            post.setEntity(stringEntity);
            // 执行post请求并得到返回对象 [ 到这一步我们的请求就开始了 ]
            HttpResponse resp = httpClient.execute(post);
            // 解析返回请求结果
            HttpEntity entity = resp.getEntity();
            result = IOUtils.toString(entity.getContent(), charset);
            logger.info("[postJsonString response:{}]", result);
            // 输出结果
        } catch (Exception exception) {
            logger.error("postJsonString exception", exception);
        } finally {
            if (post != null) {
                post.releaseConnection();
            }
        }
        return result;
    }
    
    public static final int connTimeout=10000;//连接超时参数
    public static final int readTimeout=10000;//读取超时参数
    public static final String charset="UTF-8";//字符编码
    private static HttpClient client = null;
    
    static {
        PoolingHttpClientConnectionManager cm = new PoolingHttpClientConnectionManager();
        cm.setMaxTotal(128);
        cm.setDefaultMaxPerRoute(128);
        client = HttpClients.custom().setConnectionManager(cm).build();
    }
    
    public static String postParameters(String url, String parameterStr) throws ConnectTimeoutException, SocketTimeoutException, Exception{
        return post(url,parameterStr,"application/x-www-form-urlencoded",charset,connTimeout,readTimeout);
    }
    
    public static String postParameters(String url, String parameterStr,String charset, Integer connTimeout, Integer readTimeout) throws ConnectTimeoutException, SocketTimeoutException, Exception{
        return post(url,parameterStr,"application/x-www-form-urlencoded",charset,connTimeout,readTimeout);
    }
    
    public static String postParameters(String url, Map<String, String> params) throws ConnectTimeoutException,  
        SocketTimeoutException, Exception {
         return postForm(url, params, null, connTimeout, readTimeout);
    }
    
    public static String postParameters(String url, Map<String, String> params, Integer connTimeout,Integer readTimeout) throws ConnectTimeoutException,  
        SocketTimeoutException, Exception {
         return postForm(url, params, null, connTimeout, readTimeout);
    }
      
    public static String get(String url) throws Exception {  
        return get(url, charset, connTimeout, readTimeout);  
    }
    
    public static String get(String url, String charset) throws Exception {  
        return get(url, charset, connTimeout, readTimeout);  
    }
    
    public static String get(String url, Map<String, String> params) throws Exception {  
        return get(url, params, charset, connTimeout, readTimeout);  
    }
    
    /** 
     * 发送一个 Post 请求, 使用指定的字符集编码. 
     *  
     * @param url 
     * @param body RequestBody 
     * @param mimeType 例如 application/xml "application/x-www-form-urlencoded" a=1&b=2&c=3
     * @param charset 编码 
     * @param connTimeout 建立链接超时时间,毫秒. 
     * @param readTimeout 响应超时时间,毫秒. 
     * @return ResponseBody, 使用指定的字符集编码. 
     * @throws ConnectTimeoutException 建立链接超时异常 
     * @throws SocketTimeoutException  响应超时 
     * @throws Exception 
     */  
    public static String post(String url, String body, String mimeType,String charset, Integer connTimeout, Integer readTimeout) 
            throws ConnectTimeoutException, SocketTimeoutException, Exception {
        HttpClient client = null;
        HttpPost post = new HttpPost(url);
        String result = "";
        try {
            if (StringUtils.isNotBlank(body)) {
                HttpEntity entity = new StringEntity(body, ContentType.create(mimeType, charset));
                
                post.setEntity(entity);
            }
            // 设置参数
            Builder customReqConf = RequestConfig.custom();
            if (connTimeout != null) {
                customReqConf.setConnectTimeout(connTimeout);
            }
            if (readTimeout != null) {
                customReqConf.setSocketTimeout(readTimeout);
            }
            post.setConfig(customReqConf.build());

            HttpResponse res;
            if (url.startsWith("https")) {
                // 执行 Https 请求.
                client = createSSLInsecureClient();
                res = client.execute(post);
            } else {
                // 执行 Http 请求.
                client = HttpClientUtil.client;
                res = client.execute(post);
            }
            result = IOUtils.toString(res.getEntity().getContent(), charset);
        } finally {
            post.releaseConnection();
            if (url.startsWith("https") && client != null&& client instanceof CloseableHttpClient) {
                ((CloseableHttpClient) client).close();
            }
        }
        return result;
    }  
    
    
    /** 
     * 提交form表单 
     *  
     * @param url 
     * @param params 
     * @param connTimeout 
     * @param readTimeout 
     * @return 
     * @throws ConnectTimeoutException 
     * @throws SocketTimeoutException 
     * @throws Exception 
     */  
    public static String postForm(String url, Map<String, String> params, Map<String, String> headers, Integer connTimeout,Integer readTimeout) throws ConnectTimeoutException,  
            SocketTimeoutException, Exception {
        HttpClient client = null;  
        HttpPost post = new HttpPost(url);  
        try {  
            if (params != null && !params.isEmpty()) {  
                List<NameValuePair> formParams = new ArrayList<org.apache.http.NameValuePair>();  
                Set<Entry<String, String>> entrySet = params.entrySet();  
                for (Entry<String, String> entry : entrySet) {  
                    formParams.add(new BasicNameValuePair(entry.getKey(), entry.getValue()));  
                }  
                UrlEncodedFormEntity entity = new UrlEncodedFormEntity(formParams, Consts.UTF_8);  
                post.setEntity(entity);  
            }
            
            if (headers != null && !headers.isEmpty()) {  
                for (Entry<String, String> entry : headers.entrySet()) {  
                    post.addHeader(entry.getKey(), entry.getValue());  
                }  
            }  
            // 设置参数  
            Builder customReqConf = RequestConfig.custom();  
            if (connTimeout != null) {  
                customReqConf.setConnectTimeout(connTimeout);  
            }  
            if (readTimeout != null) {  
                customReqConf.setSocketTimeout(readTimeout);  
            }  
            post.setConfig(customReqConf.build());  
            HttpResponse res = null;  
            if (url.startsWith("https")) {  
                // 执行 Https 请求.  
                client = createSSLInsecureClient();  
                res = client.execute(post);  
            } else {  
                // 执行 Http 请求.  
                client = HttpClientUtil.client;  
                res = client.execute(post);  
            }  
            return IOUtils.toString(res.getEntity().getContent(), "UTF-8");  
        } finally {  
            post.releaseConnection();  
            if (url.startsWith("https") && client != null  
                    && client instanceof CloseableHttpClient) {  
                ((CloseableHttpClient) client).close();  
            }  
        }  
    } 
    
    /** 
     * 发送一个 GET 请求 
     *  
     * @param url 
     * @param charset 
     * @param connTimeout  建立链接超时时间,毫秒. 
     * @param readTimeout  响应超时时间,毫秒. 
     * @return 
     * @throws ConnectTimeoutException   建立链接超时 
     * @throws SocketTimeoutException   响应超时 
     * @throws Exception 
     */  
    public static String get(String url, String charset, Integer connTimeout,Integer readTimeout) 
            throws ConnectTimeoutException,SocketTimeoutException, Exception {
        HttpClient client = null;  
        HttpGet get = new HttpGet(url);  
        String result = "";  
        try {  
            // 设置参数  
            Builder customReqConf = RequestConfig.custom();  
            if (connTimeout != null) {  
                customReqConf.setConnectTimeout(connTimeout);  
            }  
            if (readTimeout != null) {  
                customReqConf.setSocketTimeout(readTimeout);  
            }  
            get.setConfig(customReqConf.build());  
  
            HttpResponse res = null;  
            if (url.startsWith("https")) { 
                logger.info("httpClientUtil|get|执行https的get请求|开始");
                // 执行 Https 请求.  
                client = createSSLInsecureClient();  
                res = client.execute(get);  
                logger.info("httpClientUtil|get|执行https的get请求|结束");
            } else {  
                logger.info("httpClientUtil|get|执行http的get请求|开始");
                // 执行 Http 请求.  
                client = HttpClientUtil.client;  
                res = client.execute(get);  
                logger.info("httpClientUtil|get|执行http的get请求|结束");
            }  
  
            result = IOUtils.toString(res.getEntity().getContent(), charset);  
        } finally {  
            get.releaseConnection();  
            if (url.startsWith("https") && client != null && client instanceof CloseableHttpClient) {  
                ((CloseableHttpClient) client).close();  
            }  
        }  
        return result;  
    }  
    
    
    /** 
     * 从 response 里获取 charset 
     *  
     * @param ressponse 
     * @return 
     */  
    @SuppressWarnings("unused")  
    private static String getCharsetFromResponse(HttpResponse ressponse) {  
        // Content-Type:text/html; charset=GBK  
        if (ressponse.getEntity() != null  && ressponse.getEntity().getContentType() != null && ressponse.getEntity().getContentType().getValue() != null) {  
            String contentType = ressponse.getEntity().getContentType().getValue();  
            if (contentType.contains("charset=")) {  
                return contentType.substring(contentType.indexOf("charset=") + 8);  
            }  
        }  
        return null;  
    }  
    
    
    
    /**
     * 创建 SSL连接
     * @return
     * @throws GeneralSecurityException
     */
    private static CloseableHttpClient createSSLInsecureClient() throws GeneralSecurityException {
        try {
            SSLContext sslContext = new SSLContextBuilder().loadTrustMaterial(null, new TrustStrategy() {
                        public boolean isTrusted(X509Certificate[] chain,String authType) throws CertificateException {
                            return true;
                        }
                    }).build();
            SSLConnectionSocketFactory sslsf = new SSLConnectionSocketFactory(sslContext, new X509HostnameVerifier() {
                        @Override
                        public boolean verify(String arg0, SSLSession arg1) {
                            return true;
                        }
                        @Override
                        public void verify(String host, SSLSocket ssl)
                                throws IOException {
                        }
                        @Override
                        public void verify(String host, X509Certificate cert)
                                throws SSLException {
                        }
                        @Override
                        public void verify(String host, String[] cns,
                                String[] subjectAlts) throws SSLException {
                        }

                    });
            return HttpClients.custom().setSSLSocketFactory(sslsf).build();
        } catch (GeneralSecurityException e) {
            throw e;
        }
    }
    
    /**
     * @param url http://taobao.com/test.action
     * @param params 参数,编码之前的参数
     * @return
     * @throws IOException 
     * @throws UnsupportedEncodingException 
     * @throws ParseException 
     * @throws GeneralSecurityException 
     */
    public static String get(String url, Map<String, String> params,String charset, Integer connTimeout,Integer readTimeout) throws ParseException, UnsupportedEncodingException, IOException, GeneralSecurityException {
        HttpClient client = null;
        if(StringUtils.isBlank(url)){
            return null;
        }
        if(params != null && !params.isEmpty()){
            List<NameValuePair> pairs = new ArrayList<NameValuePair>(params.size());
            for(Map.Entry<String,String> entry : params.entrySet()){
                String value = entry.getValue();
                if(value != null){
                    pairs.add(new BasicNameValuePair(entry.getKey(),value));
                }
            }
            url += "?" + EntityUtils.toString(new UrlEncodedFormEntity(pairs, charset));
        }
        HttpGet httpget = new HttpGet(url);
        CloseableHttpResponse response = null;
        // 设置参数  
        Builder customReqConf = RequestConfig.custom();  
        if (connTimeout != null) {  
            customReqConf.setConnectTimeout(connTimeout);  
        }  
        if (readTimeout != null) {  
            customReqConf.setSocketTimeout(readTimeout);  
        }  
        httpget.setConfig(customReqConf.build());
        if (url.startsWith("https")) { 
            logger.info("httpClientUtil|get|执行https的get请求|开始");
            // 执行 Https 请求.  
            client = createSSLInsecureClient();  
            response = (CloseableHttpResponse) client.execute(httpget);  
            logger.info("httpClientUtil|get|执行https的get请求|结束");
        } else {  
            logger.info("httpClientUtil|get|执行http的get请求|开始");
            // 执行 Http 请求.  
            client = HttpClientUtil.client;  
            response = (CloseableHttpResponse) client.execute(httpget);  
            logger.info("httpClientUtil|get|执行http的get请求|结束");
        }
        int statusCode = response.getStatusLine().getStatusCode();
        if (statusCode != 200) {
            httpget.abort();
            throw new RuntimeException("HttpClient,error status code :" + statusCode);
        }
        HttpEntity entity = response.getEntity();
        String result = null;
        if (entity != null) {
                result = EntityUtils.toString(entity, "utf-8");
        }
        EntityUtils.consume(entity);
        response.close();
        return result;
    }
}

服务端

HttpGetTestController代码如下：

package com.qhfax.controller.test;

import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpSession;

import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.ResponseBody;

/**
 * 
 * HttpGet测试Controller
 * 
 * @author zhanghaitao
 *
 */
@Controller
@RequestMapping(value = "/httpGetTest")
public class HttpGetTestController {
	
    /**
     * 参数测试
     * 
     * @param request 请求
     * @param session 会话
     * @return
     */
    @RequestMapping(value = "/paramTest", method = RequestMethod.GET)
    @ResponseBody
    public String paramTest(HttpServletRequest request, HttpSession session) {
        //获取签名
        String sign = request.getParameter("sign");
        System.out.println(sign);
        
        //原签名
        String oldSign = "abcde+fghij";
        //验证服务端接收到的签名是否与客户端一致
        boolean isEqual = oldSign.equals(sign);
        
        return isEqual+"";
    }
}