猪小花1号

个人签名

281篇博客

最新博客

最新资源下载

编辑推荐

网易数帆开源API网关与容器云项目，让云原生生产落地“多快好网易汪源：统一负载与多云环境的“开放姿态”，才是云原生网易数帆如何用 Kubernetes“原语”搞定云原生中间件快手打新挤爆券商系统，网易数帆推出券商稳定性保障方案探索智慧校园新模式，网易有数在教育行业的实践分享金融行业大数据治理之路——数据模型篇

使用HttpAsyncClient进行异步压测

猪小花1号2018-09-17 13:26

作者：牛小宝


HttpAsyncClient简介

在HttpClient进化到4.x后，官方提供了基于nio的异步版本：HttpAsyncClient。HttpAsyncClient 是基于httpcore NIO 和httpclient基础上做的进一步封装，主要利用异步回调方式， 适用于高性能需求的场景下.当我们发送一个http请求后，当前线程会立刻返回，并执行完后续工作。此时和服务端只有connection进行连接，没有线程等待。当收到响应后，HttpAsyncClient将回调通知客户端，用此方法提高了并发能力。
利用HttpAsyncClient并发能力强，性能好，并且拓展性强等特点，可以将其性能测试，使用异步http请求来模拟洪水压力

模拟GET请求

使用HttpAsyncClient模拟GET请求只需要如下几步：

  1. 在pom.xml中引入HttpAsyncClient的jar包：
<dependency>
    <groupId>org.apache.httpcomponents</groupId>
  	<artifactId>httpasyncclient</artifactId>
  	<version>4.1.3</version>
</dependency>
  1. 创建HttpAsyncClients对象用于发起异步请求：
CloseableHttpAsyncClient httpclient = HttpAsyncClients.createDefault();
  1. 使用HttpGet构造HTTP GET请求
HttpGet request = new HttpGet("http://www.kaola.com/");
  1. 发起请求并使用FutureCallback<HttpResponse>进行请求响应的回调
示例
import java.io.IOException;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.ExecutionException;
import org.apache.http.HttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.concurrent.FutureCallback;
import org.apache.http.impl.nio.client.CloseableHttpAsyncClient;
import org.apache.http.impl.nio.client.HttpAsyncClients;


public class AsycPerfGetClient 
{
    public static void main( String[] args ) throws InterruptedException, ExecutionException
    {
    	CloseableHttpAsyncClient httpclient = HttpAsyncClients.createDefault();
    	try {
    	    // Start the client
    	    httpclient.start();

    	    // One most likely would want to use a callback for operation result
    	    final CountDownLatch latch = new CountDownLatch(1);
    	    final HttpGet request = new HttpGet("http://www.kaola.com/");
    	    httpclient.execute(request, new FutureCallback<HttpResponse>() {

    	        public void completed(final HttpResponse response2) {
    	            latch.countDown();
    	            System.out.println(request.getRequestLine() + "->" + response2.getStatusLine());
    	        }

    	        public void failed(final Exception ex) {
    	            latch.countDown();
    	            System.out.println(request.getRequestLine() + "->" + ex);
    	        }

    	        public void cancelled() {
    	            latch.countDown();
    	            System.out.println(request.getRequestLine() + " cancelled");
    	        }

    	    });
    	    latch.await();

    	} finally {
    	    try {
				httpclient.close();
			} catch (IOException e) {
				// TODO Auto-generated catch block
				e.printStackTrace();
			}
    	}
    }
}

模拟POST请求

POST请求分为两种：

  1. 参数为JSON（application/json）格式：使用StringEntity对象进行body构造；
  2. 参数为form（application/x-www-form-urlencoded）格式：使用NameValuePair进行body构造；

还可以在响应回调FutureCallback对象的completed方法中对响应体的正确性进行断言校验。

示例
import java.io.IOException;
import java.io.UnsupportedEncodingException;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.ExecutionException;
import org.apache.http.HttpResponse;
import org.apache.http.NameValuePair;
import org.apache.http.ParseException;
import org.apache.http.client.entity.UrlEncodedFormEntity;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.concurrent.FutureCallback;
import org.apache.http.entity.StringEntity;
import org.apache.http.impl.nio.client.CloseableHttpAsyncClient;
import org.apache.http.impl.nio.client.HttpAsyncClients;
import org.apache.http.message.BasicNameValuePair;
import org.apache.http.util.EntityUtils;

public class AsycPerfPostClient 
{
	/*
	 * method : Post request 
	 * Content-type : application/json
	 */
    public void doPostForJson() throws InterruptedException, ExecutionException
    {
    	CloseableHttpAsyncClient httpclient = HttpAsyncClients.createDefault();
    	try {
    	    // Start the client
    	    httpclient.start();

    	    // One most likely would want to use a callback for operation result
    	    final CountDownLatch latch = new CountDownLatch(1);
    	    final HttpPost request = new HttpPost("http://sp.kaola.com/api/cart");
    	    
    	    //construct request header
    	    request.setHeader("Content-type", "application/json; charset=UTF-8");
    	    request.setHeader("appVersion", "3.9.0");
    	    request.setHeader("apiVersion", "207");
    	    request.setHeader("platform", "1");
    	    request.setHeader("User-Agent", "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2062.103 Safari/537.36");
    	    request.setHeader("ursId", "7144239B529794E56E7733F2CAA95A803FAEE9F695EC30637D2D80EC2DB30FCC64C384852ED312E1ECE6E2456B8528581");
    	    request.setHeader("ursToken", "3a8f8d342f56fcfac9a0f9d5168dc189e");
    	    
    	    //construct request body
    	    String JsonParam = "{\"skuId\":\"17752-68a3e5516d7a7dc21fbe0e7ee13bfc1c\",\"isHuanGou\":0,\"goodsId\":\"23430\",\"selected\":1,\"tempBuyAmount\":1,\"innerSource\":\"DETAIL\", \"goodsType\": 0, \"activitySchemeId\": 0}";
    	    
    	    StringEntity entity = new StringEntity(JsonParam);
    	    
    	    request.setEntity(entity);
    	    
    	    httpclient.execute(request, new FutureCallback<HttpResponse>() {

    	        public void completed(final HttpResponse response) {
    	            latch.countDown();
    	            try {
						String responseBody = EntityUtils.toString(response.getEntity(), "UTF-8");
						if (response.getStatusLine().getStatusCode() != 200 || !responseBody.contains("\"code\":0")) {
							System.out.println(request.getRequestLine() + "-> error:" +  responseBody);
						}else {
							System.out.println(request.getRequestLine() + "-> right:" +  responseBody);
						}
					} catch (ParseException e1) {
						// TODO Auto-generated catch block
						e1.printStackTrace();
					} catch (IOException e1) {
						// TODO Auto-generated catch block
						e1.printStackTrace();
					}
    	        }

    	        public void failed(final Exception ex) {
    	            latch.countDown();
    	            System.out.println(request.getRequestLine() + "->" + ex);
    	        }

    	        public void cancelled() {
    	            latch.countDown();
    	            System.out.println(request.getRequestLine() + " cancelled");
    	        }

    	    });
    	    latch.await();

    	    // In real world one most likely would also want to stream
    	    // request and response body content

    	} catch (UnsupportedEncodingException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		} finally {
    	    try {
				httpclient.close();
			} catch (IOException e) {
				// TODO Auto-generated catch block
				e.printStackTrace();
			}
    	}
    }
    
	/*
	 * method : Post request 
	 * Content-type : application/x-www-form-urlencoded
	 */
    public void doPostForFrom() throws InterruptedException, ExecutionException
    {
    	CloseableHttpAsyncClient httpclient = HttpAsyncClients.createDefault();
    	try {
    	    // Start the client
    	    httpclient.start();

    	    // One most likely would want to use a callback for operation result
    	    final CountDownLatch latch = new CountDownLatch(1);
    	    final HttpPost request = new HttpPost("https://www.kaola.com/cartAjax/modify.html");
    	    
    	    //construct request header
    	    request.setHeader("Content-type", "application/x-www-form-urlencoded");
    	    request.setHeader("User-Agent", "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2062.103 Safari/537.36");
    	    request.setHeader("Cookie", "NTES_SESS=GFPzQP2ESrC7x6FtQRFxFxm_BqVjxTHcbjouDqrgZcQQM6xQ.MSx83AOErZ6..CVNzkl9Gaa6WbbiMymWLDvzLd0_ABLGn3JwQvGjtvLgLCRinMEwLh7cr1VbLC6OwZKDExj4gzPCr.wEdx_8603JN38JXh9AnPs.9QQMwwExAVpfbso6tM7Ad.yIt5RcQheU");
    	    
    	    //construct request body
    	    List<NameValuePair> nvps = new ArrayList<NameValuePair>();
    	    nvps.add(new BasicNameValuePair("number", "1"));
    	    nvps.add(new BasicNameValuePair("goodsId", "23430"));
    	    nvps.add(new BasicNameValuePair("skuId", "17752-68a3e5516d7a7dc21fbe0e7ee13bfc1c"));
    	    nvps.add(new BasicNameValuePair("innerSource", "DETAIL"));
    	    
    	    request.setEntity(new UrlEncodedFormEntity(nvps));
    	    
    	    httpclient.execute(request, new FutureCallback<HttpResponse>() {

    	        public void completed(final HttpResponse response) {
    	            latch.countDown();
    	            try {
						String responseBody = EntityUtils.toString(response.getEntity(), "UTF-8");
						if (response.getStatusLine().getStatusCode() != 200 || !responseBody.contains("\"retcode\":200")) {
							System.out.println(request.getRequestLine() + "-> error:" +  responseBody);
						}else {
							System.out.println(request.getRequestLine() + "-> right:" +  responseBody);
						}
					} catch (ParseException e1) {
						// TODO Auto-generated catch block
						e1.printStackTrace();
					} catch (IOException e1) {
						// TODO Auto-generated catch block
						e1.printStackTrace();
					}
    	        }

    	        public void failed(final Exception ex) {
    	            latch.countDown();
    	            System.out.println(request.getRequestLine() + "->" + ex);
    	        }

    	        public void cancelled() {
    	            latch.countDown();
    	            System.out.println(request.getRequestLine() + " cancelled");
    	        }

    	    });
    	    latch.await();

    	    // In real world one most likely would also want to stream
    	    // request and response body content

    	} catch (UnsupportedEncodingException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		} finally {
    	    try {
				httpclient.close();
			} catch (IOException e) {
				// TODO Auto-generated catch block
				e.printStackTrace();
			}
    	}
    }
    
    public static void main(String[] args) throws InterruptedException, ExecutionException {
		AsycPerfPostClient asycPerfPostClient = new AsycPerfPostClient();
		asycPerfPostClient.doPostForFrom();
	}
}

进行异步压测

现在已经可以用HttpAsyncClient进行GET和POST请求的发送，由于都是单一请求但是还不能用于压测，下面增加了并发能力，可以设置并发数和执行次数。

示例
import java.io.IOException;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.ExecutionException;
import org.apache.http.HttpResponse;
import org.apache.http.client.config.RequestConfig;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.concurrent.FutureCallback;
import org.apache.http.impl.nio.client.CloseableHttpAsyncClient;
import org.apache.http.impl.nio.client.HttpAsyncClients;
import org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager;
import org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor;
import org.apache.http.nio.reactor.ConnectingIOReactor;

public class AsycSimplePerfTestClient implements Runnable {
	
	private static int requestRunTime = 0;
	private static CloseableHttpAsyncClient httpclient = null;
	
	public AsycSimplePerfTestClient(int runTime) {
		// TODO Auto-generated constructor stub
		requestRunTime = runTime;
	}
	
	static{
		try {
			RequestConfig requestConfig = RequestConfig.custom()
		            .setSocketTimeout(30000)
		            .setConnectTimeout(30000)
		            .build();
			
			ConnectingIOReactor ioReactor = new DefaultConnectingIOReactor();
	        PoolingNHttpClientConnectionManager connManager = new PoolingNHttpClientConnectionManager(ioReactor);
	        connManager.setMaxTotal(500);
	        connManager.setDefaultMaxPerRoute(500);
	        
		    httpclient = HttpAsyncClients.custom().setConnectionManager(connManager)
		        .setDefaultRequestConfig(requestConfig)
		        .build();
		} catch (Exception e) {
			// TODO: handle exception
		}
		
	}
	
	public void doGetRequest(int runTime) throws InterruptedException, ExecutionException, IOException {
		
	    try {
	        httpclient.start();
	        final HttpGet request = new HttpGet("http://m.kaola.com");
	        final CountDownLatch latch = new CountDownLatch(runTime);
	        for(int i = 0; i < runTime; i++){
		        httpclient.execute(request, new FutureCallback<HttpResponse>() {
	
		            public void completed(final HttpResponse response) {
		                latch.countDown();
//		                System.out.println(request.getRequestLine() + "->" + response.getStatusLine());
		            }
	
		            public void failed(final Exception ex) {
		                latch.countDown();
		                ex.printStackTrace();
		            }
	
		            public void cancelled() {
		                latch.countDown();
		                System.out.println(request.getRequestLine() + " cancelled");
		            }
	
		        });
	        }
	        latch.await();
	        System.out.println("Shutting down");
	    } finally {
	        httpclient.close();
	    }
	    System.out.println("Done");
	}
	
	public void run() {
		try {
			doGetRequest(requestRunTime);
		} catch (Exception e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		}
    }
	
	public static void main(String[] args) {
		int threadNum = 10;
		int runTime = 1000;
		for(int i = 0; i < threadNum; i++ ){
			new Thread(new AsycSimplePerfTestClient(runTime)).start();
		}
	}
}

总结

上面的实例仅仅是可以用来发异步的压测，但对于数据的统计还不够完善，需要规范日志打印，用于统计TPS，响应时间等各种性能指标，所以使用时要根据需求进行完善。



网易云产品免费体验馆，无套路试用，零成本体验云计算价值。  

本文来自网易实践者社区，经作者牛小宝授权发布


推荐博客