XACML PDP Performance Testing with Thrift

Last time using this blog post,  We discussed on load testing the XACML PDP using Jmeter.  We uses the HTTPS transport for calling the Web Service API that has been exposed by the PDP.  With Identity Server, we can also use Thrift protocal to communicate with PDP.  It is said that thrift is more faster than the HTTP.  Therefore we hope that we can get more performance and less response time by using thrift protocol with Identity Server.

Still i could not found any open source tools that can be used to do load testing with thrift.  Jmeter also does not support it.  But there are some Jmeter plugins for thrift that you can try out.  However,  i wrote some load test tool to try out the PDP with thrift protocol.  You can find the source of this tool from here.

This tool can use to load test both Thrift and HTTP  as I developed this tool mainly to get some comparison with Thrift and HTTP.  You can run this tool as a PEP for load testing the PDP with both HTTP and Thrift.  Then you can get some idea how throughout and response time have been changed when comparing both Thrift and HTTP.

Please Note :  This load test tool is a simple java code. It can calculate throughput and response time in simple manner. Please refer following details.  There may be things that can use to improve this tool.  You can go through source code and you can find out improvements or any issues.  We are really appreciate your feedback.  Please do not compare the result of this tool with proper load testing tool. However simplicity of this tool would not be an issue as intendant of this tool to compare the Thrift and HTTP performance.

Calculating Throughput  :

Before the load test is started the time stamp is calculated, lets say  T1.

After load test is finished,  time stamp is calculated, lets say T2

Total time spend for load test  =  T2 – T1

Number of total message sent  =  Number Of Threads  *  Number Of Request  for Thread

Throughput   =  Total Messages / (T2 – T1)

Time spent for Message =  1 / Throughput  =  (T2 – T1)  / Total Messages

Calculating Average Response time :

Before message is sent from specific thread, the time stamp is calculated, lets say t1.

After message is received, the time stamp is calculated, lets say  t2.

Response time for one message in specific thread  =  t2 – t1

Calculated response time for all messages in specific thread (th1) = (t2 – t1) + …. + … +…

Calculated response time for all messages = th1 +th2 + ……….  + thn

Average response time =  Response time for all messages /  Total Messages

Please go following steps to start your load testing.

Getting Start

According to my previous post, you need to consider following four points.  More details about 1, 2 and 3 points can be found in my previous post.

1.  Generating large number of different XACML policies.
2. Generating different XACML requests.
3. Uploading XACML policies to PDP policy store in Bulk.

4. Implementing load test tool that supports thrift protocol.  You can find the source of this tool from here.  You can download the tool and can run the tool using the command line script. All dependency  jar files can be found in lib directory.  Configuration properties can be found at  resources/config.properties file.  You can go though the README file for more details.

Step to Setup

Step1. Setup Identity Server

Configure MySQL database as WSO2 Identity Server’s data store.  You need to configure master-datasources.xml file which can be found at  <IS_HOME>/repository/conf/datasources. MySQL JDBC driver can be put in to <IS_HOME>/repository/components/lib directory.
Increase memory foot print.  You can configure memory using wso2server script file which can be found at <IS_HOME>/bin directory.

Step2. Create XACML policies and XACML requests using tool. Sample Requests (request.txt) file can be found inside resource directory

Please note : when creating the new  multiple requests from the request generation tool, we need to configure the requestWrapper  as following  “requestWrapper=request”

Step3. Upload XACML policies in to Identity Server.

Step4. Run the load test tool by using run.sh or run.bat script file.  Please refer more details from README file.

Now you can run the load test using HTTP and Thrift. You would be able to do comparison of throughput and response time.