WSO2 ESB tuning performance with threads

I have written several blog posts explaining the internal behavior of the ESB and the threads created inside ESB. With this post, I am talking about the effect of threads in the WSO2 ESB and how to tune up threads for optimal performance. You can refer [1] and [2] to understand the threads created within the ESB.

[1] http://soatutorials.blogspot.com/2015/05/understanding-threads-created-in-wso2.html

[2] http://wso2.com/library/articles/2012/03/importance-performance-wso2-esb-handles-nonobvious/

Within this blog post, I am discussing about the "worker threads" which are used for processing the data within the WSO2 ESB. There are 2 types of worker threads created when you start sending the requests to the server

1) Server Worker/Client Worker Threads
2) Mediator Worker (Synapse-Worker) Threads


Server Worker/Client Worker Threads

These set of threads will be used to process all the requests/responses coming to the ESB server. ServerWorker Threads will be used to process the request path and Client Worker threads will be used to process the responses.


Mediator Worker (Synapse-Worker) Threads

These threads will only be started if you have iterate/clone mediators in your ESB mediation flow. These threads will be used for processing iterate/clone operations in separate threads for parallel processing of a single request.


WSO2 ESB uses the java ThreadPoolExecutor implementation for spawning new threads for processing requests. Both the above mentioned thread categories will be using this implementation underneath.

The java.util.concurrent.ThreadPoolExecutor is an implementation of the ExecutorService interface. The ThreadPoolExecutor executes the given task (Callable or Runnable) using one of its internally pooled threads.

The thread pool contained inside the ThreadPoolExecutor can contain a varying amount of threads. The number of threads in the pool is determined by these variables:

• corePoolSize
• maximumPoolSize

If less than corePoolSize threads are created in the the thread pool when a task is delegated to the thread pool, then a new thread is created, even if idle threads exist in the pool.

If the internal queue of tasks is full, and corePoolSize threads or more are running, but less than maximumPoolSize threads are running, then a new thread is created to execute the task.

These parameter of the thread pools can be configured in the following configuration files in the WSO2 ESB

ServerWorker/ClientWorker Thread pool (ESB_HOME/repository/conf/passthru-http.properties)

worker_pool_size_core=400

worker_pool_size_max=500

#worker_thread_keepalive_sec=60

#worker_pool_queue_length=-1

The default values given in the standalone ESB pack would be enough for most of the scenarios. But you need to do some performance testing with a similar load and tune these values accordingly. In the above configuration, there are 2 commented out parameters.

worker_thread_keepalive_sec - If the pool currently has more than corePoolSize threads, excess threads will be terminated if they have been idle for more than the keepAliveTime. This provides a means of reducing resource consumption when the pool is not being actively used. If the pool becomes more active later, new threads will be constructed.

worker_pool_queue_length - This is the task queue length to which new tasks will be delegated by the server when there are new data to be processed. The length of this queue is -1 (infinite) by default. This is one of the most important parameter when you are tuning the server for capacity. When you have infinite length queue, it will never reject any request. But the drawback with this value is that, if there are less number of processing threads and you have a peak load, the server can easily go into OOM status since the task queue will hold all the requests coming in to the server. You need to decide on a considerable value for this queue length rather than keeping this value as -1. If you have a limited value for this queue length, it will reject some requests in a high load scenario. But the server will not crash (OOM). This would be better rather than loosing all the requests. Another disadvantage of having -1 as the queue length would be that server will never create the max number of threads but only create core number of threads in any kind of load. 

MediatorWorker (SynapseWorker) Threads (ESB_HOME/repository/conf/synapse.properties)

synapse.threads.core = 20

synapse.threads.max = 100

#synapse.threads.keepalive = 5

#synapse.threads.qlen = 10

The same theory which I have described above can be applied when tuning this thread pool. Apart from that, It is always better to have a matching core value with the ServerWorker threads if you have used iterate/clone mediators heavily in your mediation flow. Considerable value for these parameters would like below.

synapse.threads.core = 100

synapse.threads.max = 200

I hope this would help you when tuning WSO2 ESB server for your production deployments.