Skip to main content

Implement user based throttling with WSO2 API Manager

WSO2 API Manager throttling implementation is done in a manner so that API designers have the full flexibility to throttle API consumers at all levels. It supports throttling at 
  • API level
  • Resource level
  • Application level
  • Subscription level
  • Oauth2 token level
with out of the box throttling tiers and configurations. Users can manage throttling at above mentioned levels using the OOTB functionalities available in the product. The below 2 figures explains how the throttling is applicable across different levels and how they are executed during the runtime.
WSO2 API Manager throttling levels and applicability
The different throttling levels which are configured from the API manager will apply during the runtime as depicted in the below figure.
WSO2 API Manager throttling execution flow
In this article, I’m going to discuss how to implement custom throttling rules to throttle API requests based on users. To do this, users needs to log in to the admin interface of the WSO2 API manager runtime which is deployed in the following URL.

Once you log in, you can go to the Throttling Policies tab and navigate to Custom rules section where you can add a new custom rule. These custom rules needs to be implemented in Siddhi query language which is somewhat similar to SQL per say. The below section is extracted from the WSO2 API Manager documentation.

Custom throttling allows system administrators to define dynamic rules for specific use cases, which are applied globally across all tenants. When a custom throttling policy is created, it is possible to define any policy you like. The Traffic Manager acts as the global throttling engine and is based on the same technology as WSO2 Complex Event Processor (CEP), which uses the Siddhi query language. Users are therefore able to create their own custom throttling policies by writing custom Siddhi queries. The specific combination of attributes being checked in the policy need to be defined as the key (also called the key template). The key template usually includes a predefined format and a set of predefined parameters. It can contain a combination of allowed keys separated by a colon (:), where each key must start with the prefix $. The following keys can be used to create custom throttling policies:

resourceKey, userId, apiContext, apiVersion, appTenant, apiTenant, appId

Let’s write a custom policy to restrict any user from sending more than 5 requests to all the APIs running on the gateway. 

FROM RequestStream
SELECT userId, userId as throttleKey
INSERT INTO EligibilityStream;
FROM EligibilityStream#throttler:timeBatch(1 min)
SELECT throttleKey, (count(userId) >= 5) as isThrottled, expiryTimeStamp group by throttleKey
INSERT ALL EVENTS into ResultStream;

In the above script, we select the userId as throttleKey and check the number of requests received to the gateway within a time span of 1 minute and make it throttled if the request count is greater than or equal to 5. 
Let’s write another script to restrict only a given user (“admin”) with a limit of 5 requests per minute.

FROM RequestStream
SELECT userId, ( userId == ‘admin@carbon.super’ ) AS isEligible , str:concat(‘admin@carbon.super’,’’) as throttleKey
INSERT INTO EligibilityStream;
FROM EligibilityStream[isEligible==true]#throttler:timeBatch(1 min)
SELECT throttleKey, (count(userId) >= 5) as isThrottled, expiryTimeStamp group by throttleKey
INSERT ALL EVENTS into ResultStream;

In the above script, we check the username to be “admin” and then apply the throttling limit.
If we want to restrict all the users other than “admin” from sending more than 5 requests per minute, we can implement it like below.

FROM RequestStream
SELECT userId, ( userId != ‘admin@carbon.super’ ) AS isEligible, userId as throttleKey
INSERT INTO EligibilityStream;
FROM EligibilityStream[isEligible==true]#throttler:timeBatch(1 min)
SELECT throttleKey, (count(userId) >= 5) as isThrottled, expiryTimeStamp group by throttleKey
INSERT ALL EVENTS into ResultStream;

Above script will block all the users other than “admin” user from sending more than 5 requests per minute.
Happy throttling with WSO2 API Manager !

Comments

  1. Thank you for this nice article.

    I have a question on this subject : is it possible to make a throttling by per caller IP address ?

    The WSO2 documentation describes how to throttle one IP address (or a range of IP address). My need is to throttle any IP address without specifying one in particular.

    Example :
    ---------

    API context : /mycredit with no authentication (so no oauth token)

    Imagine 2 callers :
    - caller 1 : IP address : 215.08.1.10
    - caller 2 : IP address : 215.08.1.11

    Throttle expected : 6 req / min / IP

    If "caller 1" make more than 6 calls to this API in the minute, their futures calls will be throttled, but "caller 2" can make their 6 calls without be throttled.

    Is it possible ? Do I need to write a Siddhi query ?

    Thank you in advance.

    ReplyDelete

Post a Comment

Popular posts from this blog

How to configure timeouts in WSO2 ESB to get rid of client timeout errors

WSO2 ESB has defined some configuration parameters which controls the timeout of a particular request which is going out of ESB. In a particular  scneario, your client sends a request to ESB, and then ESB sends a request to another endpoint to serve the request. CLIENT->WSO2 ESB->BACKEND The reason for clients getting timeout is that ESB timeout is larger than client's timeout. This can be solved by either increasing the timeout at client side or by decreasing the timeout in ESB side. In any of the case, you can control the timeout in ESB using the below properties. 1) Global timeout defined in synapse.properties (ESB_HOME\repository\conf\) file. This will decide the maximum time that a callback is waiting in the ESB for a response for a particular request. If ESB does not get any response from Back End, it will drop the message and clears out the call back. This is a global level parameter which affects all the endpoints configured in ESB. synapse.global_timeout_inte

Understanding Threads created in WSO2 ESB

WSO2 ESB is an asynchronous high performing messaging engine which uses Java NIO technology for its internal implementations. You can find more information about the implementation details about the WSO2 ESB’s high performing http transport known as Pass-Through Transport (PTT) from the links given below. [1] http://soatutorials.blogspot.com/2015/05/understanding-wso2-esb-pass-through.html [2] http://wso2.com/library/articles/2013/12/demystifying-wso2-esb-pass-through-transport-part-i/ From this tutorial, I am going to discuss about various threads created when you start the ESB and start processing requests with that. This would help you to troubleshoot critical ESB server issues with the usage of a thread dump. You can monitor the threads created by using a monitoring tool like Jconsole or java mission control (java 1.7.40 upwards). Given below is a list of important threads and their stack traces from an active ESB server.  PassThroughHTTPSSender ( 1 Thread )

How puppet works in your IT infrstructure

What is Puppet? Puppet is IT automation software that helps system administrators manage infrastructure throughout its lifecycle, from provisioning and configuration to orchestration and reporting. Using Puppet, you can easily automate repetitive tasks, quickly deploy critical applications, and proactively manage change, scaling from 10s of servers to 1000s, on-premise or in the cloud. How the puppet works? It works like this..Puppet agent is a daemon that runs on all the client servers(the servers where you require some configuration, or the servers which are going to be managed using puppet.) All the clients which are to be managed will have puppet agent installed on them, and are called nodes in puppet. Puppet Master: This machine contains all the configuration for different hosts. Puppet master will run as a daemon on this master server. Puppet Agent: This is the daemon that will run on all the servers, which are to be managed using p