Skip to main content

Understanding Apache Kafka architecture

Apache Kafka has evolved as the defacto standard for building reliable event based systems with ultra high volumes. The unique, yet simple architecture has made Kafka an easy to use component which integrate well with existing enterprise architectures. At a very high level, it is a messaging platform which decouples the message producers from the message consumers while providing the reliability of message delivery to consumers at scale. Kafka has message producers which send messages (events) to kafka which kafka stores in a entity called a topic which will deliver the messages to one or more consumers in a reliable manner. There can be different types of producers and consumers depending on the use case.

 In the below post, I'm explaining how kafka architecture works and integrate with other systems.

https://medium.com/@chanakaudaya/understanding-apache-kafka-the-messaging-technology-for-modern-applications-4fbc18f220d3


Labels:

Comments

  1. OnlinearchitecturalservicesFebruary 25, 2020 at 1:50 AM

    We are providing Online Architectural Services, This is the most trustworthy site for planning application and building regulation drawings Architectural services,

    ReplyDelete

Post a Comment

Popular posts from this blog

How to setup an WSO2 API manager distributed setup with a clustered gateway with WSO2 ELB

In this blog post I am going to describe about how to configure a WSO2 API Manager in a distributed setup with a clustered gateway with WSO2 ELB and the WSO2 G-REG for a distributed deployment in your production environment. Before continuing with this post, you need to download the above mentioned products from the WSO2 website. WSO2 APIM - http://wso2.com/products/api-manager/ WSO2 ELB - http://wso2.com/products/elastic-load-balancer/ Understanding the API Manager architecture API Manager uses the following four main components: Publisher Creates and publishes APIs Store Provides a user interface to search, select, and subscribe to APIs Key Manager Used for authentication, security, and key-related operations Gateway Responsible for securing, protecting, managing, and scaling API calls Here is the deployment diagram that we are going to configure. In this setup, you have 5 APIM nodes with 2 gateway...
2 comments
Read more

WSO2 ESB tuning performance with threads

I have written several blog posts explaining the internal behavior of the ESB and the threads created inside ESB. With this post, I am talking about the effect of threads in the WSO2 ESB and how to tune up threads for optimal performance. You can refer [1] and [2] to understand the threads created within the ESB. [1] http://soatutorials.blogspot.com/2015/05/understanding-threads-created-in-wso2.html [2] http://wso2.com/library/articles/2012/03/importance-performance-wso2-esb-handles-nonobvious/ Within this blog post, I am discussing about the "worker threads" which are used for processing the data within the WSO2 ESB. There are 2 types of worker threads created when you start sending the requests to the server 1) Server Worker/Client Worker Threads 2) Mediator Worker (Synapse-Worker) Threads Server Worker/Client Worker Threads These set of threads will be used to process all the requests/responses coming to the ESB server. ServerWorker Threads will be used to pr...
5 comments
Read more

Performance Tuning WSO2 ESB with a practical example

WSO2 ESB is arguably the highest performing open source ESB available in the industry. With the default settings, it will provide a really good performance OOTB. You can find the official performance tuning guide in the below link. https://docs.wso2.com/display/ESB481/Performance+Tuning Above document includes lot of parameters and how to change them for best performance. Even though it provides some recommended values for the performance tuning, that is not straightforward. The values you put in the configuration files will highly reliant on your use case. Idea of this blog post is to provide a practical example of tuning these parameters with a sample use case. For the performance testing, I am using a simple proxy service which iterate through set of xml elements and send request to a sample server and aggregate the responses back to the client. <?xml version="1.0" encoding="UTF-8"?> <proxy xmlns="http://ws.apache.org/ns/synapse"     ...
3 comments
Read more