Skip to main content

Understanding Apache Kafka architecture

Apache Kafka has evolved as the defacto standard for building reliable event based systems with ultra high volumes. The unique, yet simple architecture has made Kafka an easy to use component which integrate well with existing enterprise architectures. At a very high level, it is a messaging platform which decouples the message producers from the message consumers while providing the reliability of message delivery to consumers at scale. Kafka has message producers which send messages (events) to kafka which kafka stores in a entity called a topic which will deliver the messages to one or more consumers in a reliable manner. There can be different types of producers and consumers depending on the use case.

 In the below post, I'm explaining how kafka architecture works and integrate with other systems.

https://medium.com/@chanakaudaya/understanding-apache-kafka-the-messaging-technology-for-modern-applications-4fbc18f220d3


Labels:

Comments

  1. OnlinearchitecturalservicesFebruary 25, 2020 at 1:50 AM

    We are providing Online Architectural Services, This is the most trustworthy site for planning application and building regulation drawings Architectural services,

    ReplyDelete

Post a Comment

Popular posts from this blog

How to setup an WSO2 API manager distributed setup with a clustered gateway with WSO2 ELB

In this blog post I am going to describe about how to configure a WSO2 API Manager in a distributed setup with a clustered gateway with WSO2 ELB and the WSO2 G-REG for a distributed deployment in your production environment. Before continuing with this post, you need to download the above mentioned products from the WSO2 website. WSO2 APIM - http://wso2.com/products/api-manager/ WSO2 ELB - http://wso2.com/products/elastic-load-balancer/ Understanding the API Manager architecture API Manager uses the following four main components: Publisher Creates and publishes APIs Store Provides a user interface to search, select, and subscribe to APIs Key Manager Used for authentication, security, and key-related operations Gateway Responsible for securing, protecting, managing, and scaling API calls Here is the deployment diagram that we are going to configure. In this setup, you have 5 APIM nodes with 2 gateway...
2 comments
Read more

WSO2 ESB tuning performance with threads

I have written several blog posts explaining the internal behavior of the ESB and the threads created inside ESB. With this post, I am talking about the effect of threads in the WSO2 ESB and how to tune up threads for optimal performance. You can refer [1] and [2] to understand the threads created within the ESB. [1] http://soatutorials.blogspot.com/2015/05/understanding-threads-created-in-wso2.html [2] http://wso2.com/library/articles/2012/03/importance-performance-wso2-esb-handles-nonobvious/ Within this blog post, I am discussing about the "worker threads" which are used for processing the data within the WSO2 ESB. There are 2 types of worker threads created when you start sending the requests to the server 1) Server Worker/Client Worker Threads 2) Mediator Worker (Synapse-Worker) Threads Server Worker/Client Worker Threads These set of threads will be used to process all the requests/responses coming to the ESB server. ServerWorker Threads will be used to pr...
5 comments
Read more

Understanding Threads created in WSO2 ESB

WSO2 ESB is an asynchronous high performing messaging engine which uses Java NIO technology for its internal implementations. You can find more information about the implementation details about the WSO2 ESB’s high performing http transport known as Pass-Through Transport (PTT) from the links given below. [1] http://soatutorials.blogspot.com/2015/05/understanding-wso2-esb-pass-through.html [2] http://wso2.com/library/articles/2013/12/demystifying-wso2-esb-pass-through-transport-part-i/ From this tutorial, I am going to discuss about various threads created when you start the ESB and start processing requests with that. This would help you to troubleshoot critical ESB server issues with the usage of a thread dump. You can monitor the threads created by using a monitoring tool like Jconsole or java mission control (java 1.7.40 upwards). Given below is a list of important threads and their stack traces from an active ESB server.  PassThroughHTTPSSender ( 1 Thread ...
2 comments
Read more