Skip to main content

Understanding Apache Kafka architecture

Apache Kafka has evolved as the defacto standard for building reliable event based systems with ultra high volumes. The unique, yet simple architecture has made Kafka an easy to use component which integrate well with existing enterprise architectures. At a very high level, it is a messaging platform which decouples the message producers from the message consumers while providing the reliability of message delivery to consumers at scale. Kafka has message producers which send messages (events) to kafka which kafka stores in a entity called a topic which will deliver the messages to one or more consumers in a reliable manner. There can be different types of producers and consumers depending on the use case.

 In the below post, I'm explaining how kafka architecture works and integrate with other systems.

https://medium.com/@chanakaudaya/understanding-apache-kafka-the-messaging-technology-for-modern-applications-4fbc18f220d3


Labels:

Comments

  1. OnlinearchitecturalservicesFebruary 25, 2020 at 1:50 AM

    We are providing Online Architectural Services, This is the most trustworthy site for planning application and building regulation drawings Architectural services,

    ReplyDelete

Post a Comment

Popular posts from this blog

How to setup an WSO2 API manager distributed setup with a clustered gateway with WSO2 ELB

In this blog post I am going to describe about how to configure a WSO2 API Manager in a distributed setup with a clustered gateway with WSO2 ELB and the WSO2 G-REG for a distributed deployment in your production environment. Before continuing with this post, you need to download the above mentioned products from the WSO2 website. WSO2 APIM - http://wso2.com/products/api-manager/ WSO2 ELB - http://wso2.com/products/elastic-load-balancer/ Understanding the API Manager architecture API Manager uses the following four main components: Publisher Creates and publishes APIs Store Provides a user interface to search, select, and subscribe to APIs Key Manager Used for authentication, security, and key-related operations Gateway Responsible for securing, protecting, managing, and scaling API calls Here is the deployment diagram that we are going to configure. In this setup, you have 5 APIM nodes with 2 gateway...
2 comments
Read more

WSO2 ESB tuning performance with threads

I have written several blog posts explaining the internal behavior of the ESB and the threads created inside ESB. With this post, I am talking about the effect of threads in the WSO2 ESB and how to tune up threads for optimal performance. You can refer [1] and [2] to understand the threads created within the ESB. [1] http://soatutorials.blogspot.com/2015/05/understanding-threads-created-in-wso2.html [2] http://wso2.com/library/articles/2012/03/importance-performance-wso2-esb-handles-nonobvious/ Within this blog post, I am discussing about the "worker threads" which are used for processing the data within the WSO2 ESB. There are 2 types of worker threads created when you start sending the requests to the server 1) Server Worker/Client Worker Threads 2) Mediator Worker (Synapse-Worker) Threads Server Worker/Client Worker Threads These set of threads will be used to process all the requests/responses coming to the ESB server. ServerWorker Threads will be used to pr...
4 comments
Read more

How puppet works in your IT infrstructure

What is Puppet? Puppet is IT automation software that helps system administrators manage infrastructure throughout its lifecycle, from provisioning and configuration to orchestration and reporting. Using Puppet, you can easily automate repetitive tasks, quickly deploy critical applications, and proactively manage change, scaling from 10s of servers to 1000s, on-premise or in the cloud. How the puppet works? It works like this..Puppet agent is a daemon that runs on all the client servers(the servers where you require some configuration, or the servers which are going to be managed using puppet.) All the clients which are to be managed will have puppet agent installed on them, and are called nodes in puppet. Puppet Master: This machine contains all the configuration for different hosts. Puppet master will run as a daemon on this master server. Puppet Agent: This is the daemon that will run on all the servers, which are to be managed using p...
7 comments
Read more