Hackforge Academy

Explanation

Apache Kafka is a distributed event streaming and message broker platform used for handling real-time data feeds. It allows applications to publish, store, process, and consume large amounts of messages efficiently.

Code:

Producer --> Kafka Topic --> Consumer

Explanation

Kafka works using Topics. Producers send messages to topics, and Consumers read messages from those topics. Kafka stores messages durably and can process millions of events per second.

Code:

Order Service --> orders-topic --> 
Payment Service

Explanation

Main components in Kafka are: Producer, Consumer, Broker, Topic, Partition, and Zookeeper/KRaft.

Code:

Producer = Sends messages
Consumer = Reads messages
Broker = Kafka server
Topic = Message category
Partition = Splits topic for scalability

Explanation

Uses of Kafka: Real-time analytics, log aggregation, event-driven microservices, activity tracking, financial transactions, IoT data streaming, and notification systems.

Code:

Website Clicks --> Kafka -->
 Analytics Dashboard

Explanation

Kafka is highly scalable, fault-tolerant, and supports distributed processing across multiple servers.

Code:

Multiple Producers --> Kafka Cluster 
--> Multiple Consumers

Explanation

Simple Python Kafka Producer example using kafka-python library.

Code:

from kafka import KafkaProducer

producer = KafkaProducer(
bootstrap_servers='localhost:9092')

producer.send('test-topic', b'Hello Kafka')
producer.flush()

print('Message Sent')

Explanation

Simple Python Kafka Consumer example.

Code:

from kafka import KafkaConsumer

consumer = KafkaConsumer('test-topic',
 bootstrap_servers='localhost:9092')

for message in consumer:
    print(message.value.decode())