Apache Kafka is a distributed publish-subscribe messaging system designed to replace traditional message brokers.
Originally created and developed by LinkedIn, then open sourced in 2011, Kafka is currently developed by the Apache Software Foundation to exploit new data infrastructures made possible by massively parallel commodity clusters.
Message brokers are a type of middleware that translates messages of one language to another, usually more commonly-accepted language. Message brokers can also be used to decouple data streams from processing and buffer unsent messages. Apache Kafka improves on traditional message brokers through advances in throughput, built-in partitioning, replication, latency and reliability.
Kafka can be used for a number of purposes: Messaging, real time website activity tracking, monitoring operational metrics of distributed applications, log aggregation from numerous servers, event sourcing where state changes in a database are logged and ordered, commit logs where distributed systems sync data and restoring data from failed systems.