Category: Kafka
-
How to use librdkafka to build resilient systems
Every good story has a beginning, and beginnings are arbitrary since one thing leads to another, so this story may as well start with a Kafka production incident. The root cause was innocuous, something that was easily fixed (a failed network switch), but since one thing leads to another, one by one, the dominoes fell…
-
New Book – Understanding Message Brokers
If this blog has been unusually quiet over the last few months, it is with good reason – I wrote another book! Understanding Message Brokers has just been released for FREE download by O’Reilly. It is a 70 page concentrated brain dump of how to reason about messaging, based on a presentation I gave at…
-
Performance Test Tool for Apache Kafka
Towards the end of last year, I developed a performance test tool which is available at GitHub for testing the behaviour of Apache Kafka (0.9 so far). The tool was inspired by, and informed by some of the limitations of, the performance tooling in ActiveMQ, which I’m currently retrofitting with the same mechanisms. The kafka-perf-test…
-
Message Distribution and Topic Partitioning in Kafka
When coming over to Apache Kafka from other messaging systems, there’s a conceptual hump that needs to first be crossed, and that is – what is a this topic thing that messages get sent to, and how does message distribution inside it work? Unlike regular brokers, Kafka only has one destination type – a topic…