Jakub Korab

Category: Integration

  • How to use librdkafka to build resilient systems

    Every good story has a beginning, and beginnings are arbitrary since one thing leads to another, so this story may as well start with a Kafka production incident. The root cause was innocuous, something that was easily fixed (a failed network switch), but since one thing leads to another, one by one, the dominoes fell…

  • New Book – Understanding Message Brokers

    If this blog has been unusually quiet over the last few months, it is with good reason – I wrote another book! Understanding Message Brokers has just been released for FREE download by O’Reilly. It is a 70 page concentrated brain dump of how to reason about messaging, based on a presentation I gave at…

  • A Milestone

    Last week I completed a consulting engagement with my 100th client since starting work in this field 5.5 years ago. Looking at these last few years in hindsight, it feels like an unfathomable point to be at. I am struck by how much I have learned and how my perspective has developed. It has been…

  • Performance Test Tool for Apache Kafka

    Towards the end of last year, I developed a performance test tool which is available at GitHub for testing the behaviour of Apache Kafka (0.9 so far). The tool was inspired by, and informed by some of the limitations of, the performance tooling in ActiveMQ, which I’m currently retrofitting with the same mechanisms. The kafka-perf-test…

  • Message Distribution and Topic Partitioning in Kafka

    When coming over to Apache Kafka from other messaging systems, there’s a conceptual hump that needs to first be crossed, and that is – what is a this topic thing that messages get sent to, and how does message distribution inside it work? Unlike regular brokers, Kafka only has one destination type – a topic…

  • How to reason about queue depth graphs

    Occasionally I come across something that seems really obvious to me, but actually isn’t obvious from an outsider’s perspective. A few weeks ago I spent some time with a client (and their client) in trying to understand the behaviour of a message-driven platform that made heavy use of ActiveMQ. Without going into too many details,…

  • Java DSLs for ActiveMQ config and testing

    As part of a larger project that I am working on, I needed to define numerous different ActiveMQ configurations inside non-trivial integration tests. Rather than instantiating brokers by hand using the internal ActiveMQ classes, or writing lots of similar-but-different XML configurations, I wrote a pair of Java DSLs to allow me to quickly spin up…

  • Digesting Microservices at muCon

    On Friday, I had the privilege of presenting at the very first Microservices conference – muCon. In my talk, Engineering Sanity into Microservices, I spoke about the technical issues surrounding state in distributed systems as a whole, how these become a bigger problem as the number of deployed services goes up, and a few suggested…

  • Running ActiveMQ Replicated LevelDB on VirtualBox

    I have wanted to spend some more time recently playing with Replicated LevelDB in ActiveMQ. Not wanting to hit a cloud environment such as EC2, I set up 3 VirtualBox nodes on my workstation running Fedora. The host system needs to be fairly beefy – you need at least a dedicated CPU core per image…

  • ActiveMQ Network Connectors Demystified

    When trying to get your head around how ActiveMQ’s networks of brokers work, it helps to have an understanding of the underlying mechanisms involved. I’ve always thought of it as the “Lego brick model” – once you understand the smaller pieces, you can reason about how they can be combined together. When a regular client…