New Book Review: "Designing Event-Driven Systems"
New book review for Designing Event-Driven Systems: Concepts and Patterns for Streaming Services with Apache Kafka, by Ben Stopford, O'Reilly, 2018:
Sam Newman, the author of "Building Microservices: Designing Fine-Grained Systems" (see my review), mentions Steve Yegge's "Platform Rant" in the foreward to this text. This rant outlined the mandate from Amazon CEO Jeff Bezos regarding how teams should work together and how they should design systems, making extensive use of communication via services interfaces in the process. Newman comments that he has been pushing use of well-defined interfaces that should guide when and how data is to be accessed and manipulated, and while people increasingly get this idea, challenges still remain and he he has been exploring the concept of "turning the database inside out" (a phrase that Stopford later correctly explains was coined by Martin Kleppman) with streams the last couple years.
This freely available 150-page text is broken down into 5 parts: (1) "Setting the Stage" which walks the reader through the origins of streaming and an overview of Apache Kafka, (2) "Designing Event-Driven Systems" which discusses events, processing events, and stateful patterns, (3) "Rethinking Architecture at Company Scales" which talks about sharing data and services across an organization, and the concepts of "event streams as a shared source of truth" and "lean data", (4) Consistency, Concurrency, and Evolution" which discusses consistency and concurrency in event-driven systems, transactions, and schema and data evolution, and (5) "Implementing Streaming Services with Kafka" which provides a brief example that uses Kafka Streams and KSQL, followed by a more involved example provided by Confluent in GitHub which revolves around a simple order processing workflow that validates and processes orders in response to requests.
While many new to Kafka will find value in the earlier chapters of this book, I especially appreciated chapter 5 ("Events: A Basis for Collaboration"), chapter 6 ("Processing Events with Stateful Functions"), chapter 7 ("Event Sourcing, CQRS, and Other Stateful Patterns"), chapter 8 ("Sharing Data and Services Across an Organization"), and chapter 12 ("Transactions, but Not as We Know Them"). Some overlap exists here with other texts, such as "Making Sense of Stream Processing: The Philosophy Behind Apache Kafka and Scalable Stream Data Platforms" (see my review), by Martin Kleppman, but I recommend Stopford's work as a starting point (partially because Kleppman's work is from 2016), although keep in mind that Kleppman also came out with "Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems" (see my review) the following year.