GlobalKTable vs KTable in Kafka Streams

KTable vs GlobalKTable

Kafka Streams presents two options for materialized views in the forms of GlobalKTable vs KTables.  We will describe the meaning of “materialized views” in a moment, but for now, let’s just agree there are pros and cons to GlobalKTable vs KTables. Need to learn more about Kafka Streams in Java? Here’s a pretty good option […]

Kafka Streams – Transformations Examples

Kafka Streams Transformations

Kafka Streams Transformations provide the ability to perform actions on Kafka Streams such as filtering and updating values in the stream.  Kafka Stream’s transformations contain operations such as `filter`, `map`, `flatMap`, etc. and have similarities to functional combinators found in languages such as Scala.  And, if you are coming from Spark, you will also notice […]

Kafka Streams Joins Examples

Kafka Join Examples

Performing Kafka Streams Joins presents interesting design options when implementing streaming processor architecture patterns. There are numerous applicable scenarios, but let’s consider an application might need to access multiple database tables or REST APIs in order to enrich a topic’s event record with context information. For example, perhaps we could augment records in a topic with sensor […]

Kafka Streams Testing with Scala Part 1

Kafka Streams Testing in Scala Part 1

After experimenting with Kafka Streams with Scala, I started to wonder how one goes about Kafka Streams testing in Java or Scala.  How does one create and run automated tests for Kafka Streams applications?  How does it compare to Spark Streaming testing? In this tutorial, I’ll describe what I’ve learned so far.  Also, if you […]

Kafka Streams Tutorial with Scala for Beginners Example

Kafka Streams Tutorial with Scala for Beginners

If you’re new to Kafka Streams, here’s a Kafka Streams Tutorial with Scala tutorial which may help jumpstart your efforts.  My plan is to keep updating the sample project, so let me know if you would like to see anything in particular with Kafka Streams with Scala.  In this example, the intention is to 1) provide an SBT project you […]