kotlin-kafka
#rationale
#goals
#adding-dependency
#example
Rationale
At the time of starting this repository I didn't find any bindings between Kafka SDK and Kotlin suspension, or KotlinX Coroutines Flow. These operators should be implemented low-level, so they can guarantee correct cancellation support, and high optimised runtimes.
Some important aspects of Kafka are tricky to implement with the "low-level" Kafka API, especially properly streaming records from Kafka and correctly committing them. Additional complexity is involved in this process, more details here.
To solve these problems a couple of projects in the JVM already exist:
There was no implementation for KotlinX Coroutines Flow, you can however quite easily use reactor-kafka with KotlinX Coroutines Reactor bindings.
This project implements the same strategies as reactor-kafka directly on top of KotlinX Coroutines to benefit from all their benefits, and to open the door to potentially becoming a Kotlin MPP library in the future.
Goals
Lean Core library built on top of Kotlin Std & KotlinX Coroutines
Extensions to easily operate over the Kafka SDK with KotlinX Coroutines and
suspend.Flow based operators, so you can easily compose KotlinX Flow based Kafka programs
Strong guarantees about committing record offsets, and performance optimisations in regard to re-balancing/partitioning.
example for testing Kafka with Test Containers in Kotlin.
Adding Dependency
Simply add the following dependency as implementation in the build.gradle dependencies` block.
dependencies {
implementation("io.github.nomisrev:kotlin-kafka:0.3.0")
}Example